Natália(Natália Mendonça)
Minor nit: following strict rules without weighing the costs and benefits each time could be motivated by rule utilitarianism, not only by deontology. It could also be motivated by act utilitarianism, if you deem that weighing the costs and benefits every single time would not be worth it. (Though I don’t think EA veganism is often motivated by act utilitarianism).
This made me wonder about a few things:
How responsible is CSET for this? CSET is the most highly funded longtermist-ish org, as far as I can tell from checking openbook.fyi (I could be wrong), so I’ve been trying to understand them better, since I don’t hear much about them on LW or the EA Forum. I suspected they were having a lot of impact “behind the scenes” (from my perspective), and maybe this is a reflection of that?
Aaron Bergman said on Twitter that for him, “the ex ante probability of something at least this good by the US federal government relative to AI progress, from the perspective of 5 years ago was ~1%[.] Ie this seems 99th-percentile-in-2018 good to me”, and many people seemed to agree. Stefan Schubert then said that “if people think the policy response is “99th-percentile-in-2018″, then that suggests their models have been seriously wrong.” I was wondering, do people here agree with Aaron that this EO appeared unlikely back then, and, if so, what do you think the correct takeaway from the existence of this EO is?
Thanks for this information. When I did this, it was because I was misunderstanding someone’s position, and only realized it later. I’ll refrain from deleting comments excessively in the future and will use the “retract” feature when something like this happens again.
Hi, that was an oversight, I’ve edited it now.
See this comment.
Several people cited the AHS-2 as a pseudo-RCT that supported veganism (EDIT 2023-10-03: as superior to low meat omnivorism).
[…]
My complaint is that the study was presented as strong evidence in one direction, when it’s both very weak and, if you treat it as strong, points in a different direction than reported
[Note: this comment was edited heavily after people replied to it.]
I think this is wrong in a few ways:
1. None of the comments referred to “low meat omnivorism.” AHS-2 had a “semi-vegetarian” category composed of people who eat meat in low quantities, but none of the comments referred to it
2. The study indeed found that vegans had lower mortality than omnivores (the hazard ratio was 0.85 (95% CI, 0.73–1.01)); your post makes it sound like it’s the opposite by saying that the association “points in a different direction than reported.” I think what you mean to say is that vegan diets were not the best option if we look only at the point estimates of the study, because pescatariansim was very slightly better. But the confidence intervals were wide and overlapped too much for us to say with confidence which diet was better.
Here’s a hypothetical scenario. Suppose a hypertension medication trial finds that Presotex monotherapy reduced stroke incidence by 34%. The trial also finds that Systovar monotherapy decreased the incidence of stroke by 40%, though the confidence intervals were very similar to Presotex’s.
Now suppose Bob learns this information and tells Chloe: “Alice said something misleading about Presotex. She said that a trial supported Prestotex monotherapy for stroke prevention, but the evidence pointed in a different direction than she reported.”
I think Chloe would likely come out with the wrong impression about Presotex.
3. My comment, which you refer to in this section, didn’t describe the AHS-2 as having RCT-like characteristics. I just thought it was a good observational study. A person I quoted in my comment (Froolow) was originally the person who mistakenly described it as a quasi-RCT (in another post I had not read at the time), but Froolow’s comment that I quoted didn’t describe it as such, and I thought it made sense without that assumption.
4. Froolow’s comment and mine were both careful to notice that the study findings are weak and consistent with veganism having no effect on lifespan. I don’t see how they presented it as strong evidence.
[Note: I deleted a previous comment making those points and am re-posting a reworded version.]
I think the original post was a bit confusing in what it claimed the Faunalytics study was useful for.
For example, the section
The ideal study is a longitudinal RCT where diet is randomly assigned, cost (across all dimensions, not just money) is held constant, and participants are studied over multiple years to track cumulative effects. I assume that doesn’t exist, but the closer we can get the better.
I’ve spent several hours looking for good studies on vegan nutrition, of which the only one that was even passable was the Faunalytics study.[...]
A non-exhaustive list of common flaws:
Studies rarely control for supplements. [...]
makes it sound like the author is interested on the effects of vegan diets on health, both with and without supplementation, and that they’re claiming that the Faunalytics study is the best study we have to answer that question. This is what I and Matthew would strongly disagree with.
This post uses the Faunalytics study in a different (and IMO more reasonable) way, to show which proportion of veg*ans report negative health effects and quit in practice. This is a different question because it can loosely track how much veg*ans follow dietary guidelines. For example, vitamin B12 deficiency should affect close to 100% of vegans who don’t supplement and have been vegan for long enough, and, on the other side of the spectrum, it likely affects close to 0% of those who supplement, monitor their B12 levels and take B12 infusions when necessary.A “longitudinal RCT where diet is randomly assigned” and that controls for supplements would not be useful for answering the second question, and neither would the RCTs and systematic reviews I brought up. But they would be more useful than the Faunalytcis survey for answering the first question.
for me, the question is “what should vegan activist’s best guess be right now”
Best guess of what, specifically?
[deleted]
[deleted]
The second point here was not intended and I fixed it within 2 minutes of orthonormal pointing it out, so it doesn’t seem charitable to bring that up. (Though I just re-edited that comment to make this clearer).
The first point was already addressed here.
I’m not sure what to say regarding the third point other than that I didn’t mean to imply that you “should have known and deliberately left out” that study. I just thought it was (literally) useful context. Just edited that comment.
All of this also seems unrelated to this discussion. I’m not sure why me addressing your arguments is being construed as “holding [you] to an exacting standard.”
Here are some Manifold questions about this situation (most from me):
That means they’re making two errors (overstating effect, and effect in wrong direction) rather than just one (overstating effect).
Froolow’s comment claimed that “there’s somewhere between a small signal and no signal that veganism is better with respect to all-cause mortality than omnivorism.” How is that a misleading way of summarizing the adjusted hazard ratio 0.85 (95% CI, 0.73–1.01), in either magnitude or direction? Should he have said that veganism is associated with higher mortality instead?
None of the comments you mentioned in that section claimed that veganism was associated with lower mortality in all subgroups (e.g. women). But even if they had, the hazard ratio for veganism among women was still in the “right” direction (below 1, though just slightly and not meaningfully). Other diets were (just slightly and not meaningfully) better among women, but none of the commenters claimed that veganism was better than all diets either.
Unrelatedly, I noticed that in this comment (and in other comments you’ve made regarding my points about confidence intervals) you don’t seem to argue that the sentence “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” is not misleading.
To be clear, the study found that veganism and pescetarianism were meaningfully associated with lower mortality among men (aHR 0.72 , 95% CI [0.56, 0.92] and 0.73 , 95% CI [0.57, 0.93], respectively), and that no dietary patterns were meaningfully associated with mortality among women. I don’t think it’s misleading to conclude from this that veganism likely has neutral-to-positive effects on lifespan given this study’s data, which was ~my conclusion in the comment I wrote that Elizabeth linked on that section, which was described as “deeply misleading.”
Outcomes for veganism are [...] worse than everything except for omnivorism in women.
As I explained elsewhere a few days ago (after this post was published), this is a very misleading way to describe that study. The correct takeaway is that they could not find any meaningful difference between each diet’s association with mortality among women, not that “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women.”
It’s very important to consider the confidence intervals in addition to the point estimates when interpreting this study (or any study, really, when confidence intervals are available). They provide valuable context to the data.
- EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (28 Sep 2023 23:30 UTC; 317 points)
- EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (EA Forum; 29 Sep 2023 4:04 UTC; 115 points)
- 's comment on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (EA Forum; 4 Oct 2023 23:28 UTC; 1 point)
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here
The first sentence is the title of the post, which has no hedging. The post also claims that“[i]f you’re going to conclude anything from these papers, it’s that fish are great,” which doesn’t seem to be the correct takeaway of this specific study given how wide and similar so many of the aHR 95% CIs are. You also claim “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” in another post without hedging.
In any case, taking the data at face value doesn’t imply “ignoring the confidence intervals.” I don’t see why it would imply that.I thought you were using AHS2 to endorse much larger claims than you are.
My original comment concluded that there was a “substantial probability” that vegan diets are healthier, and quoted someone (correctly) showing that vegans have lower mortality than omnivores in the AHS-2. I didn’t mean to claim it was the healthiest option for everyone; the comment is perfectly compatible with this null result for women. I also claimed that it wasn’t obvious that vegan diets can be expected to make you less healthy ex ante, modulo things like B12. I apologize if it wasn’t clear.
[note: this comment was edited after people’s replies.]
I think it’s useful to look at the confidence intervals, rather than only the point estimates.
The 95% confidence intervals of the adjusted hazard ratios for overall mortality, for men, were [0.56, 0.92] and [0.57, 0.93] for vegan and pescatarian diets, respectively, and for women the CIs are [0.72, 1.07] and [0.78, 1.20], respectively. For women, the confidence intervals for all diets are [0.78, 1.2], [0.83, 1.07], [0.72, 1.07] and [0.7, 1.22].
What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, both of which are better than omnivorism, and likely no difference between any of the diets for women.
The CIs for women specifically look so similar that you could pretend that all of those CIs came from different studies examining the exact same diet, and write a meta-analysis with them, and readers of the meta-analysis would think, “oh, cool, there’s no heterogeneity among the studies!”
In fact, we can go ahead and run a meta-analysis of those aHRs (the ones for women), pretending they’re all for the same diet, and quantitatively check the heterogeneity we get. Doing so, with a random-effects meta-analysis, we find that the is exactly 0%, as is the . The p-value for heterogeneity is 0.92. Whereas this study should update us a little bit on pescatarian diets being better than vegan diets for women, these differences are almost certainly due to chance. No one would suspect that these are actually different diets if you had a meta-analysis with those numbers.
Since the total meta-analytic aHR is also very close to 1, it also looks like none of the diets are meaningfully associated with increased or decreased mortality for women, though there was a slight trend towards lower mortality compared to the nonvegetarian reference diet (p-value: 0.11).
Code:library(metafor) # This is a file with the point estimate and 95% confidence interval for the aHR of each AHS-2 diet. # We're looking at data from women specifically. diet_dataset <- read.csv("diets.csv") # Calculate the SEs from the 95% CIs diet_dataset$logHR <- log(diet_dataset$HR) diet_dataset$SElogHR <- (log(diet_dataset$CI_high) - log(diet_dataset$CI_low)) / 3.92 # Run the meta-analysis res <- rma(yi = logHR, sei = SElogHR, data = diet_dataset, method = "REML") # Print the results summary(res)- 's comment on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (EA Forum; 29 Sep 2023 16:37 UTC; 20 points)
- 's comment on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (EA Forum; 2 Oct 2023 15:49 UTC; 20 points)
- 's comment on EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem by (EA Forum; 4 Oct 2023 22:46 UTC; 7 points)
Given the wide and greatly overlapping confidence intervals for all diets among women, it might be more fitting to interpret these tables as suggesting that “animal product consumption pattern doesn’t seem associated with mortality among women in this sample” than that “small-but-present meat consumption, in addition to millk and eggs, are good for women.” Based on the data presented, a variety of diets could potentially be optimal, and there isn’t a big difference between them. I think this fits my initial conclusion that veganism isn’t obviously bad for your health ex-ante if you supplement e.g. B12.
[Details here.]
If you don’t mind me asking — what was the motivation behind posting 3 separate posts on the same day with very similar content, rather than a single one?
It looks like a large chunk (around a ~quarter or a third or something similar) of the sentences in this post are identical to those in “Cost-effectiveness of student programs for AI safety research” or differ only slightly (by e.g. replacing the word “students” with “professionals” or “participants”).
Moreover, some paragraphs in both of those posts can be found verbatim in the introductory post, “Modeling the impact of AI safety field-building programs,” as well.
This can generate confusion, as people usually don’t expect blog posts to be this similar.
He explains it in this post.