[note: this comment was edited after people’s replies.]
I think it’s useful to look at the confidence intervals, rather than only the point estimates.
The 95% confidence intervals of the adjusted hazard ratios for overall mortality, for men, were [0.56, 0.92] and [0.57, 0.93] for vegan and pescatarian diets, respectively, and for women the CIs are [0.72, 1.07] and [0.78, 1.20], respectively. For women, the confidence intervals for all diets are [0.78, 1.2], [0.83, 1.07], [0.72, 1.07] and [0.7, 1.22].
What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, both of which are better than omnivorism, and likely no difference between any of the diets for women.
The CIs for women specifically look so similar that you could pretend that all of those CIs came from different studies examining the exact same diet, and write a meta-analysis with them, and readers of the meta-analysis would think, “oh, cool, there’s no heterogeneity among the studies!”
In fact, we can go ahead and run a meta-analysis of those aHRs (the ones for women), pretending they’re all for the same diet, and quantitatively check the heterogeneity we get. Doing so, with a random-effects meta-analysis, we find that the I2 is exactly 0%, as is the τ2. The p-value for heterogeneity is 0.92. Whereas this study should update us a little bit on pescatarian diets being better than vegan diets for women, these differences are almost certainly due to chance. No one would suspect that these are actually different diets if you had a meta-analysis with those numbers.
Since the total meta-analytic aHR is also very close to 1, it also looks like none of the diets are meaningfully associated with increased or decreased mortality for women, though there was a slight trend towards lower mortality compared to the nonvegetarian reference diet (p-value: 0.11).
Code:
library(metafor)
# This is a file with the point estimate and 95% confidence interval for the aHR of each AHS-2 diet.
# We're looking at data from women specifically.
diet_dataset <- read.csv("diets.csv")
# Calculate the SEs from the 95% CIs
diet_dataset$logHR <- log(diet_dataset$HR)
diet_dataset$SElogHR <- (log(diet_dataset$CI_high) - log(diet_dataset$CI_low)) / 3.92
# Run the meta-analysis
res <- rma(yi = logHR, sei = SElogHR, data = diet_dataset, method = "REML")
# Print the results
summary(res)
Let me make sure I understand correctly: you view AHS-2 as supporting the belief that vegan diets can be healthy, for some people, but not anything larger than that?
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here, and the area of contention is that I thought you were using AHS2 to endorse much larger claims than you are.
it seems slightly misleading to claim “Adventist Health Study-2 supports pescetarianism more than veganism,” or that vegan diets were “the worst choice besides unconstrained meat consumption” for women. What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, and likely no difference between any of the diets for women
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here
The first sentence is the title of the post, which has no hedging. The post also claims that“[i]f you’re going to conclude anything from these papers, it’s that fish are great,” which doesn’t seem to be the correct takeaway of this specific study given how wide and similar so many of the aHR 95% CIs are. You also claim “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” in another post without hedging.
In any case, taking the data at face value doesn’t imply “ignoring the confidence intervals.” I don’t see why it would imply that.
I thought you were using AHS2 to endorse much larger claims than you are.
My original comment concluded that there was a “substantial probability” that vegan diets are healthier, and quoted someone (correctly) showing that vegans have lower mortality than omnivores in the AHS-2. I didn’t mean to claim it was the healthiest option for everyone; the comment is perfectly compatible with this null result for women. I also claimed that it wasn’t obvious that vegan diets can be expected to make you less healthy ex ante, modulo things like B12. I apologize if it wasn’t clear.
I think everything I said makes sense in a context where people (including the study authors) are using this paper to argue for veg*nism. I’m arguing that the study is both weaker than reported, and (weakly) pointing in a different direction than reported. I agree that if a fish farm lobbying group was using this paper to claim that fish were the cure for aging, that would be very misleading and I’d argue against them too.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
It would be helpful if you clarified the population you are talking about. I’ve already said I think some people’s optimal diet is vegan, and for some other people vegan is the best out of the options they can realistically achieve. So unless you mean a substantial probability everyone’s optimal diet is vegan, and there is no such thing as a prohibitive health issue, we’re not disagreeing.
I also feel like saying “modulo things like B12” is burying the lede. A lot of my point is that vegan advocates are recruiting people without providing the necessary nutritional education, and are in some cases fighting that education even when it’s done in a vegan-compatible way.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
This might be semantics, but when you said “Change my mind: Veganism entails trade-offs, and health is one of the axes” I (until now) interpreted the claim as vegans needing to trade off health (writ large) against other desirable properties (taste, cost, convenience, etc), not a tradeoff within different components of health.
I don’t have a sense of how common my reading was, however, and I don’t want to put words in Natalia’s mouth.
Thank you for the write-up! Just as a minor quibble, veganism has not been considered the “healthiest choice” ever, or at least not for a long time, if I were to make a guess about “consensus” in the field. While it has been clear for a while that a diet biased towards plants is healthy, the data for the addition of certain food groups (fatty fish, fermented and low-fat dairy, etc) is pretty strong as is the data for the health benefits of individual carninutrients (creatine or even taurine).
As you correctly point out, the issue of residual confounding is unsolvable. All we can take from these studies are hints and ideas. The recent failure of Vitamin D to live up to the hype, initially generated by observational studies, is a case in point.
I am particularly weary of studies of dietary patterns, whether they are vegetarian or Mediterranean or others, since I would expect to see the strongest biases here (because these patterns are associated with lifestyles, class, belief etc). Nevertheless, studies on surrogate endpoints like cholesterol and studies on single food groups do support the whole idea of reducing meat consumption.
The Adventist Study is interesting, but surely not definitive. It’s not even a RCT. To my knowledge, the only respectable RCT ever conducted analyzing a diet pattern is PREDIMED. It doesn’t test veganism specifically (the treatment group adopts a mediterranean diet), but it does increase the strength of the association between plant based diets and lower overall mortality risk. Overall, I think the level of evidence against meat is only suggestive, but since it’s so time-consuming and expensive to conduct these trials, I don’t expect much further light coming from investigations using metrics like mortality, heart attacks, strokes, etc. I think epigenetic clocks will in the future be a much better way to quickly analyze the effects of diet interventions, and I suspect plant-based diets will have an advantage over other diet patterns.
[note: this comment was edited after people’s replies.]
I think it’s useful to look at the confidence intervals, rather than only the point estimates.
The 95% confidence intervals of the adjusted hazard ratios for overall mortality, for men, were [0.56, 0.92] and [0.57, 0.93] for vegan and pescatarian diets, respectively, and for women the CIs are [0.72, 1.07] and [0.78, 1.20], respectively. For women, the confidence intervals for all diets are [0.78, 1.2], [0.83, 1.07], [0.72, 1.07] and [0.7, 1.22].
What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, both of which are better than omnivorism, and likely no difference between any of the diets for women.
The CIs for women specifically look so similar that you could pretend that all of those CIs came from different studies examining the exact same diet, and write a meta-analysis with them, and readers of the meta-analysis would think, “oh, cool, there’s no heterogeneity among the studies!”
In fact, we can go ahead and run a meta-analysis of those aHRs (the ones for women), pretending they’re all for the same diet, and quantitatively check the heterogeneity we get. Doing so, with a random-effects meta-analysis, we find that the I2 is exactly 0%, as is the τ2. The p-value for heterogeneity is 0.92. Whereas this study should update us a little bit on pescatarian diets being better than vegan diets for women, these differences are almost certainly due to chance. No one would suspect that these are actually different diets if you had a meta-analysis with those numbers.
Since the total meta-analytic aHR is also very close to 1, it also looks like none of the diets are meaningfully associated with increased or decreased mortality for women, though there was a slight trend towards lower mortality compared to the nonvegetarian reference diet (p-value: 0.11).
Code:
Let me make sure I understand correctly: you view AHS-2 as supporting the belief that vegan diets can be healthy, for some people, but not anything larger than that?
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here, and the area of contention is that I thought you were using AHS2 to endorse much larger claims than you are.
The first sentence is the title of the post, which has no hedging. The post also claims that“[i]f you’re going to conclude anything from these papers, it’s that fish are great,” which doesn’t seem to be the correct takeaway of this specific study given how wide and similar so many of the aHR 95% CIs are. You also claim “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” in another post without hedging.
In any case, taking the data at face value doesn’t imply “ignoring the confidence intervals.” I don’t see why it would imply that.
My original comment concluded that there was a “substantial probability” that vegan diets are healthier, and quoted someone (correctly) showing that vegans have lower mortality than omnivores in the AHS-2. I didn’t mean to claim it was the healthiest option for everyone; the comment is perfectly compatible with this null result for women. I also claimed that it wasn’t obvious that vegan diets can be expected to make you less healthy ex ante, modulo things like B12. I apologize if it wasn’t clear.
I think everything I said makes sense in a context where people (including the study authors) are using this paper to argue for veg*nism. I’m arguing that the study is both weaker than reported, and (weakly) pointing in a different direction than reported. I agree that if a fish farm lobbying group was using this paper to claim that fish were the cure for aging, that would be very misleading and I’d argue against them too.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
It would be helpful if you clarified the population you are talking about. I’ve already said I think some people’s optimal diet is vegan, and for some other people vegan is the best out of the options they can realistically achieve. So unless you mean a substantial probability everyone’s optimal diet is vegan, and there is no such thing as a prohibitive health issue, we’re not disagreeing.
I also feel like saying “modulo things like B12” is burying the lede. A lot of my point is that vegan advocates are recruiting people without providing the necessary nutritional education, and are in some cases fighting that education even when it’s done in a vegan-compatible way.
This might be semantics, but when you said “Change my mind: Veganism entails trade-offs, and health is one of the axes” I (until now) interpreted the claim as vegans needing to trade off health (writ large) against other desirable properties (taste, cost, convenience, etc), not a tradeoff within different components of health.
I don’t have a sense of how common my reading was, however, and I don’t want to put words in Natalia’s mouth.
Life is trade-offs all the way down.
@LW team: LW still seems to struggle with importing stuff like emojis, trademark symbols, etc. from other blogs.
Here’s how that paragraph looks like for me on Elizabeth’s website:
I just edited this to fix the giant emoji in this instance, and made a code fix that should (hopefully) stop it from happening in the future.
Thanks <3! I’ve seen this occasionally in Zvi’s crossposts too, e.g. giant “tm” symbols. Will that fix those, too?
Thank you for the write-up! Just as a minor quibble, veganism has not been considered the “healthiest choice” ever, or at least not for a long time, if I were to make a guess about “consensus” in the field. While it has been clear for a while that a diet biased towards plants is healthy, the data for the addition of certain food groups (fatty fish, fermented and low-fat dairy, etc) is pretty strong as is the data for the health benefits of individual carninutrients (creatine or even taurine).
As you correctly point out, the issue of residual confounding is unsolvable. All we can take from these studies are hints and ideas. The recent failure of Vitamin D to live up to the hype, initially generated by observational studies, is a case in point.
I am particularly weary of studies of dietary patterns, whether they are vegetarian or Mediterranean or others, since I would expect to see the strongest biases here (because these patterns are associated with lifestyles, class, belief etc). Nevertheless, studies on surrogate endpoints like cholesterol and studies on single food groups do support the whole idea of reducing meat consumption.
The Adventist Study is interesting, but surely not definitive. It’s not even a RCT. To my knowledge, the only respectable RCT ever conducted analyzing a diet pattern is PREDIMED. It doesn’t test veganism specifically (the treatment group adopts a mediterranean diet), but it does increase the strength of the association between plant based diets and lower overall mortality risk. Overall, I think the level of evidence against meat is only suggestive, but since it’s so time-consuming and expensive to conduct these trials, I don’t expect much further light coming from investigations using metrics like mortality, heart attacks, strokes, etc. I think epigenetic clocks will in the future be a much better way to quickly analyze the effects of diet interventions, and I suspect plant-based diets will have an advantage over other diet patterns.