Wow, I just read Robin’s writeup on this and it caused me to significantly lower the amount of credence I place on his other positions (but very slightly lower my opinion of supplements). It just struck me as overwhelmingly sloppy and rhetorical. Particularly his justification attempt in response to this thread. (But I suppose Robin’s responses to criticism have never impressed me anyway.)
The only causal hypothesis I’ve heard for the supplements leading to an higher rate of death is Phil’s claim that the doses are too high and are resulting in vitamin toxicity. If we accept Robin’s claim that “vitamins kill” (that is that the causal claim is true and the results aren’t just the result of uncontrolled-for correlations) one of three things has to be true: vitamins kill no matter how the body obtains them; there is something involved in taking ‘supplements’ that kills; or Phil is right and vitamin toxicity is the cause. (Is there an option I’m missing?) It seems extremely clear to me that absent any hypothesis for the second of these, vitamin toxicity is by far the most likely explanation. And the best part about this hypothesis is that is also the easiest to test. All you have to do is look for hockeysticks!
Why refuse to test the one explanation for the results we have?
Another possibility is that cancers have higher nutritional needs than normal cells, and some vitamins might be feeding cancer more than they’re feeding the person.
One more explanation: some supplements might come from untrustworthy manufacturers and be contaminated with an unidentified toxin. That’s a tough one to test, since it’s unlikely that any of the studies saved samples of the pills they used or even documented where they came from.
Plants and animals are our fellow biological creatures. While the chemical combinations in them might not be ideal for us especially if a person isn’t eating a varied diet, supplements can give proportions of vitamins, minerals, and whatever which have never been seen in nature.
One more angle: I’ve heard claims that agricultural soil has much less minerals than it did in the ancestral environment, and more so in recent decades. It’s been a long time since the glaciers came through, grinding the rocks and as far as I know, modern agriculture typically doesn’t include replacing the minerals taken out with each harvest. On the organic side, manure isn’t going to help that much if it’s from animals that were fed low-mineral food.
I file this under plausible theory. It doesn’t address which minerals are low, or how much should be added. One of my friends uses it as a reason to supplement.
I’ve worried about this too, but isn’t this something that can be easily tested? I mean, if nutritionists have really identified the vitamins, etc. we need, divided the thingspace appropriately (are all things called “protein” functionally the same?), and developed reliable ways of measuring nutritional information that goes on the food label, this should show up pretty quickly. (I’m becoming less confident of assumptions like these, in part because of errors like what Phil Goetz found here.)
When standard produce is packaged, how do they get the data for the label? Do they have to regularly test that source’s produce, or is there just a standard lookup table that e.g. all baby carrots can give as their data? (If the latter, that screams “information cascade!”)
If you look at Table 2 in the paper, it shows doses of each vitamin for every study that is considered low risk for bias. I count 9 studies that have vitamin A <10,000 IU and vitamin E <300 IU, which is what PhilGoetz said are good dosage levels.
The point estimates from those 9 studies (see figure 2) are: 2.88, 0.18, 3.3, 2.11, 1.05, 1.02, 0.78, 0.87, 1.99. (1 favors control)
Based on this quick look at the studies, I don’t see any reason to believe that a “hockey stick” model will show a benefit of supplements at lower dose levels.
Based on this quick look at the studies, I don’t see any reason to believe that a “hockey stick” model will show a benefit of supplements at lower dose levels.
The titular contention used the word ‘kill’. That’s what hockey sticks tend to do.
But Goetz implies that the vitamins may have benefits in the right regime:
This is not how vitamins work. Vitamin A is toxic in doses over 15,000 IU/day, and vitamin E is toxic in doses over 400 IU/day (Miller et al. 2004, Meta-Analysis: High-Dosage Vitamin E Supplementation May Increase All-Cause Mortality; Berson et al. 1993, Randomized trial of vitamin A and vitamin E supplementation for retinitis pigmentosa.). The RDA for vitamin A is 2500 IU/day for adults. Good dosage levels for vitamin A appear to be under 10,000 IU/day, and for E, less than 300 IU/day. (Sadly, studies rarely discriminate in their conclusions between dosage levels for men and women. Doing so would give more useful results, but make it harder to reach the coveted P < .05 or P < .01.)
...Vitamins, like any medicine, have an inverted-J-shaped response curve. If you graph their health effects, with dosage on the horizontal access, and some measure of their effects—say, change to average lifespan—on the vertical axis, you would get an upside-down J. (If you graph the death rate on the vertical axis, as in this study, you would get a rightside-up J.) That is, taking a moderate amount has some good effect; taking a huge a mount has a large bad effect.
It would be a strange usage of ‘good’ if all Goetz meant by it was ‘increases fatalities by too small an amount to easily detect’ rather than ‘increases some desirable outcome’.
Robin generates a lot of clever ideas. That’s awesome. I’ve heard him retract before, but it is tempting to make the minimum feasible patch to your stated ideas when someone exposes some shallowness or flaw in your analysis.
I’m interested in a large single prospective on vitamin supplementation. I can’t believe any good will come at looking retrospectively at correlations (I’m too lazy right now to see if that’s actually a problem with any of the studies used in the meta-analysis) - people with more health problems (and especially older people) tend to take more vitamins.
people with more health problems (and especially older people) tend to take more vitamins.
Confounding things further but in the opposite direction—people who are health conscious (and wealthier) take more vitamins too.
My approach regarding this topic is that I plan on going and doing/funding research on this kind of thing myself and are actively re-educating myself and acquiring resources in order to do so. In the mean time I’m just not going to take excessive doses of fat soluble vitamins—because that’d just be a stupid idea in the first place!
Cool. Huge prospective study. They demonstrated that there’s no reduction in (several types of) cancer or heart attack mortality with either vit. E or vit. C supplementation (400 IU of vitamin E every other day and 500 mg of vitamin C daily). It’s strange to me that they don’t look at all-cause mortality.
I think they did look at overall mortality. Quoting from the abstract of the 2008 paper
Neither vitamin E (HR, 1.07 [95% CI, 0.97-1.18]; P = .15) nor vitamin C (HR, 1.07 [95% CI, 0.97-1.18]; P = .16) had a significant effect on total mortality but vitamin E was associated with an increased risk of hemorrhagic stroke (HR, 1.74 [95% CI, 1.04-2.91]; P = .04).)
You’re right. I missed it. I know they say it’s not significant, but in fact the two P=.15 are weakly convincing to me (95% CI of .97-1.18). The old men are dying off 7% more often if they have the (thought to be reasonable at the beginning of the study) dose of vit C or E (compared to placebo). Redo the study and you’ll probably get something like 4-10% instead of 7%. I think this is pretty good evidence for Robin’s claim.
This sort of binary treatment-variable study can always be criticized for overly high doseage, as Phil Goetz pointed out. The 400 IU vit E every 2 days is well under the dose already commonly accepted to cause long-term problems (400 IU daily). The 500mg vit. C daily is well above the highest dietary recommendation of 100mg/day, but it’s well below the amount some people take.
That was my reaction, too. But given that his complaint was that no one redid the analysis on the actual data, I think it’s appropriate to wait a bit and see if he does it himself, and what the result is. If he doesn’t follow through by looking at the actual data, then I’ll unsubscribing from OB, because that would be a strong signal of bias; but for now I prefer to wait and see.
Also, that relying on that sort of broad statistical analysis makes sense if you have no other source of information. If you have ways of telling whether particular supplements make your life or health better, then experimenting with supplements for yourself could be a good idea.
Is the data easily available anywhere? If it’s not, that might explain why no one redid the analysis. If it is, I might be persuaded to try a few different models out.
EDIT: After some digging, I’m pretty sure I know what statistical model they used in the study and should be able to reproduce their results and try a few different things IF I can get the data in a sufficiently nice form.
EDIT2: see my recent post in the discussion section
I didn’t think I understood enough of what they did to comment usefully anymore, and I don’t have the time currently (i.e., this week) to put into understanding it. Did you read it after I had put the edit at the end? If so, did you still find it useful?
Wow, I just read Robin’s writeup on this and it caused me to significantly lower the amount of credence I place on his other positions (but very slightly lower my opinion of supplements). It just struck me as overwhelmingly sloppy and rhetorical. Particularly his justification attempt in response to this thread. (But I suppose Robin’s responses to criticism have never impressed me anyway.)
Here is what is bothering me:
The only causal hypothesis I’ve heard for the supplements leading to an higher rate of death is Phil’s claim that the doses are too high and are resulting in vitamin toxicity. If we accept Robin’s claim that “vitamins kill” (that is that the causal claim is true and the results aren’t just the result of uncontrolled-for correlations) one of three things has to be true: vitamins kill no matter how the body obtains them; there is something involved in taking ‘supplements’ that kills; or Phil is right and vitamin toxicity is the cause. (Is there an option I’m missing?) It seems extremely clear to me that absent any hypothesis for the second of these, vitamin toxicity is by far the most likely explanation. And the best part about this hypothesis is that is also the easiest to test. All you have to do is look for hockeysticks!
Why refuse to test the one explanation for the results we have?
Another possibility is that cancers have higher nutritional needs than normal cells, and some vitamins might be feeding cancer more than they’re feeding the person.
One more explanation: some supplements might come from untrustworthy manufacturers and be contaminated with an unidentified toxin. That’s a tough one to test, since it’s unlikely that any of the studies saved samples of the pills they used or even documented where they came from.
One more hypothesis: supplements are risky because you may end up with vitamins and minerals which are out of proportion with each other.
Do we have reason to think this out of proportion issue would arise more often with supplements than it would just from diet?
Plants and animals are our fellow biological creatures. While the chemical combinations in them might not be ideal for us especially if a person isn’t eating a varied diet, supplements can give proportions of vitamins, minerals, and whatever which have never been seen in nature.
One more angle: I’ve heard claims that agricultural soil has much less minerals than it did in the ancestral environment, and more so in recent decades. It’s been a long time since the glaciers came through, grinding the rocks and as far as I know, modern agriculture typically doesn’t include replacing the minerals taken out with each harvest. On the organic side, manure isn’t going to help that much if it’s from animals that were fed low-mineral food.
I file this under plausible theory. It doesn’t address which minerals are low, or how much should be added. One of my friends uses it as a reason to supplement.
I’ve worried about this too, but isn’t this something that can be easily tested? I mean, if nutritionists have really identified the vitamins, etc. we need, divided the thingspace appropriately (are all things called “protein” functionally the same?), and developed reliable ways of measuring nutritional information that goes on the food label, this should show up pretty quickly. (I’m becoming less confident of assumptions like these, in part because of errors like what Phil Goetz found here.)
When standard produce is packaged, how do they get the data for the label? Do they have to regularly test that source’s produce, or is there just a standard lookup table that e.g. all baby carrots can give as their data? (If the latter, that screams “information cascade!”)
I don’t know whether nutritionists have identified all the nutrients we need.
And I’m pretty sure that the functioning of living organisms isn’t terribly well understood, even for “ideal” cases.
If you look at Table 2 in the paper, it shows doses of each vitamin for every study that is considered low risk for bias. I count 9 studies that have vitamin A <10,000 IU and vitamin E <300 IU, which is what PhilGoetz said are good dosage levels.
The point estimates from those 9 studies (see figure 2) are: 2.88, 0.18, 3.3, 2.11, 1.05, 1.02, 0.78, 0.87, 1.99. (1 favors control)
Based on this quick look at the studies, I don’t see any reason to believe that a “hockey stick” model will show a benefit of supplements at lower dose levels.
The titular contention used the word ‘kill’. That’s what hockey sticks tend to do.
But Goetz implies that the vitamins may have benefits in the right regime:
It would be a strange usage of ‘good’ if all Goetz meant by it was ‘increases fatalities by too small an amount to easily detect’ rather than ‘increases some desirable outcome’.
Robin generates a lot of clever ideas. That’s awesome. I’ve heard him retract before, but it is tempting to make the minimum feasible patch to your stated ideas when someone exposes some shallowness or flaw in your analysis.
I’m interested in a large single prospective on vitamin supplementation. I can’t believe any good will come at looking retrospectively at correlations (I’m too lazy right now to see if that’s actually a problem with any of the studies used in the meta-analysis) - people with more health problems (and especially older people) tend to take more vitamins.
Confounding things further but in the opposite direction—people who are health conscious (and wealthier) take more vitamins too.
My approach regarding this topic is that I plan on going and doing/funding research on this kind of thing myself and are actively re-educating myself and acquiring resources in order to do so. In the mean time I’m just not going to take excessive doses of fat soluble vitamins—because that’d just be a stupid idea in the first place!
I’d be extremely grateful if you do investigate this.
See http://phs.bwh.harvard.edu/
Cool. Huge prospective study. They demonstrated that there’s no reduction in (several types of) cancer or heart attack mortality with either vit. E or vit. C supplementation (400 IU of vitamin E every other day and 500 mg of vitamin C daily). It’s strange to me that they don’t look at all-cause mortality.
I think they did look at overall mortality. Quoting from the abstract of the 2008 paper
You’re right. I missed it. I know they say it’s not significant, but in fact the two P=.15 are weakly convincing to me (95% CI of .97-1.18). The old men are dying off 7% more often if they have the (thought to be reasonable at the beginning of the study) dose of vit C or E (compared to placebo). Redo the study and you’ll probably get something like 4-10% instead of 7%. I think this is pretty good evidence for Robin’s claim.
This sort of binary treatment-variable study can always be criticized for overly high doseage, as Phil Goetz pointed out. The 400 IU vit E every 2 days is well under the dose already commonly accepted to cause long-term problems (400 IU daily). The 500mg vit. C daily is well above the highest dietary recommendation of 100mg/day, but it’s well below the amount some people take.
That was my reaction, too. But given that his complaint was that no one redid the analysis on the actual data, I think it’s appropriate to wait a bit and see if he does it himself, and what the result is. If he doesn’t follow through by looking at the actual data, then I’ll unsubscribing from OB, because that would be a strong signal of bias; but for now I prefer to wait and see.
Also, that relying on that sort of broad statistical analysis makes sense if you have no other source of information. If you have ways of telling whether particular supplements make your life or health better, then experimenting with supplements for yourself could be a good idea.
Is the data easily available anywhere? If it’s not, that might explain why no one redid the analysis. If it is, I might be persuaded to try a few different models out.
EDIT: After some digging, I’m pretty sure I know what statistical model they used in the study and should be able to reproduce their results and try a few different things IF I can get the data in a sufficiently nice form.
EDIT2: see my recent post in the discussion section
Why’d you delete it? :-( I was wondering earlier where it went.
I didn’t think I understood enough of what they did to comment usefully anymore, and I don’t have the time currently (i.e., this week) to put into understanding it. Did you read it after I had put the edit at the end? If so, did you still find it useful?