Unadjusted associations in observational studies should not guide decisions (“hospitals have a lot of sick people, therefore I should stay away from hospitals because they will make me sick!”). Either use a randomized trial, which is the gold standard for establishing causal association, or use an observational study and adjust for confounding and other biases appropriately.
“hospitals have a lot of sick people, therefore I should stay away from hospitals because they will make me sick!”
That’s not necessarily as silly as your offhand treatment seems to suggest; it’s precisely one of the explanations proposed for some of the RAND health insurance experiment’s results. (One of the conclusions the RAND study suggested was that, for people who aren’t both poor and chronically ill, reduction in healthcare services (resulting from increased use fees) might not have a significant impact on health. It wasn’t designed to test this relationship, however, and I’m unaware of any subsequent studies which were.)
Your reply applies (at least) equally well to the following argument: all plants have green leaves; roses have green leaves; therefore roses are plants. In both cases your reply would defend an obviously silly inference pattern by pointing to one instance in which it leads to a true conclusion. (Although in your case the inference actually predicts that hospital-provided medical care has a negative marginal effect on health, not a neutral one.)
Not really. Illness is a proximally transitive property; being in the same room as an ill person can, indeed, make you ill. It only leads to support of silly inference patterns if you disregard the actual meaning of the words used in the logical statement.
Consider the actual meaning of “therefore” used in IlyaShpitser’s post. It’s logically stronger than “and”.
In the least convenient possible world where health care is good and diseases aren’t transmissible, the naive causal inference would lead to the same conclusion.
Except that it wasn’t formal logic; the conclusion didn’t follow from the premise, because there’s nothing indicating that sickness is something which should be avoided. Therefore, the reader is expected to use information contextual to the world the author is in; namely, that sickness is bad, and should be avoided. Because there’s also another possible world where -diseases- are good, and the construction wouldn’t even make sense.
Yes, I of course agree that hospitals are cauldrons of disease, and can make you sicker (especially if recovering from surgery). Medical errors by staff is a big issue as well.
a randomized trial, which is the gold standard for establishing causal association
even then you have placebo and file drawer effects.
and adjust for confounding and other biases appropriately
from the article: “Whenever possible, adjusted relative risks were extracted; otherwise, crude relative risks and 95% confidence intervals (CIs) were calculated from the number of events.”
“Forty-eight curves (908 182 subjects and 86 941 deaths) were adjusted at least for age; among them, 28 were adjusted for social status too, and 10 for social status and dietary markers.”
This is adjusting for 1-4 confounders out of many possible confounders. Even doing so they lost half of the association.
Note that the scientists themselves do not claim a causal effect, but only association. It’s reasonable to take their word for this.
I personally would not change dietary habits just based on studies like this.
No because saying alcohol is healthy could get them in lots of trouble.
I personally would not change dietary habits just based on studies like this.
Not even a little? Does it at least slightly increase your estimate of alcohol being healthy and so if at a social event deciding whether to have a drink shift your cost benefit analysis?
| No because saying alcohol is healthy could get them in lots of trouble.
You may be right—on the other hand, one can get a lot of publicity for a controversial finding. I don’t think these sorts of studies use conservative language because they fear getting in trouble due to subject matter. I think they fear getting in trouble for using the wrong statistical methodology or the wrong language to describe it.
| Not even a little? Does it at least slightly increase your estimate of alcohol being healthy and so if at a social event deciding whether to have a drink shift your cost benefit analysis?
I am not a rationalist, and I don’t use these kinds of cost benefit analyses when out drinking :). These sorts of studies are simply never a swing vote in my decision making. I agree that these studies are weak evidence.
Unadjusted associations in observational studies should not guide decisions (“hospitals have a lot of sick people, therefore I should stay away from hospitals because they will make me sick!”). Either use a randomized trial, which is the gold standard for establishing causal association, or use an observational study and adjust for confounding and other biases appropriately.
I do avoid hospitals (except in emergencies) and precisely because they are dangerous.
“hospitals have a lot of sick people, therefore I should stay away from hospitals because they will make me sick!”
That’s not necessarily as silly as your offhand treatment seems to suggest; it’s precisely one of the explanations proposed for some of the RAND health insurance experiment’s results. (One of the conclusions the RAND study suggested was that, for people who aren’t both poor and chronically ill, reduction in healthcare services (resulting from increased use fees) might not have a significant impact on health. It wasn’t designed to test this relationship, however, and I’m unaware of any subsequent studies which were.)
My go-to reductio is “Olympic sprinters have lots of gold medals; I should wear lots of gold medals to run faster!”
Clearly, it only fails because there’s too many Olympic sports.
You have no way of telling if it’s making you run faster, swim faster, shoot better, or do backflips better.
I dub this the Bling Fitness Theory.
This is the clearest example of what’s wrong with evidential decision theory I’ve ever seen!
Your reply applies (at least) equally well to the following argument: all plants have green leaves; roses have green leaves; therefore roses are plants. In both cases your reply would defend an obviously silly inference pattern by pointing to one instance in which it leads to a true conclusion. (Although in your case the inference actually predicts that hospital-provided medical care has a negative marginal effect on health, not a neutral one.)
Not really. Illness is a proximally transitive property; being in the same room as an ill person can, indeed, make you ill. It only leads to support of silly inference patterns if you disregard the actual meaning of the words used in the logical statement.
Consider the actual meaning of “therefore” used in IlyaShpitser’s post. It’s logically stronger than “and”.
In the least convenient possible world where health care is good and diseases aren’t transmissible, the naive causal inference would lead to the same conclusion.
+1 to writing up my tentative ‘How to use the LCPW principle safely and sanely’ post. Or at least sending a draft to Yvain for thoughts.
Until that post comes, I’m curious what kind of an example you intend to make of my use of the concept—safe/sane or otherwise?
Otherwise—the post was basically “remember that the least convenient possible world has to at least be possible”.
Except that it wasn’t formal logic; the conclusion didn’t follow from the premise, because there’s nothing indicating that sickness is something which should be avoided. Therefore, the reader is expected to use information contextual to the world the author is in; namely, that sickness is bad, and should be avoided. Because there’s also another possible world where -diseases- are good, and the construction wouldn’t even make sense.
Yes, I of course agree that hospitals are cauldrons of disease, and can make you sicker (especially if recovering from surgery). Medical errors by staff is a big issue as well.
even then you have placebo and file drawer effects.
from the article: “Whenever possible, adjusted relative risks were extracted; otherwise, crude relative risks and 95% confidence intervals (CIs) were calculated from the number of events.”
From the paper itself:
“Forty-eight curves (908 182 subjects and 86 941 deaths) were adjusted at least for age; among them, 28 were adjusted for social status too, and 10 for social status and dietary markers.”
This is adjusting for 1-4 confounders out of many possible confounders. Even doing so they lost half of the association. Note that the scientists themselves do not claim a causal effect, but only association. It’s reasonable to take their word for this.
I personally would not change dietary habits just based on studies like this.
They claim causation in several places, albeit sprinkled with perhapses and maybes. From the abstract:
From the Comment section:
This is all causal language.
No because saying alcohol is healthy could get them in lots of trouble.
Not even a little? Does it at least slightly increase your estimate of alcohol being healthy and so if at a social event deciding whether to have a drink shift your cost benefit analysis?
| No because saying alcohol is healthy could get them in lots of trouble.
You may be right—on the other hand, one can get a lot of publicity for a controversial finding. I don’t think these sorts of studies use conservative language because they fear getting in trouble due to subject matter. I think they fear getting in trouble for using the wrong statistical methodology or the wrong language to describe it.
| Not even a little? Does it at least slightly increase your estimate of alcohol being healthy and so if at a social event deciding whether to have a drink shift your cost benefit analysis?
I am not a rationalist, and I don’t use these kinds of cost benefit analyses when out drinking :). These sorts of studies are simply never a swing vote in my decision making. I agree that these studies are weak evidence.