I’m really confused by this. Both examples sound like exactly what you should be doing.
If the Tibetan study was to determine how many people died in childbirth, and they refused to publish because not enough people were dying to support their preferred narrative, that would be bad. But the way you wrote it sounds like it was to determine differences in deaths across two conditions, but there weren’t enough deaths to tell. In that case, stopping the study instead of publishing results that are probably false is the best possible course of action.
Likewise, doing studies in isolated villages is a good way to control for confounding variables. If you wanted to measure effectiveness of a cancer drug, you do it on people who are only taking the one drug, not people taking lots of drugs that might differ across groups. If you’re measuring life expectancy in AIDS, you do it on patients who only have AIDS, not patients who have AIDS and also cancer. If a physicist is measuring the radioactivity of U-235, ey makes sure to use a purified sample of U-235, not a sample mixed with U-238. I don’t even see what this has to do with double-blinding (which is about placebos and making sure subjects don’t know what group they’re in). This is just a basic attempt to isolate the experimental variable. I agree that given infinite time and resources they should try follow-up studies in less isolated villages to see if the results carry over, but in reality this seems like the best way to do things.
I suspect that the Tibet study issue could be solved with better statistical methodology (i.e. using a Bayesian framework and including semi-realistic prior information). However, setting that aside, I agree completely.
I’m really confused by this. Both examples sound like exactly what you should be doing.
If the Tibetan study was to determine how many people died in childbirth, and they refused to publish because not enough people were dying to support their preferred narrative, that would be bad. But the way you wrote it sounds like it was to determine differences in deaths across two conditions, but there weren’t enough deaths to tell. In that case, stopping the study instead of publishing results that are probably false is the best possible course of action.
Likewise, doing studies in isolated villages is a good way to control for confounding variables. If you wanted to measure effectiveness of a cancer drug, you do it on people who are only taking the one drug, not people taking lots of drugs that might differ across groups. If you’re measuring life expectancy in AIDS, you do it on patients who only have AIDS, not patients who have AIDS and also cancer. If a physicist is measuring the radioactivity of U-235, ey makes sure to use a purified sample of U-235, not a sample mixed with U-238. I don’t even see what this has to do with double-blinding (which is about placebos and making sure subjects don’t know what group they’re in). This is just a basic attempt to isolate the experimental variable. I agree that given infinite time and resources they should try follow-up studies in less isolated villages to see if the results carry over, but in reality this seems like the best way to do things.
Am I misunderstanding the book’s points?
I suspect that the Tibet study issue could be solved with better statistical methodology (i.e. using a Bayesian framework and including semi-realistic prior information). However, setting that aside, I agree completely.