I remember one scientist who pointed out that observational studies were one of the weakest forms of evidence possible. This type of study can detect things like “smoking is bad for you”, because smokers are 30 times more likely to die of lung cancer. But once you get down to smaller effect sizes, you run into the problem that observational studies hopelessly mix up different correlated variables. So for many things, observational studies essentially return random noise. This is allegedly what happened with HRT for post-menopausal women, where the observational studies failed to note that the people taking HRT contained a much larger proportion of nurses and other people who complied with medical advice. And then there’s nutrition, where every study feels like it gets reversed every 5 years.
Or the way that Vitamin D levels are apparently correlated with almost every measure of good health, but Vitamin D supplementation notoriously fails to actually improve any of those measures.
“2x” is a big enough effect size that this may actually be real, and not a spurious correlation. And of course, there’s the underlying history of other sleep medications apparently being cursed to have horrible side effects, so “melatonin is actually terrible for you” wouldn’t be surprising.
(Since we are speaking of observational studies, I suspect that at least two of the things I have claimed in this post are Officially Wrong. Which two things are officially wrong may depend on what year you read it.)
I remember one scientist who pointed out that observational studies were one of the weakest forms of evidence possible. This type of study can detect things like “smoking is bad for you”, because smokers are 30 times more likely to die of lung cancer. But once you get down to smaller effect sizes, you run into the problem that observational studies hopelessly mix up different correlated variables. So for many things, observational studies essentially return random noise. This is allegedly what happened with HRT for post-menopausal women, where the observational studies failed to note that the people taking HRT contained a much larger proportion of nurses and other people who complied with medical advice. And then there’s nutrition, where every study feels like it gets reversed every 5 years.
Or the way that Vitamin D levels are apparently correlated with almost every measure of good health, but Vitamin D supplementation notoriously fails to actually improve any of those measures.
“2x” is a big enough effect size that this may actually be real, and not a spurious correlation. And of course, there’s the underlying history of other sleep medications apparently being cursed to have horrible side effects, so “melatonin is actually terrible for you” wouldn’t be surprising.
(Since we are speaking of observational studies, I suspect that at least two of the things I have claimed in this post are Officially Wrong. Which two things are officially wrong may depend on what year you read it.)