I’ve read a few articles on this issue, and the problem seems pretty alarming. From what I understand, there are only a small handful of journals that accept null hypothesis (i.e. hypothesis that X is not true), in fact I think there might be only one (JASN).
The vast majority of journals reject (or at least discourage) non-positive results except in the case of famous researchers or contentious issues, which means studies that show negative results tend to not get published. In fact, most researchers don’t even attempt to publish—they start over, or give up and move to a different project. If the study was critical to their career, they may even move to another field entirely.
This meta-study examines studies of publication bias and reporting bias (unfavorable results omitted from conclusions). It comes to the conclusion that studies that show positive results are significantly more likely to be published than studies that don’t.
If the academic culture is discouraging studies that show negative results via the publication process, doesn’t that seem to imply there is, at the very least, a major inefficiency in our process of learning new things?
I’m no scientist, but I do have to do a lot of troubleshooting for my job, and while knowing what works is most important, knowing what doesn’t takes a close second.
If journals encouraged negative results as much as positive results, I’d imagine we’d see major new scientific breakthroughs twice as often as we currently do—and that’s kind of a big deal, I think. Right now the huge academic pressure is not to produce valid results, but to produce positive results. That’s a major problem, in my opinion, precisely because human beings are very vulnerable to such pressure. There are a whole slew of biases that arise because of that type of pressure, and any researcher not very cognizant of their vulnerability is another potential Staple.
I’ve read a few articles on this issue, and the problem seems pretty alarming. From what I understand, there are only a small handful of journals that accept null hypothesis (i.e. hypothesis that X is not true), in fact I think there might be only one (JASN).
The vast majority of journals reject (or at least discourage) non-positive results except in the case of famous researchers or contentious issues, which means studies that show negative results tend to not get published. In fact, most researchers don’t even attempt to publish—they start over, or give up and move to a different project. If the study was critical to their career, they may even move to another field entirely.
This meta-study examines studies of publication bias and reporting bias (unfavorable results omitted from conclusions). It comes to the conclusion that studies that show positive results are significantly more likely to be published than studies that don’t.
If the academic culture is discouraging studies that show negative results via the publication process, doesn’t that seem to imply there is, at the very least, a major inefficiency in our process of learning new things?
I’m no scientist, but I do have to do a lot of troubleshooting for my job, and while knowing what works is most important, knowing what doesn’t takes a close second.
If journals encouraged negative results as much as positive results, I’d imagine we’d see major new scientific breakthroughs twice as often as we currently do—and that’s kind of a big deal, I think. Right now the huge academic pressure is not to produce valid results, but to produce positive results. That’s a major problem, in my opinion, precisely because human beings are very vulnerable to such pressure. There are a whole slew of biases that arise because of that type of pressure, and any researcher not very cognizant of their vulnerability is another potential Staple.