and I’m sure even more to-the-point sources could be found, though these are no slouch themselves. of course, I was quite literally searching for articles that claim peer review is not working, so there’s an obvious bias to the articles I found! but that’s because I’d recently seen the dec 2022 one. The full metaphor results have a wider variety of nuanced takes, I just selected some that seemed like they made the point I wanted to make, so there’s an additional layer of selection there. nonetheless, it seems like they’re making an interesting point.
This seems important. Can you crystallize more of the causality, from your reading? E.g. is it because peer review creates cabals and entrenched interests who upvote work that makes their work seem “in the hot areas”, or similar? Or because it creates wasteful work for the academics trying to conform to logistical peer review requirements? Or predatory journals select for bad editors? Or it creates an illusion of consensus, obscuring that there are gaping wide open questions? Or...?
Hmm. Perhaps it could be a good idea. However, what do y’all think about the various takes that much of the problem with science boils down to the peer review process? eg:
https://experimentalhistory.substack.com/p/the-rise-and-fall-of-peer-review (dec 2022, general)
https://www.vox.com/2015/12/7/9865086/peer-review-science-problems (2015, general)
https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/peerreview-practices-of-psychological-journals-the-fate-of-published-articles-submitted-again/AFE650EB49A6B17992493DE5E49E4431 (2010, psych replication crisis)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0166387 (2016, biomedical)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/ (2006, general)
and I’m sure even more to-the-point sources could be found, though these are no slouch themselves. of course, I was quite literally searching for articles that claim peer review is not working, so there’s an obvious bias to the articles I found! but that’s because I’d recently seen the dec 2022 one. The full metaphor results have a wider variety of nuanced takes, I just selected some that seemed like they made the point I wanted to make, so there’s an additional layer of selection there. nonetheless, it seems like they’re making an interesting point.
This seems important. Can you crystallize more of the causality, from your reading? E.g. is it because peer review creates cabals and entrenched interests who upvote work that makes their work seem “in the hot areas”, or similar? Or because it creates wasteful work for the academics trying to conform to logistical peer review requirements? Or predatory journals select for bad editors? Or it creates an illusion of consensus, obscuring that there are gaping wide open questions? Or...?
I don’t feel qualified to distill, which is why I did not. I only have a fuzzy grasp of the issue myself. Your hypotheses all seem plausible to me.