This reminds me of the Replication Crisis. Psychologists placed a high value on novelty and surprising results, the sort of narratives you can build a TED Talk on, and thought that verifying correctness in psychology was cheap and reliable. You didn’t have to eat your veggies, you could just skip straight to the dessert. Turns out, that is not the case: the effects are small, analyst flexibility far higher than known, required sample sizes at least 1 order of magnitude (and often 2) larger than used, and verification is expensive and extremely difficult. The areas of psychology that most prize novelty and cool stories, like social psychology, have turned out to be appallingly riddled with entire fake fields of study and now have a bad tummyache from trying to digest decades of dessert; meanwhile, the most boring, tedious, mathematical areas, like psychophysics, don’t even know what ‘the Replication Crisis’ is.
This reminds me of the Replication Crisis. Psychologists placed a high value on novelty and surprising results, the sort of narratives you can build a TED Talk on, and thought that verifying correctness in psychology was cheap and reliable. You didn’t have to eat your veggies, you could just skip straight to the dessert. Turns out, that is not the case: the effects are small, analyst flexibility far higher than known, required sample sizes at least 1 order of magnitude (and often 2) larger than used, and verification is expensive and extremely difficult. The areas of psychology that most prize novelty and cool stories, like social psychology, have turned out to be appallingly riddled with entire fake fields of study and now have a bad tummyache from trying to digest decades of dessert; meanwhile, the most boring, tedious, mathematical areas, like psychophysics, don’t even know what ‘the Replication Crisis’ is.