Contrarian viewpoint: I think this isn’t as bad as the article makes it sound.
It makes sense to pay more attention to positive results. That’s where the breakthroughs are. That’s the reason humans evolved positive bias in the first place.
It makes sense to focus more on new research than on replication The end of science is precise mechanisms explaining how stuff works—if someone throws out a wrong result, we’ll figure out something is wrong via conceptual replication, and maybe then an exact replication might be in order. Exact replication is expensive and benefits are likely lower than the equivalent resources into new research. (Of course, this does not apply when resources are not a limiting factor)
The real-world costs of being wrong in science are low—there are costs, but these costs only increase if you add in replication and stuff. The real world benefits of being right in science are tremendous. The signal-to-noise ratio is somewhat less important than the absolute amount of signal in this scenario.
Not saying that there isn’t room any for improvement, but let’s carefully think about what we want to reform. I don’t think replication is necessarily the answer, and if we are going to pay more attention to null results we have to be smart about how we do it and not err in the opposite direction—as in, you shouldn’t ignore nulls but it really is better to pay more attention to positive results.
(Unless you are doing extremely practical research, as in medicine. Then all of this is really really bad. I’m talking about strictly about research which is still several steps away from a practical implementation)
Contrarian viewpoint: I think this isn’t as bad as the article makes it sound.
It makes sense to pay more attention to positive results. That’s where the breakthroughs are. That’s the reason humans evolved positive bias in the first place.
It makes sense to focus more on new research than on replication The end of science is precise mechanisms explaining how stuff works—if someone throws out a wrong result, we’ll figure out something is wrong via conceptual replication, and maybe then an exact replication might be in order. Exact replication is expensive and benefits are likely lower than the equivalent resources into new research. (Of course, this does not apply when resources are not a limiting factor)
The real-world costs of being wrong in science are low—there are costs, but these costs only increase if you add in replication and stuff. The real world benefits of being right in science are tremendous. The signal-to-noise ratio is somewhat less important than the absolute amount of signal in this scenario.
Not saying that there isn’t room any for improvement, but let’s carefully think about what we want to reform. I don’t think replication is necessarily the answer, and if we are going to pay more attention to null results we have to be smart about how we do it and not err in the opposite direction—as in, you shouldn’t ignore nulls but it really is better to pay more attention to positive results.
(Unless you are doing extremely practical research, as in medicine. Then all of this is really really bad. I’m talking about strictly about research which is still several steps away from a practical implementation)