I guess the context is important here. If the first experiment was wrong, and the second experiment is wrong, will you publish the failure of the second experiment? Will you also publish your suspicion that the first experiment was wrong? How likely will people believe you that your results prove the first experiment was wrong, if you did something else?
In practice, individual scientists like to be able to say “my work causes updates”. If you do something that rests on someone else’s work and the experiment doesn’t come out, you have an incentive to say “Someonewrongonthenet’s hypothesis X implies A and B. Someonewrongonthenet showed A [citation], but I tried B and that means X isn’t completely right.
Cue further investigation which eventually tosses out X. Whether or not A was a false positive is less important than whether or not X is right.
Is there a chance that the process I described was responsible for this?
Yes, that’s possible. I’m not sure direct replication actually solves that issue, though—you’d just shift over to favoring false negatives instead false positives. The existing mechanism that works against this is the incentive to overturn other people’s work.
In practice, individual scientists like to be able to say “my work causes updates”. If you do something that rests on someone else’s work and the experiment doesn’t come out, you have an incentive to say “Someonewrongonthenet’s hypothesis X implies A and B. Someonewrongonthenet showed A [citation], but I tried B and that means X isn’t completely right.
Cue further investigation which eventually tosses out X. Whether or not A was a false positive is less important than whether or not X is right.
Yes, that’s possible. I’m not sure direct replication actually solves that issue, though—you’d just shift over to favoring false negatives instead false positives. The existing mechanism that works against this is the incentive to overturn other people’s work.