Related question: wouldn’t some findings garner replication-style efforts by default once they become important enough? My sense is that once some finding becomes load-bearing enough (e.g. the METR graph), it inevitably receives critical scrutiny (e.g. critiques of the METR graph). What’s the story for why this doesn’t happen? Or perhaps it only happens once the paper is past some threshold of notoriety, meaning there’s a ton of important but un-replicated papers just below that threshold?
meaning there’s a ton of important but un-replicated papers just below that threshold?
Yes! I also think that waiting for something to be maximally load bearing before taking a closer look is bad practice. We want to build up organizational knowledge so we are able to catch things before lots of other research is built upon it.
Related question: wouldn’t some findings garner replication-style efforts by default once they become important enough? My sense is that once some finding becomes load-bearing enough (e.g. the METR graph), it inevitably receives critical scrutiny (e.g. critiques of the METR graph). What’s the story for why this doesn’t happen? Or perhaps it only happens once the paper is past some threshold of notoriety, meaning there’s a ton of important but un-replicated papers just below that threshold?
Yes! I also think that waiting for something to be maximally load bearing before taking a closer look is bad practice. We want to build up organizational knowledge so we are able to catch things before lots of other research is built upon it.