https://twitter.com/AlvaroDeMenard/status/1304399452816519169/photo/1 - probability of reproduction from one forecaster for DARPA SCORE study. The distribution is bimodal because the lower hump is p=.05 and the upper hump is some lower p-value like p=.01 or something. It looks like even the higher p values have reproduction rates at .8 to .9 . This updates toward Shem’s skepticism. Even though the p value is very small, reproduction rate is still stuck at .85 for good studies. Since this study has the problem’s shem pointed out, we might expect a reproduction probability lower at like .75. So a likelihood ratio of 3:1.
I could imagine people with knowledge of the subject having sufficiently lower priors.
https://twitter.com/AlvaroDeMenard/status/1304399452816519169/photo/1 - probability of reproduction from one forecaster for DARPA SCORE study. The distribution is bimodal because the lower hump is p=.05 and the upper hump is some lower p-value like p=.01 or something. It looks like even the higher p values have reproduction rates at .8 to .9 . This updates toward Shem’s skepticism. Even though the p value is very small, reproduction rate is still stuck at .85 for good studies. Since this study has the problem’s shem pointed out, we might expect a reproduction probability lower at like .75. So a likelihood ratio of 3:1.
I could imagine people with knowledge of the subject having sufficiently lower priors.