I have no idea about what’s done in actual statistical practice, but it seems to make sense to do this:
Publish the likelihood ratio for each correlation. The likelihood ratio for the correlation being real and replicable will be very high.
Since they bothered to do the test, you can figure that people in the know have decently sized prior odds for the association being real and replicable. There must have been animal studies or a biochemical argument or something. Consequently, a high likelihood ratio for this hypothesis may been enough to convinced them—that is, when it’s multiplied with the prior, the resulting posterior may have been high enough to represent the “I’m convinced” state of knowledge.
But the prior odds for the correlation being real and replicable are the same tiny prior odds you would have for any equally unsupported correlation. When they combine the likelihood ratio with their prior odds they do end up with a much higher posterior odds for than they do for other arbitrary-seeming correlations. But, still insignificant.
The critical thing that distinguishes the two hypotheses is whatever previous evidence led them to attempt the test; that’s why the prior for the association is higher. It’s subjective only in the sense that it depends on what you’ve already seen—it doesn’t depend on your thoughts. Whereas, in what Kindly says is the standard solution, you apply a different test depending upon what the researcher’s intentions were.
(I have no idea how you would calculate the prior odds. I mean, Solomonoff induction with your previous observations is the Carnot engine for doing it, but I have no idea how you would actually do it in practice)
I have no idea about what’s done in actual statistical practice, but it seems to make sense to do this:
Publish the likelihood ratio for each correlation. The likelihood ratio for the correlation being real and replicable will be very high.
Since they bothered to do the test, you can figure that people in the know have decently sized prior odds for the association being real and replicable. There must have been animal studies or a biochemical argument or something. Consequently, a high likelihood ratio for this hypothesis may been enough to convinced them—that is, when it’s multiplied with the prior, the resulting posterior may have been high enough to represent the “I’m convinced” state of knowledge.
But the prior odds for the correlation being real and replicable are the same tiny prior odds you would have for any equally unsupported correlation. When they combine the likelihood ratio with their prior odds they do end up with a much higher posterior odds for than they do for other arbitrary-seeming correlations. But, still insignificant.
The critical thing that distinguishes the two hypotheses is whatever previous evidence led them to attempt the test; that’s why the prior for the association is higher. It’s subjective only in the sense that it depends on what you’ve already seen—it doesn’t depend on your thoughts. Whereas, in what Kindly says is the standard solution, you apply a different test depending upon what the researcher’s intentions were.
(I have no idea how you would calculate the prior odds. I mean, Solomonoff induction with your previous observations is the Carnot engine for doing it, but I have no idea how you would actually do it in practice)