But in the Jaynes example we’re talking about, there are clear observable differences. One had announced that he would continue until he got a certain proportion of success, the other had announced that he would stop at 100.
The key is that Jaynes gives a further piece of data: that somehow we know that “Neither would stoop to falsifying the data”. In Bayesian terms, this information, if reliable, screens out our knowledge that their plans had differed. But in real life, you’re never 100% certain that “neither would stoop to falsifying the data”, especially when there’s often more wiggle room than you’d realize about exactly which data get counted how. In that sense, a rigorous pre-announced plan may be useful evidence about whether there’s funny business going on. The reviled “frequentist” assumptions, then, can be expressed in Bayesian terms as a prior distribution that assumes that researchers cheat whenever the rules aren’t clear. That’s clearly over-pessimistic in many cases (though over-optimistic in others; some researchers cheat even when the rules ARE clear); but, like other heuristics of “significance”, it has some value in developing a “scientific consensus” that doesn’t need to be updated minute-by-minute.
In general: sure, the world is Bayesian. But that doesn’t mean that frequentist math isn’t math. Good frequentist statistics is better than bad Bayesian statistics any day, and anyone who shuts their ears or perks them up just based on a simplistic label is doing themselves a disservice.
How can anyone other than the researchers themselves distinguish between them if their thoughts are private?
I understand “private thoughts” to imply that there are no other observable differences between the two researchers.
But in the Jaynes example we’re talking about, there are clear observable differences. One had announced that he would continue until he got a certain proportion of success, the other had announced that he would stop at 100.
The key is that Jaynes gives a further piece of data: that somehow we know that “Neither would stoop to falsifying the data”. In Bayesian terms, this information, if reliable, screens out our knowledge that their plans had differed. But in real life, you’re never 100% certain that “neither would stoop to falsifying the data”, especially when there’s often more wiggle room than you’d realize about exactly which data get counted how. In that sense, a rigorous pre-announced plan may be useful evidence about whether there’s funny business going on. The reviled “frequentist” assumptions, then, can be expressed in Bayesian terms as a prior distribution that assumes that researchers cheat whenever the rules aren’t clear. That’s clearly over-pessimistic in many cases (though over-optimistic in others; some researchers cheat even when the rules ARE clear); but, like other heuristics of “significance”, it has some value in developing a “scientific consensus” that doesn’t need to be updated minute-by-minute.
In general: sure, the world is Bayesian. But that doesn’t mean that frequentist math isn’t math. Good frequentist statistics is better than bad Bayesian statistics any day, and anyone who shuts their ears or perks them up just based on a simplistic label is doing themselves a disservice.