I don’t think my first Bayesian critique is “nine nines is too many”; there are physical problems with too much Bayesian confidence (eg “my brain isn’t reliable enough that I should really ever be that confident”), but the simple math of Bayesian probability admits the possibility of nine nines just like anyone else.
I think my first critique is the false dichotomy between the null hypotheses and the hypothesis being tested.
Speaking for the frequentist, you say:
If you roll the die nine times and get nine 10s then you can say that the die is weighted with confidence 0.999999999.
I don’t think this is what a real frequentist says. I think a real frequentist says something more like, the null hypotheses (the fair-dice hypothesis) can be rejected w/ that confidence. Specifically, the number you are giving is 1 - P(evidence|fair). This is not the numerical confidence in the new hypothesis! This confers some confidence to the alternative hypothesis, but that’s better left unquantified, particularly if “falsification is the philosophy of science” as you say. We don’t confirm hypotheses; hypotheses are rejected or left standing.
But whether you wear your heart on your sleeve by naively stating that nine nines is the confidence in the new hypothesis, or carefully hedge your words by only stating that we’ve rejected the null hypothesis of fair dice (and haven’t rejected the alternative, wink wink), still, my critique of the reasoning is going to center around the false dichotomy. Frequentism makes it too easy to bury mistakes under mountains of evidence, because it’s too easy to be right about what’s wrong but wrong about what’s right.
I don’t think my first Bayesian critique is “nine nines is too many”; there are physical problems with too much Bayesian confidence (eg “my brain isn’t reliable enough that I should really ever be that confident”), but the simple math of Bayesian probability admits the possibility of nine nines just like anyone else.
I think my first critique is the false dichotomy between the null hypotheses and the hypothesis being tested.
Speaking for the frequentist, you say:
I don’t think this is what a real frequentist says. I think a real frequentist says something more like, the null hypotheses (the fair-dice hypothesis) can be rejected w/ that confidence. Specifically, the number you are giving is 1 - P(evidence|fair). This is not the numerical confidence in the new hypothesis! This confers some confidence to the alternative hypothesis, but that’s better left unquantified, particularly if “falsification is the philosophy of science” as you say. We don’t confirm hypotheses; hypotheses are rejected or left standing.
But whether you wear your heart on your sleeve by naively stating that nine nines is the confidence in the new hypothesis, or carefully hedge your words by only stating that we’ve rejected the null hypothesis of fair dice (and haven’t rejected the alternative, wink wink), still, my critique of the reasoning is going to center around the false dichotomy. Frequentism makes it too easy to bury mistakes under mountains of evidence, because it’s too easy to be right about what’s wrong but wrong about what’s right.