we know that at every intermediate step before the final result the cure rate was less than 70%.
Actually, why? Bessel was worrying whether the rate of cures is greater than 60%, and after the 99th experiment there were either 69 cures out of 99 patients (~69.7%) or 70 cures out of 99 patients (~70.7%), so he could’ve stopped there and that would not hurt his reputation.
IIRC, when we discussed this essay in the reading group, one member said that Eliezer here did a bad job describing the thought experiment, and that actually both experimenters precommitted to treat at least 100 patients.
and that actually both experimenters precommitted to treat at least 100 patients.
That would be an interesting wrinkle. I haven’t read the original source. Supposing this I would think Mr. Frequentist would still say Bessel is still more likely to be fooled by unlikely data than George (in the positive only direction) but honestly only by a very small amount. One could call that the trade-off for a method that won’t be fooled by unlikely negative data.
Actually, why?
I was treating the description of Bessel and having a distinct stopping condition of 70%, otherwise he would have stopped at 69.7% like you said. If he was doing the tests one at a time we know 70.7% at 99 didn’t occur because he stopped at 100.
It can be found (for example) here, pp 12-13. A higher quality but paywalled PDF is here.
The point Jaynes is making is that it does not matter what the stopping rule is. Once the data are obtained, the experimenter’s state of mind is irrelevant to what the data imply about the phenomenon under study.
“Bayesian inference will not get us into this absurd situation, because it perceives automatically what common sense demands; that what is relevant for this inference is not the
relative probabilities of imaginary data sets which were not observed, but the relative likelihoods of different parameter values, based on the one real data set which was observed;
and this is the same for all the experimenters.”
Actually, why? Bessel was worrying whether the rate of cures is greater than 60%, and after the 99th experiment there were either 69 cures out of 99 patients (~69.7%) or 70 cures out of 99 patients (~70.7%), so he could’ve stopped there and that would not hurt his reputation.
IIRC, when we discussed this essay in the reading group, one member said that Eliezer here did a bad job describing the thought experiment, and that actually both experimenters precommitted to treat at least 100 patients.
That would be an interesting wrinkle. I haven’t read the original source. Supposing this I would think Mr. Frequentist would still say Bessel is still more likely to be fooled by unlikely data than George (in the positive only direction) but honestly only by a very small amount. One could call that the trade-off for a method that won’t be fooled by unlikely negative data.
I was treating the description of Bessel and having a distinct stopping condition of 70%, otherwise he would have stopped at 69.7% like you said. If he was doing the tests one at a time we know 70.7% at 99 didn’t occur because he stopped at 100.
It can be found (for example) here, pp 12-13. A higher quality but paywalled PDF is here.
The point Jaynes is making is that it does not matter what the stopping rule is. Once the data are obtained, the experimenter’s state of mind is irrelevant to what the data imply about the phenomenon under study.
“Bayesian inference will not get us into this absurd situation, because it perceives automatically what common sense demands; that what is relevant for this inference is not the relative probabilities of imaginary data sets which were not observed, but the relative likelihoods of different parameter values, based on the one real data set which was observed; and this is the same for all the experimenters.”