let’s strategically take the risk of one-in-a-million that we discard a true curing method homoeopathy with such and such clinical effect
Where does your choice of “such and such clinical effect” come from? Keeping your one-in-a-million chance of being wrong fixed, the scale of the clinical trials required depends on the effect size of homeopathy. If homeopathy is a guaranteed cure, it’s enough to dose one incurably sick person. If it helps half of the patients, you might need to dose on the order of twenty. And so on for smaller effect sizes. The homeopathy claim is not just a single hypothesis but a compound hypothesis consisting of all these hypotheses. Choosing which of these hypotheses to entertain is a probabilistic judgment; it can’t be escaped by just picking one of the hypotheses, since that’s just concentrating the prior mass at one point.
(Pardon the goofy notation. Don’t want to deal with the LaTeX engine.)
The compound hypothesis is well-defined. Suppose that the baseline cure probability for a placebo is θ ∈ [0,1). Then hypotheses take the form H ⊂ [0,1], which have the interpretation that the cure rate for homeopathy is in H. The standing null hypothesis in this case is Hθ = { θ }. The alternative hypothesis that homeopathy works is H>θ = (θ,1] = { x : x > θ }. For any θ′ ∈ H>θ, we can construct a “one-in-a-million chance of being wrong” test for the simple hypothesis Hθ′ that homeopathy is effective with effect size exactly θ′. It is convenient that such tests work just as well for the hypothesis H≥θ′. However, we can’t construct a test for H>θ.
Bringing in falsifiability only confuses the issue. No clinical data exist that will strictly falsify any of the hypotheses considered above. On the other hand, rejecting Hθ′ seems like it should provide weak support for rejecting H>θ. My take on this is that since such a research program seems to work in practice, falsifiability doesn’t fully describe how science works in this case (see Popper vs. Kuhn, Lakatos, Feyerabend, etc.).
Clinical data still exists that would allow a strategy to stop doing more tests at specific cut off point as the payoff from the hypothesis being right is dependent to the size of the effect and there will be clinical data at some point where the integral of payoff over lost clinical effects is small enough. It just gets fairly annoying to calculate. . Taking the strategy will be similar to gambling decision.
I do agree that there is a place for occam’s razor here but there exist no formalism that actually lets you quantify this weak support. There’s the Solomonoff induction, which is un-computable and awesome for work like putting an upper bound on how good induction can (or rather, can’t) ever be.
Where does your choice of “such and such clinical effect” come from? Keeping your one-in-a-million chance of being wrong fixed, the scale of the clinical trials required depends on the effect size of homeopathy. If homeopathy is a guaranteed cure, it’s enough to dose one incurably sick person. If it helps half of the patients, you might need to dose on the order of twenty. And so on for smaller effect sizes. The homeopathy claim is not just a single hypothesis but a compound hypothesis consisting of all these hypotheses. Choosing which of these hypotheses to entertain is a probabilistic judgment; it can’t be escaped by just picking one of the hypotheses, since that’s just concentrating the prior mass at one point.
It’s part of the hypothesis, without it the idea is not a defined hypothesis. See falsifiability.
(Pardon the goofy notation. Don’t want to deal with the LaTeX engine.)
The compound hypothesis is well-defined. Suppose that the baseline cure probability for a placebo is θ ∈ [0,1). Then hypotheses take the form H ⊂ [0,1], which have the interpretation that the cure rate for homeopathy is in H. The standing null hypothesis in this case is Hθ = { θ }. The alternative hypothesis that homeopathy works is H>θ = (θ,1] = { x : x > θ }. For any θ′ ∈ H>θ, we can construct a “one-in-a-million chance of being wrong” test for the simple hypothesis Hθ′ that homeopathy is effective with effect size exactly θ′. It is convenient that such tests work just as well for the hypothesis H≥θ′. However, we can’t construct a test for H>θ.
Bringing in falsifiability only confuses the issue. No clinical data exist that will strictly falsify any of the hypotheses considered above. On the other hand, rejecting Hθ′ seems like it should provide weak support for rejecting H>θ. My take on this is that since such a research program seems to work in practice, falsifiability doesn’t fully describe how science works in this case (see Popper vs. Kuhn, Lakatos, Feyerabend, etc.).
Clinical data still exists that would allow a strategy to stop doing more tests at specific cut off point as the payoff from the hypothesis being right is dependent to the size of the effect and there will be clinical data at some point where the integral of payoff over lost clinical effects is small enough. It just gets fairly annoying to calculate. . Taking the strategy will be similar to gambling decision.
I do agree that there is a place for occam’s razor here but there exist no formalism that actually lets you quantify this weak support. There’s the Solomonoff induction, which is un-computable and awesome for work like putting an upper bound on how good induction can (or rather, can’t) ever be.