What is the likelihood of the “something else” hypothesis? I don’t think this is really a general remedy.
Also, you can get the same thing in the hypothesis testing framework by doing two hypothesis tests, one of which is a comparison to the “something else” hypothesis and one of which is a comparison to the original null hypothesis.
Finally, while I forgot to mention this above, in most cases where hypothesis testing is applied, you actually are considering all possibilities, because you are doing something like P0 = “X ⇐ 0”, P1 = “X > 0″ and these really are logically the only possibilities =) [although I guess often you need to make some assumptions on the probabilistic dependencies among your samples to get good bounds].
Yes, you can say it in that framework. And you should. That’s part of the steelmanning exercise—putting in the things that are missing. If you steelman enough, you get to be a good bayesian.
P0 = “X ⇐ 0” and {All My other assumptions} NOT(P0) = NOT(“X ⇐ 0″) or NOT({All My other assumptions})
What is the likelihood of the “something else” hypothesis? I don’t think this is really a general remedy.
Also, you can get the same thing in the hypothesis testing framework by doing two hypothesis tests, one of which is a comparison to the “something else” hypothesis and one of which is a comparison to the original null hypothesis.
Finally, while I forgot to mention this above, in most cases where hypothesis testing is applied, you actually are considering all possibilities, because you are doing something like P0 = “X ⇐ 0”, P1 = “X > 0″ and these really are logically the only possibilities =) [although I guess often you need to make some assumptions on the probabilistic dependencies among your samples to get good bounds].
Yes, you can say it in that framework. And you should. That’s part of the steelmanning exercise—putting in the things that are missing. If you steelman enough, you get to be a good bayesian.
P0 = “X ⇐ 0” and {All My other assumptions}
NOT(P0) = NOT(“X ⇐ 0″) or NOT({All My other assumptions})