So what I’m wondering is whether under frequentism P(hypothesis | data) is actually meaningless. The hypothesis is either true or false and depending on whether its true or not the data has a certain propensity of turning out one way or the other. Its meaningless to ask what the probability of the hypothesis is, you can only ask what the probability of obtaining your data is under certain assumptions.
is correct. Frequentists do indeed claim that P(hypothesis | data) is meaningless for exactly the reasons you gave. However there are some little details in the rest of your post that are incorrect.
null hypothesis (which is the complement of the hypothesis that you are trying to test).
The hypothesis you are trying to test is typically not the complement of the null hypothesis. For example we could have:
H0: theta=0
H1:theta>0
where theta is some variable that we care about. Note that the region theta<0 isn’t in either hypothesis. If we were instead testing
H1′:theta isn’t equal to 0
then frequentists would suggest a different test. They would use a one-tailed test to test H1 and a two-tailed test to test H1′. See here.
P(data | hypothesis) = 1 - p-value
No. This is just mathematically wrong. P(A|B) is not necessarily equal to 1-P(A|¬B). Just think about it for a bit and you’ll see why. If that doesn’t work, take A=”sky is blue” and B=”my car is red” and note that P(A|B)=P(A|¬B)~1.
Your conclusion
is correct. Frequentists do indeed claim that P(hypothesis | data) is meaningless for exactly the reasons you gave. However there are some little details in the rest of your post that are incorrect.
The hypothesis you are trying to test is typically not the complement of the null hypothesis. For example we could have:
where theta is some variable that we care about. Note that the region theta<0 isn’t in either hypothesis. If we were instead testing
then frequentists would suggest a different test. They would use a one-tailed test to test H1 and a two-tailed test to test H1′. See here.
No. This is just mathematically wrong. P(A|B) is not necessarily equal to 1-P(A|¬B). Just think about it for a bit and you’ll see why. If that doesn’t work, take A=”sky is blue” and B=”my car is red” and note that P(A|B)=P(A|¬B)~1.