But it seems more useful to me to calculate P(hypothesis | data). And that’s not quite the same thing.
It is not the same thing and knowing P(hypothesis | data) would be very useful. Unfortunately, it is also very hard to estimate because usually the best you can do is calculate the probability, given the data, of a hypothesis out of a fixed set of hypotheses which you know about and for which you can estimate probabilities. If your understanding of the true data-generation process is not so good (which is very common in real life) your P(hypothesis | data) is going to be pretty bad and what’s worse, you have no idea how bad it is.
Not having a good grasp on the set of all hypotheses does not distinguish bayesians from frequentists and does not seem to me to motivate any difference in their methodologies.
Added: I don’t think it has much to do with the original comment, but testing a model without specific competition is called “model checking.” It is a common frequentist complaint that bayesians don’t do it. I don’t think that this is an accurate complaint, but it is true that it is easier to fit it into a frequentist framework than a bayesian framework.
I have said nothing about the differences between bayesians and frequentists. I just pointed out some issues with trying to estimate P(hypothesis | data).
It is not the same thing and knowing P(hypothesis | data) would be very useful. Unfortunately, it is also very hard to estimate because usually the best you can do is calculate the probability, given the data, of a hypothesis out of a fixed set of hypotheses which you know about and for which you can estimate probabilities. If your understanding of the true data-generation process is not so good (which is very common in real life) your P(hypothesis | data) is going to be pretty bad and what’s worse, you have no idea how bad it is.
Not having a good grasp on the set of all hypotheses does not distinguish bayesians from frequentists and does not seem to me to motivate any difference in their methodologies.
Added: I don’t think it has much to do with the original comment, but testing a model without specific competition is called “model checking.” It is a common frequentist complaint that bayesians don’t do it. I don’t think that this is an accurate complaint, but it is true that it is easier to fit it into a frequentist framework than a bayesian framework.
I have said nothing about the differences between bayesians and frequentists. I just pointed out some issues with trying to estimate P(hypothesis | data).