I mean, not only is the “p-value” threshold arbitrary, not only are we depriving ourselves of valuable information by “accepting” or “not accepting” a hypothesis rather than quantifying our certainty level, but...what about P(E|H)?? (Not to mention P(H).)
Well, P(E|H) is actually pretty easy to calculate under a frequentist framework. That’s the basis of power analysis, a topic covered in any good intro stat course. The real missing ingredient, as you point out, is P(H).
I’m not fully fluent in Bayesian statistics, so while I’m on the topic I have a question: do Bayesian methods involve any decision making? In other words, once we’ve calculated P(H|E), do we just leave it at that? No criteria to decide on, just revising of probabilities?
This is my current understanding, but it just seems so contrary to everyday human reasoning. What we would really like to say at the end of the day (or, rather, research program) is something like “Aha! Given the accumulated evidence, we can now cease replication. Hypothesis X must be true.” Being humans, we want to make a decision. But decision making necessarily involves the ultimately arbitrary choice of where to set the criterion. Is this anti-Bayesian?
I don’t think formal decision theory is common in applied Bayesian stats in science; the only paper I can quickly recall that did a decision analysis is Andrew Gelman’s radon remediation study. Maybe econometrics is different, since it’s a lot easier to define losses in that context.
Well, P(E|H) is actually pretty easy to calculate under a frequentist framework. That’s the basis of power analysis, a topic covered in any good intro stat course. The real missing ingredient, as you point out, is P(H).
I’m not fully fluent in Bayesian statistics, so while I’m on the topic I have a question: do Bayesian methods involve any decision making? In other words, once we’ve calculated P(H|E), do we just leave it at that? No criteria to decide on, just revising of probabilities?
This is my current understanding, but it just seems so contrary to everyday human reasoning. What we would really like to say at the end of the day (or, rather, research program) is something like “Aha! Given the accumulated evidence, we can now cease replication. Hypothesis X must be true.” Being humans, we want to make a decision. But decision making necessarily involves the ultimately arbitrary choice of where to set the criterion. Is this anti-Bayesian?
The formal decision-making machinery involves picking a loss function and minimizing posterior expected loss.
Okay, but is it a part of the typical Bayesian routine to wield formal decision theory, or do we just calculate P(H|E) and call it a day?
I don’t think formal decision theory is common in applied Bayesian stats in science; the only paper I can quickly recall that did a decision analysis is Andrew Gelman’s radon remediation study. Maybe econometrics is different, since it’s a lot easier to define losses in that context.