While I’m not Ilya, I find the ‘beautiful probability’ discussion somewhat frustrating.
Sure, if we test different hypotheses with the same low sample data, we can get different results. However, starting from different priors, we can also get different results with that same data. Bayesianism won’t let you escape the problem, which is ultimately a problem of data volume.
LW (including myself) is very influenced by ET Jaynes, who believed that for every state of knowledge, there’s a single probability distribution that represents it. Therefore, you’d only get different results from the same data if you started with different knowledge.
It makes a lot of sense for your conclusions to depend on your knowledge. It’s not a problem.
Finding the prior that represents your knowledge is a problem, though.
I’ve read Jaynes (I used to spend long hours trying to explain to a true-believer why I thought MaxEnt was a bad approach to out-of-equilibrium thermo), but my point is that for small sample data, assumptions will (of course) matter. For our frequentist, this means that the experimental specification will lead to small changes in confidence intervals. For the Bayesian this means that the choice of the prior will lead to small changes in credible intervals.
Neither is wrong, and neither is “the one true path”- they are different, equally useful approaches to the same problem.
While I’m not Ilya, I find the ‘beautiful probability’ discussion somewhat frustrating.
Sure, if we test different hypotheses with the same low sample data, we can get different results. However, starting from different priors, we can also get different results with that same data. Bayesianism won’t let you escape the problem, which is ultimately a problem of data volume.
LW (including myself) is very influenced by ET Jaynes, who believed that for every state of knowledge, there’s a single probability distribution that represents it. Therefore, you’d only get different results from the same data if you started with different knowledge.
It makes a lot of sense for your conclusions to depend on your knowledge. It’s not a problem.
Finding the prior that represents your knowledge is a problem, though.
I’ve read Jaynes (I used to spend long hours trying to explain to a true-believer why I thought MaxEnt was a bad approach to out-of-equilibrium thermo), but my point is that for small sample data, assumptions will (of course) matter. For our frequentist, this means that the experimental specification will lead to small changes in confidence intervals. For the Bayesian this means that the choice of the prior will lead to small changes in credible intervals.
Neither is wrong, and neither is “the one true path”- they are different, equally useful approaches to the same problem.