Tremorbond on Tumblr replies, which I find reasonably compelling:
The probability that theta is exactly 0.25 is not just practically 0 but actually 0. It doesn’t seem at all a problem to me that a bayesian is not able to learn that something they assigned a prior of 0 to is not true.
If you restrict your problem statement to only talk about ranges, the issue disappears. (I’m quite confident but have not checked the math on this.) It only looks wrong because you assume a point value theta, but test the model on ranges of theta.
In general, if you have a smooth distribution over some range of the real numbers, then any individual point in that range will have 0 probability assigned to it, so you can’t expect to come to accurate beliefs about point-values (I think), but you can expect to have accurate beliefs about arbitrary ranges.
Tremorbond on Tumblr replies, which I find reasonably compelling:
In general, if you have a smooth distribution over some range of the real numbers, then any individual point in that range will have 0 probability assigned to it, so you can’t expect to come to accurate beliefs about point-values (I think), but you can expect to have accurate beliefs about arbitrary ranges.