Sorry, I’ll ask a really dumb question here because it’s the middle of the night and my brain doesn’t work. What’s the “official” Bayesian response to this joke (see part 2)? To summarize, when a Bayesian talks about a coin with unknown bias, that involves a prior over possible biases, i.e. a subjective probability distribution over objective probability distributions. But Bayesians are supposed to think that objective probabilities don’t exist (“meaningless speculations about the propensities of different coins”). So how does that make sense?
The coin was made in one of several ways. (Perhaps these ways are parametrized by the ratio of weights between the two sides of the coin.) You have a subjective probability distribution over this set W of possible ways in which the coin was made, according to which p(w) is the probability that the coin was made in way w. This distribution should come from a maximum entropy prior incorporating all your knowledge about the origin of the coin.
Furthermore, for each way w in W, you have a conditional probability p(H | w), which is your subjective probability that the coin will turn up Heads given that the coin was made in way w. This conditional probability distribution incorporates your physical knowledge about how weight ratios in coins influence the dynamics of their flipping.
Finally, you compute the unconditional probability p(H) that the coin will come up heads by summing up the values p(H | w) * p(w) over all ways w in W.
Laplace’s rule of succession is for this problem. Laplace defines orthodox bayesianism, not the heretic (infidel?) you quote. Why shouldn’t I believe that sometimes objective probabilities make sense, even if not all probabilities are objective? In any event, I can choose it as a model of the coin. I have a prior over biases and update it every time I see a flip of the coin. Ahead of time, my prediction for each flip is the same, but after a few flips, my new prediction is different from my old prediction. Of course, I could have other models, like that Alice always flips heads and Bob always flips tails, but if I’m flipping and I know I’m not cheating, then a constant bias seems like a pretty good model to me.
If you have no information, you choose a prior that reflects the fact that you have no information. One way of doing this is the principle of maximum entropy, which tells you in particular that if you’re trying to choose a prior over a parameter that lies in the interval [0, 1] (the bias of the coin), the maximum entropy prior is the uniform prior. If you have other information, you incorporate that into your choice of prior.
Sorry, I’ll ask a really dumb question here because it’s the middle of the night and my brain doesn’t work. What’s the “official” Bayesian response to this joke (see part 2)? To summarize, when a Bayesian talks about a coin with unknown bias, that involves a prior over possible biases, i.e. a subjective probability distribution over objective probability distributions. But Bayesians are supposed to think that objective probabilities don’t exist (“meaningless speculations about the propensities of different coins”). So how does that make sense?
The coin was made in one of several ways. (Perhaps these ways are parametrized by the ratio of weights between the two sides of the coin.) You have a subjective probability distribution over this set W of possible ways in which the coin was made, according to which p(w) is the probability that the coin was made in way w. This distribution should come from a maximum entropy prior incorporating all your knowledge about the origin of the coin.
Furthermore, for each way w in W, you have a conditional probability p(H | w), which is your subjective probability that the coin will turn up Heads given that the coin was made in way w. This conditional probability distribution incorporates your physical knowledge about how weight ratios in coins influence the dynamics of their flipping.
Finally, you compute the unconditional probability p(H) that the coin will come up heads by summing up the values p(H | w) * p(w) over all ways w in W.
Ah, right, my beliefs about each particular kind of coin are also subjective. That’s a good answer, thanks.
Laplace’s rule of succession is for this problem. Laplace defines orthodox bayesianism, not the heretic (infidel?) you quote. Why shouldn’t I believe that sometimes objective probabilities make sense, even if not all probabilities are objective? In any event, I can choose it as a model of the coin. I have a prior over biases and update it every time I see a flip of the coin. Ahead of time, my prediction for each flip is the same, but after a few flips, my new prediction is different from my old prediction. Of course, I could have other models, like that Alice always flips heads and Bob always flips tails, but if I’m flipping and I know I’m not cheating, then a constant bias seems like a pretty good model to me.
Laplace’s rule of succession implicitly relies on a particular choice of prior over possible biases, so I don’t see how this answers the question.
If you have no information, you choose a prior that reflects the fact that you have no information. One way of doing this is the principle of maximum entropy, which tells you in particular that if you’re trying to choose a prior over a parameter that lies in the interval [0, 1] (the bias of the coin), the maximum entropy prior is the uniform prior. If you have other information, you incorporate that into your choice of prior.