Thanks for the replies. Let me rephrase to see if I understood correctly. My problem is that I don’t really have a single degree-of-belief, I have a distribution over failure frequencies, and as I’ve set it up my distribution is a delta function—in effect, I’ve assigned something a `probability’ of 1, which naturally breaks the formula. Instead I ought to have something like a Gaussian, or whatever, with mean 0.03 and signma (let’s say) 0.01. (Of course it won’t be a true Gaussian since it is cut off at 0 and at 1, but that’s a detail.) Then, to calculate my new distribution, I do Bayes at each point, thus:
where P(x) is my Gaussian prior and P(one successful launch) is the integral from 0 to 1 of P(x)(1-x). We can easily see that in the case of the delta function, this reduces to what I have in my OP. In effect I did the arithmetic correctly, but started with a bad prior—you can’t shift yourself away from a prior probability of 1, no matter what evidence you get. We can also see that this procedure will shift the distribution down, towards lower failure probabilities.
Right. But rather than to say that you have started with a bad prior (which you effectively did, bud hadn’t noticed that you had had such a prior) I would say that the confusion stemmed from bad choice of words. You thought about frequency of failures and said probability of failure, which caused you to think that this is what has to be updated. Frequency of failures isn’t a Bayesian probability, it’s an objective property of the system. But once you say “probability of failure”, it appears that the tested hypothesis is “next launch will fail” rather than “frequency of failures is x”. “Next launch will fail” says apparently nothing about this launch, so one intuitively concludes that observing this launch is irrelevant as for that hypothesis, more so if one correctly assumes that failure next time doesn’t causally influence chances for failure this time.
Of course this line of thought is wrong: both launches are instances of the same process and by observing one launch one can learn something which applies to all other launches. But it is easy to overlook if one speaks about probabilities of single event outcomes rather than a general model which includes some objective frequencies. So, before you write down the Bayes’ formula, make sure what hypothesis you are testing and that you don’t mix objective frequencies and subjective probabilities, even if they may be (under some conditions) the same.
(I hope I have described the thought processes correctly. I have experienced the same confusion when I was trying to figure out how Bayesian updating works for the first time.)
Thanks for the replies. Let me rephrase to see if I understood correctly. My problem is that I don’t really have a single degree-of-belief, I have a distribution over failure frequencies, and as I’ve set it up my distribution is a delta function—in effect, I’ve assigned something a `probability’ of 1, which naturally breaks the formula. Instead I ought to have something like a Gaussian, or whatever, with mean 0.03 and signma (let’s say) 0.01. (Of course it won’t be a true Gaussian since it is cut off at 0 and at 1, but that’s a detail.) Then, to calculate my new distribution, I do Bayes at each point, thus:
P(failure rate x | one successful launch) = P(one successful launch | failure rate x) * P(x) / P(one successful launch)
where P(x) is my Gaussian prior and P(one successful launch) is the integral from 0 to 1 of P(x)(1-x). We can easily see that in the case of the delta function, this reduces to what I have in my OP. In effect I did the arithmetic correctly, but started with a bad prior—you can’t shift yourself away from a prior probability of 1, no matter what evidence you get. We can also see that this procedure will shift the distribution down, towards lower failure probabilities.
Thanks for clearing up my confusion. :)
Right. But rather than to say that you have started with a bad prior (which you effectively did, bud hadn’t noticed that you had had such a prior) I would say that the confusion stemmed from bad choice of words. You thought about frequency of failures and said probability of failure, which caused you to think that this is what has to be updated. Frequency of failures isn’t a Bayesian probability, it’s an objective property of the system. But once you say “probability of failure”, it appears that the tested hypothesis is “next launch will fail” rather than “frequency of failures is x”. “Next launch will fail” says apparently nothing about this launch, so one intuitively concludes that observing this launch is irrelevant as for that hypothesis, more so if one correctly assumes that failure next time doesn’t causally influence chances for failure this time.
Of course this line of thought is wrong: both launches are instances of the same process and by observing one launch one can learn something which applies to all other launches. But it is easy to overlook if one speaks about probabilities of single event outcomes rather than a general model which includes some objective frequencies. So, before you write down the Bayes’ formula, make sure what hypothesis you are testing and that you don’t mix objective frequencies and subjective probabilities, even if they may be (under some conditions) the same.
(I hope I have described the thought processes correctly. I have experienced the same confusion when I was trying to figure out how Bayesian updating works for the first time.)
Yes! Exactly right.
By the way, the idea that the if the frequency is known it is equal to the probability, is wittily known as the “Principal Principle”.