# Two probabilities

Con­sider the fol­low­ing state­ments:

1. The re­sult of this coin flip is heads.

2. There is life on Mars.

3. The mil­lionth digit of pi is odd.

What is the prob­a­bil­ity of each state­ment?

A fre­quen­tist might say, “P1 = 0.5. P2 is ei­ther ep­silon or 1-ep­silon, we don’t know which. P3 is ei­ther 0 or 1, we don’t know which.”

A Bayesian might re­ply, “P1 = P2 = P3 = 0.5. By the way, there’s no such thing as a prob­a­bil­ity of ex­actly 0 or 1.”

Which is right? As with many such long-un­re­solved de­bates, the prob­lem is that two differ­ent con­cepts are be­ing la­beled with the word ‘prob­a­bil­ity’. Let’s sep­a­rate them and re­place P with:

F = the frac­tion of pos­si­ble wor­lds in which a state­ment is true. F can be ex­actly 0 or 1.

B = the Bayesian prob­a­bil­ity that a state­ment is true. B can­not be ex­actly 0 or 1.

Clearly there must be a re­la­tion­ship be­tween the two con­cepts, or the con­fu­sion wouldn’t have arisen in the first place, and there is: apart from both obey­ing var­i­ous laws of prob­a­bil­ity, in the case where we know F but don’t know which world we are in, B = F. That’s what’s go­ing on in case 1. In the other cases, we know F != 0.5, but our ig­no­rance of its ac­tual value makes it rea­son­able to as­sign B = 0.5.

When does the differ­ence mat­ter?

Sup­pose I offer to bet my \$200 the mil­lionth digit of pi is odd, ver­sus your \$100 that it’s even. With B3 = 0.5, that looks like a good bet from your view­point. But you also know F3 = ei­ther 0 or 1. You can also in­fer that I wouldn’t have offered that bet un­less I knew F3 = 1, from which in­fer­ence you are likely to up­date your B3 to more than 23, and de­cline.

On a larger scale, sup­pose we search Mars thor­oughly enough to be con­fi­dent there is no life there. Now we know F2 = ep­silon. Our Bayesian es­ti­mate of the prob­a­bil­ity of life on Europa will also de­cline to­ward 0.

Once we un­der­stand F and B are differ­ent func­tions, there is no con­tra­dic­tion.

• Why does the Bayesian say that the prob­a­bil­ity of there be­ing life on Mars is 0.5?

• We don’t. I’m not sure what’s up with that, un­less it was a de­liber­ately bad ex­am­ple.

• I can’t imag­ine any­one as­sign­ing the event prob­a­bil­ity 0.5 just be­cause it’s a Yes/​No ques­tion. Does the prob­a­bil­ity drop to 13 if I added 1 more op­tion to the ques­tion?

The per­son who as­signs prob­a­bil­ity 1/​k to all out­comes of any ques­tion with k op­tions is NOT a Bayesian. That’s some­one who has mi­s­un­der­stood Bayes rule and should re-read all of Eliezer’s posts.

• “So roughly speak­ing, what are the chances the world is go­ing to be de­stroyed? One in a mil­lion, one in a billion?”

“Well, the best we can say is about a 1 in 2 chance.”

• I seem to re­mem­ber see­ing the idea that “all pos­si­bil­ities equally likely” is sort of a “de­fault prior”. In the case of life on Mars: Imag­ine get­ting all your in­for­ma­tion about life and Mars in lit­tle dribs and drabs, each one of which lets you up­date your prob­a­bil­ity of life on Mars. The place you start from (be­fore you know stuff like what DNA is and whether Mars has an at­mo­sphere) is 0.5.

• We don’t know the an­swer, or even have data for an es­ti­mate, and ig­no­rance trans­lates to B = 0.5. Even if you feel you do have jus­tifi­ca­tion for say­ing e.g. B = 0.2 or 0.8 (I did say might in the origi­nal), B will have a much less ex­treme value than F.

edit: Of all the com­ments I’ve made on LW that I ex­pected to get down voted, this wasn’t one of them. Can the down vot­ers ex­plain your rea­sons for dis­agree­ment?

edit 2: To clar­ify, I’m not claiming we have no data bear­ing on the ques­tion of whether there is life on Mars—of course we have. I’m claiming differ­ent chunks of data sup­port differ­ent con­clu­sions, noth­ing is any­where near be­ing con­clu­sive, and af­ter all of it is added up, the only rea­son­able and im­par­tial con­clu­sion at this time is “I don’t know”, for B some­where in the neigh­bor­hood of 0.5.

• We don’t [...] even have data for an es­ti­mate, and ig­no­rance trans­lates to B = 0.5.

(Down­voted.) No, it doesn’t: we know quite a bit about Mars and about life, surely enough to have some sort of prior prob­a­bil­ity be­fore en­coun­ter­ing spe­cific data. More to the point, we’re never that ig­no­rant. If you liter­ally had no in­for­ma­tion about a topic, then you wouldn’t know enough to even phrase a ques­tion about it, or rec­og­nize an an­swer to such a ques­tion. By ask­ing the ques­tion, you must have some no­tion in your mind of what you’re ask­ing about, and it is from that no­tion that we must draw our prior prob­a­bil­ities, rather than ar­bi­trar­ily pick­ing 0.5. “Whereof we can­not speak, we must pass over in silence.”

• So pri­ors mat­ter if you and I have already up­dated our be­liefs on re­ally di­ver­gent sets of data. But if you’re com­ing at a ques­tion from a place of to­tal ig­no­rance as­sign­ing even pos­si­bil­ities as your pri­ors should work just fine. 0.5 sounds just fine to me. It doesn’t re­ally mat­ter though be­cause as soon as we have any sig­nifi­cant amount of ev­i­dence to up­date on our be­liefs will rapidly con­verge. My prior prob­a­bil­ity for life on Mars could be 99.99, as soon as my rover gets there and doesn’t find any life that num­ber drops dra­mat­i­cally. My prior could also be 0.01 but as soon as I learn about the ge­olog­i­cal in­di­ca­tors that sug­gest early Mars was much like early Earth that is go­ing to go up some­what (and come down again when I show up and don’t see any life) ei­ther way the whole point is that it shouldn’t mat­ter much at all what your pri­ors are.

At least that is how it was ex­plained to me, I could be to­tally off base.

What doesn’t make sense about rwal­lace’s po­si­tion is just that he doesn’t seem to think we have any ev­i­dence to up­date on when it comes to life on Mars.

• I’m not say­ing we have no data on Mars. I’m say­ing we have ev­i­dence one per­son rea­son­ably be­lieves is in fa­vor of life on Mars, and ev­i­dence an­other per­son rea­son­ably be­lieves is against life on Mars; we even have knowl­edge­able sci­en­tists hold­ing very strong opinions on ei­ther side of the is­sue. My con­clu­sion is that when you add it all up, the net ev­i­dence doesn’t jus­tify a po­si­tion very far from 0.5, and to take a po­si­tion like 0.01 or 0.99 is re­ally an ex­pres­sion of per­sonal bias.

• My con­clu­sion is that when you add it all up, the net ev­i­dence doesn’t jus­tify a po­si­tion very far from 0.5, and to take a po­si­tion like 0.01 or 0.99 is re­ally an ex­pres­sion of per­sonal bias.

Well thats an in­ter­est­ing con­clu­sion and maybe some­one has writ­ten some­thing some­where demon­strat­ing that the right pos­te­rior prob­a­bil­ity given our sci­ence is around 0.5. But you can hardly ex­pect your reader to have any idea where that num­ber is com­ing from. 0.5 sounds much too high to me though what I know I ba­si­cally know from gen­eral sci­en­tific knowl­edge and hav­ing done a sci­ence re­port on Mars in the 4th grade.

I agree that 99.99 or 00.01 seem much too ex­treme for es­ti­ma­tions given our ev­i­dence- but they’d func­tion perfectly fine as pri­ors, was my point.

• More to the point, we’re never that ig­no­rant.

Sure we are. A pri­ori, the prob­a­bil­ity that there’s some­thing in­stead of noth­ing is .5.

• What prob­a­bil­ities do you as­sign to the fol­low­ing propo­si­tions?

1. There is life on the north­ern hemi­sphere of Mars.

2. There is life on the south­ern hemi­sphere of Mars.

3. There is life on Mars.

• Good ex­am­ple. I as­sign B = 0.5 in all three cases, but I ex­pect the (un­known) value of F to be very similar (and close to 0 or 1) for all three, un­like in the case of three coin flips.

• The above prob­a­bil­ity as­sess­ments are only co­her­ent if you judge that propo­si­tion 1 has the same truth value as propo­si­tion 2; I don’t know how that could be jus­tified.

• Now, with­out us­ing prob­a­bil­ities of 0 or 1, can you co­her­ently as­sign prob­a­bil­ities to

``````A. 1 and 2
B. 1 and not 2
C. not 1 and 2
D. not 1 and not 2
``````
• Sure. B(A) = B(D) = 0.5, B(B) = B(C) = ep­silon. (The 0.5 is only good to one sig­nifi­cant figure, and even that’s a stretch.)

• Just how small is this ep­silon? I might want to pro­pose a bet.

• If I had a num­ber, I would’ve given the num­ber in­stead of say­ing “ep­silon” :) What’s your pro­posed bet?

• I might bet on B or C against A or D at odds of ep­silon to 1, to be set­tled when we have thor­oughly ex­plored Mars, as­sum­ing that if there is life, we will find it. This of course de­pends on the ac­tual value of ep­silon.

• So ba­si­cally you’re say­ing that the prob­a­bil­ity of there be­ing life on only one of the hemi­spheres is ar­bi­trar­ily small?

• Math­e­mat­i­cally nonzero, but small enough that we can treat it as zero for prac­ti­cal pur­poses, yes.

• Can you link to some of the ma­jor chunks of data sup­port­ing life on Mars?

• Even if ig­no­rance were to­tal, some kind of Oc­cam’s-Ra­zor prior would be ap­pli­ca­ble.

• Put it this way: Of all the things that might have turned out to be con­tro­ver­sial, I’m just sur­prised “we have no idea whether there is life on Mars” turned out to be the one; it still strikes me as fairly ob­vi­ously rea­son­able. Oh well, sur­prises are part of the point of talk­ing to peo­ple :-)

• The prob­lem may be that “we have no idea whether there is life on Mars” sounds a lot like “we have no ev­i­dence about the pres­ence of life on Mars”, which is sim­ply not the case. (Of course, “we have no idea whether there is life on Mars” was op­er­a­tional­ized as a prob­a­bil­ity as­sess­ment of 0.5, which does not im­ply zero ev­i­dence, so I can’t be sure if my re-quo­ta­tion is what you meant.)

• Ah, in­deed the para­phrased ver­sion would be in­cor­rect, and wasn’t what I meant; per­haps I phrased my ver­sion badly. I’ve added a clar­ifi­ca­tion to my ear­lier com­ment, does that help?

• Yup.

• Surely we have some data.

• Right? I’ve seen lots of pic­tures of Mars and not iden­ti­fied any life in any of them!

• F = the frac­tion of pos­si­ble wor­lds in which a state­ment is true. F can be ex­actly 0 or 1.

If “pos­si­ble world” means “any imag­in­able world his­tory which is con­sis­tent with our knowl­edge”, F and B will prob­a­bly col­lapse into one con­cept.

If it means “any world his­tory which is con­sis­tent with logic, some speci­fied set of phys­i­cal laws and ini­tial con­di­tions given in one mo­ment”, and if we ex­pect the laws to be de­ter­minis­tic, then F1 would also be 0 or 1 and not 0.5.

I would like to see your defi­ni­tion of “pos­si­ble world”.

• The fre­quen­tist’s prob­a­bil­ity you re­fer to is called “phys­i­cal prob­a­bil­ity”. See for ex­am­ple http://​​en.wikipe­dia.org/​​wiki/​​Prob­a­bil­ity_interpretations

• Yes, that’s ex­actly what I’m talk­ing about. Thanks for the link.

• A fre­quen­tist might say, “P1 = 0.5. P2 is ei­ther ep­silon or 1-ep­silon, we don’t know which. P3 is ei­ther 0 or 1, we don’t know which.”

I’m not sure why the fre­quen­tist would put an ep­silon in P2. Surely there is a fact of the mat­ter about state­ment 2 just as there is for state­ment 3.

• I was as­sum­ing per Teg­mark that we live in (at least one va­ri­ety of) big world, and “I” de­notes a set of en­tities in­dis­t­in­guish­able with cur­rent in­for­ma­tion, but who live in differ­ent parts of the mul­ti­verse. But more pro­saically, you could note that there is a nonzero albeit small prob­a­bil­ity that atoms on a life­less Mars will ar­range them­selves into a life form be­tween one visit and the next.

• I grant that a big world pro­vides an en­sem­ble such that ep­silon could make sense. I think that the pro­saic ex­pla­na­tion fails, though—a fully speci­fied ver­sion of state­ment two refers ei­ther to an in­stant in time or an in­ter­val, and ei­ther way, in a small world there will be a fact of the mat­ter.

• Hmm, fre­quen­tist prob­a­bil­ity is most usu­ally de­scribed in terms of, er, fre­quency; what frac­tion of the time we will get a given re­sult when we run the test. But if you take it as refer­ring to an in­stant of time (and you as­sume small world and no fuzzi­ness) in that case I agree the ep­silon would dis­ap­pear.

• frac­tion of the time

It’s a minor point, but wack­ily enough, the above quote is a sub­tle equiv­o­ca­tion on the word “time”. I can flip N ex­change­able coins si­mul­ta­neously and count the num­ber of times I see “heads”, and this is perfectly sen­si­ble in the fre­quen­tist in­ter­pre­ta­tion. Phys­i­cal clock time is some­thing else again.

• Sure, B and F don’t di­rectly con­tra­dict each other, but which one should we use when rea­son­ing un­der un­cer­tainty?

edit: bet­ter state­ment of what I was get­ting at

• I an­swered a differ­ent ques­tion than what this sits be­low, but I think that the an­swer is still both. B is prob­a­bly the one that fits in the for­mu­las, but you should also re­mem­ber that the cases where B/​=F are the cases where such for­mu­las are least likely to serve you well.

• Yep. To ex­pand: cor­rectly used, they should not con­tra­dict each other. If they give differ­ent an­swers, then at least one of them is be­ing used in a way in which it is not ap­pli­ca­ble.

• You’re right, so far as it goes, but I don’t think it gets you very far. My point is that this dodges what the de­bate is about. Pro­po­nents of F say that it should be used for do­ing in­duc­tive in­fer­ence while pro­po­nents of B say they’re wrong, B is what should be used. If you’re not an­swer­ing that ques­tion, you’re not set­tling the de­bate.

Now if you’re not try­ing to set­tle the de­bate, then we have no ar­gu­ment.

• Well, the only de­bates I’m claiming to defini­tively set­tle are the philo­soph­i­cal ones about “what does prob­a­bil­ity re­ally mean?”, “are 0 and 1 re­ally prob­a­bil­ities?” and such­like, over which I’ve seen enough elec­trons spilled that I con­sid­ered it well worth try­ing to put them to rest.

But in a typ­i­cal in­duc­tive sce­nario, it seems to me that since we can’t work di­rectly with F, whereas we can work di­rectly with B, Bayesian rea­son­ing is the ap­pro­pri­ate tool to use. Do you have any coun­terex­am­ples in mind where the two ap­proaches give differ­ent an­swers and the differ­ence can’t be re­solved by not­ing that they aren’t an­swer­ing the same ques­tion?

• Do you have any coun­terex­am­ples in mind where the two ap­proaches give differ­ent an­swers and the differ­ence can’t be re­solved by not­ing that they aren’t an­swer­ing the same ques­tion?

Well, no, but but pro­po­nents of F may dis­agree.