# Bayes’ Theorem Illustrated (My Way)

(This post is el­e­men­tary: it in­tro­duces a sim­ple method of vi­su­al­iz­ing Bayesian calcu­la­tions. In my defense, we’ve had other el­e­men­tary posts be­fore, and they’ve been found use­ful; plus, I’d re­ally like this to be on­line some­where, and it might as well be here.)

I’ll ad­mit, those Monty-Hall-type prob­lems in­vari­ably trip me up. Or at least, they do if I’m not think­ing very care­fully—do­ing quite a bit more work than other peo­ple seem to have to do.

What’s more, peo­ple’s ex­pla­na­tions of how to get the right an­swer have al­most never been satis­fac­tory to me. If I con­cen­trate hard enough, I can usu­ally fol­low the rea­son­ing, sort of; but I never quite “see it”, and nor do I feel equipped to solve similar prob­lems in the fu­ture: it’s as if the solu­tions seem to work only in ret­ro­spect.

Minds work differ­ently, illu­sion of trans­parency, and all that.

For­tu­nately, I even­tu­ally man­aged to iden­tify the source of the prob­lem, and I came up a way of think­ing about—vi­su­al­iz­ing—such prob­lems that suits my own in­tu­ition. Maybe there are oth­ers out there like me; this post is for them.

I’ve men­tioned be­fore that I like to think in very ab­stract terms. What this means in prac­tice is that, if there’s some sim­ple, gen­eral, el­e­gant point to be made, tell it to me right away. Don’t start with some messy con­crete ex­am­ple and at­tempt to “work up­ward”, in the hope that difficult-to-grasp ab­stract con­cepts will be made more palat­able by re­lat­ing them to “real life”. If you do that, I’m li­able to get stuck in the trees and not see the for­est. Chances are, I won’t have much trou­ble un­der­stand­ing the ab­stract con­cepts; “real life”, on the other hand...

...well, let’s just say I pre­fer to start at the top and work down­ward, as a gen­eral rule. Tell me how the trees re­late to the for­est, rather than the other way around.

Many peo­ple have found Eliezer’s In­tu­itive Ex­pla­na­tion of Bayesian Rea­son­ing to be an ex­cel­lent in­tro­duc­tion to Bayes’ the­o­rem, and so I don’t usu­ally hes­i­tate to recom­mend it to oth­ers. But for me per­son­ally, if I didn’t know Bayes’ the­o­rem and you were try­ing to ex­plain it to me, pretty much the worst thing you could do would be to start with some de­tailed sce­nario in­volv­ing breast-can­cer screen­ings. (And not just be­cause it tar­nishes beau­tiful math­e­mat­ics with images of sick­ness and death, ei­ther!)

So what’s the right way to ex­plain Bayes’ the­o­rem to me?

Like this:

We’ve got a bunch of hy­pothe­ses (states the world could be in) and we’re try­ing to figure out which of them is true (that is, which state the world is ac­tu­ally in). As a con­ces­sion to con­crete­ness (and for ease of draw­ing the pic­tures), let’s say we’ve got three (mu­tu­ally ex­clu­sive and ex­haus­tive) hy­pothe­ses—pos­si­ble world-states—which we’ll call H1, H2, and H3. We’ll rep­re­sent these as blobs in space:

Figure 0

Now, we have some prior no­tion of how prob­a­ble each of these hy­pothe­ses is—that is, each has some prior prob­a­bil­ity. If we don’t know any­thing at all that would make one of them more prob­a­ble than an­other, they would each have prob­a­bil­ity 13. To illus­trate a more typ­i­cal situ­a­tion, how­ever, let’s as­sume we have more in­for­ma­tion than that. Speci­fi­cally, let’s sup­pose our prior prob­a­bil­ity dis­tri­bu­tion is as fol­lows: P(H1) = 30%, P(H2)=50%, P(H3) = 20%. We’ll rep­re­sent this by re­siz­ing our blobs ac­cord­ingly:

Figure 1

That’s our prior knowl­edge. Next, we’re go­ing to col­lect some ev­i­dence and up­date our prior prob­a­bil­ity dis­tri­bu­tion to pro­duce a pos­te­rior prob­a­bil­ity dis­tri­bu­tion. Speci­fi­cally, we’re go­ing to run a test. The test we’re go­ing to run has three pos­si­ble out­comes: Re­sult A, Re­sult B, and Re­sult C. Now, since this test hap­pens to have three pos­si­ble re­sults, it would be re­ally nice if the test just flat-out told us which world we were liv­ing in—that is, if (say) Re­sult A meant that H1 was true, Re­sult B meant that H2 was true, and Re­sult 3 meant that H3 was true. Un­for­tu­nately, the real world is messy and com­plex, and things aren’t that sim­ple. In­stead, we’ll sup­pose that each re­sult can oc­cur un­der each hy­poth­e­sis, but that the differ­ent hy­pothe­ses have differ­ent effects on how likely each re­sult is to oc­cur. We’ll as­sume for in­stance that if Hy­poth­e­sis H1 is true, we have a 12 chance of ob­tain­ing Re­sult A, a 13 chance of ob­tain­ing Re­sult B, and a 16 chance of ob­tain­ing Re­sult C; which we’ll write like this:

P(A|H1) = 50%, P(B|H1) = 33.33...%, P(C|H1) = 16.166...%

and illus­trate like this:

Figure 2

(Re­sult A be­ing rep­re­sented by a tri­an­gle, Re­sult B by a square, and Re­sult C by a pen­tagon.)

If Hy­poth­e­sis H2 is true, we’ll as­sume there’s a 10% chance of Re­sult A, a 70% chance of Re­sult B, and a 20% chance of Re­sult C:

Figure 3

(P(A|H2) = 10% , P(B|H2) = 70%, P(C|H2) = 20%)

Fi­nally, we’ll say that if Hy­poth­e­sis H3 is true, there’s a 5% chance of Re­sult A, a 15% chance of Re­sult B, and an 80% chance of Re­sult C:

Figure 4

(P(A|H3) = 5%, P(B|H3) = 15% P(C|H3) = 80%)

Figure 5 be­low thus shows our knowl­edge prior to run­ning the test:

Figure 5

Note that we have now carved up our hy­poth­e­sis-space more finely; our pos­si­ble world-states are now things like “Hy­poth­e­sis H1 is true and Re­sult A oc­curred”, “Hy­poth­e­sis H1 is true and Re­sult B oc­curred”, etc., as op­posed to merely “Hy­poth­e­sis H1 is true”, etc. The num­bers above the slanted line seg­ments—the like­li­hoods of the test re­sults, as­sum­ing the par­tic­u­lar hy­poth­e­sis—rep­re­sent what pro­por­tion of the to­tal prob­a­bil­ity mass as­signed to the hy­poth­e­sis Hn is as­signed to the con­junc­tion of Hy­poth­e­sis Hn and Re­sult X; thus, since P(H1) = 30%, and P(A|H1) = 50%, P(H1 & A) is there­fore 50% of 30%, or, in other words, 15%.

(That’s re­ally all Bayes’ the­o­rem is, right there, but—shh! -- don’t tell any­one yet!)

Now, then, sup­pose we run the test, and we get...Re­sult A.

What do we do? We cut off all the other branches:

Figure 6

So our up­dated prob­a­bil­ity dis­tri­bu­tion now looks like this:

Figure 7

...ex­cept for one thing: prob­a­bil­ities are sup­posed to add up to 100%, not 21%. Well, since we’ve con­di­tioned on Re­sult A, that means that the 21% prob­a­bil­ity mass as­signed to Re­sult A is now the en­tirety of our prob­a­bil­ity mass -- 21% is the new 100%, you might say. So we sim­ply ad­just the num­bers in such a way that they add up to 100% and the pro­por­tions are the same:

Figure 8

There! We’ve just performed a Bayesian up­date. And that’s what it looks like.

Figure 9

then our up­dated prob­a­bil­ity dis­tri­bu­tion would have looked like this:

Figure 10

Similarly, for Re­sult C:

Figure 11

Bayes’ the­o­rem is the for­mula that calcu­lates these up­dated prob­a­bil­ities. Us­ing H to stand for a hy­poth­e­sis (such as H1, H2 or H3), and E a piece of ev­i­dence (such as Re­sult A, Re­sult B, or Re­sult C), it says:

P(H|E) = P(H)*P(E|H)/​P(E)

In words: to calcu­late the up­dated prob­a­bil­ity P(H|E), take the por­tion of the prior prob­a­bil­ity of H that is al­lo­cated to E (i.e. the quan­tity P(H)*P(E|H)), and calcu­late what frac­tion this is of the to­tal prior prob­a­bil­ity of E (i.e. di­vide it by P(E)).

What I like about this way of vi­su­al­iz­ing Bayes’ the­o­rem is that it makes the im­por­tance of prior prob­a­bil­ities—in par­tic­u­lar, the differ­ence be­tween P(H|E) and P(E|H) -- vi­su­ally ob­vi­ous. Thus, in the above ex­am­ple, we eas­ily see that even though P(C|H3) is high (80%), P(H3|C) is much less high (around 51%) -- and once you have as­similated this vi­su­al­iza­tion method, it should be easy to see that even more ex­treme ex­am­ples (e.g. with P(E|H) huge and P(H|E) tiny) could be con­structed.

Now let’s use this to ex­am­ine two tricky prob­a­bil­ity puz­zles, the in­fa­mous Monty Hall Prob­lem and Eliezer’s Draw­ing Two Aces, and see how it illus­trates the cor­rect an­swers, as well as how one might go wrong.

### The Monty Hall Problem

The situ­a­tion is this: you’re a con­tes­tant on a game show seek­ing to win a car. Be­fore you are three doors, one of which con­tains a car, and the other two of which con­tain goats. You will make an ini­tial “guess” at which door con­tains the car—that is, you will se­lect one of the doors, with­out open­ing it. At that point, the host will open a goat-con­tain­ing door from among the two that you did not se­lect. You will then have to de­cide whether to stick with your origi­nal guess and open the door that you origi­nally se­lected, or switch your guess to the re­main­ing un­opened door. The ques­tion is whether it is to your ad­van­tage to switch—that is, whether the car is more likely to be be­hind the re­main­ing un­opened door than be­hind the door you origi­nally guessed.

(If you haven’t thought about this prob­lem be­fore, you may want to try to figure it out be­fore con­tin­u­ing...)

The an­swer is that it is to your ad­van­tage to switch—that, in fact, switch­ing dou­bles the prob­a­bil­ity of win­ning the car.

Peo­ple of­ten find this coun­ter­in­tu­itive when they first en­counter it—where “peo­ple” in­cludes the au­thor of this post. There are two pos­si­ble doors that could con­tain the car; why should one of them be more likely to con­tain it than the other?

As it turns out, while con­struct­ing the di­a­grams for this post, I “re­dis­cov­ered” the er­ror that led me to in­cor­rectly con­clude that there is a 12 chance the car is be­hind the origi­nally-guessed door and a 12 chance it is be­hind the re­main­ing door the host didn’t open. I’ll pre­sent that er­ror first, and then show how to cor­rect it. Here, then, is the wrong solu­tion:

We start out with a perfectly cor­rect di­a­gram show­ing the prior prob­a­bil­ities:

Figure 12

The pos­si­ble hy­pothe­ses are Car in Door 1, Car in Door 2, and Car in Door 3; be­fore the game starts, there is no rea­son to be­lieve any of the three doors is more likely than the oth­ers to con­tain the car, and so each of these hy­pothe­ses has prior prob­a­bil­ity 13.

The game be­gins with our se­lec­tion of a door. That it­self isn’t ev­i­dence about where the car is, of course—we’re as­sum­ing we have no par­tic­u­lar in­for­ma­tion about that, other than that it’s be­hind one of the doors (that’s the whole point of the game!). Once we’ve done that, how­ever, we will then have the op­por­tu­nity to “run a test” to gain some “ex­per­i­men­tal data”: the host will perform his task of open­ing a door that is guaran­teed to con­tain a goat. We’ll rep­re­sent the re­sult Host Opens Door 1 by a tri­an­gle, the re­sult Host Opens Door 2 by a square, and the re­sult Host Opens Door 3 by a pen­tagon—thus carv­ing up our hy­poth­e­sis space more finely into pos­si­bil­ities such as “Car in Door 1 and Host Opens Door 2” , “Car in Door 1 and Host Opens Door 3″, etc:

Figure 13

Be­fore we’ve made our ini­tial se­lec­tion of a door, the host is equally likely to open ei­ther of the goat-con­tain­ing doors. Thus, at the be­gin­ning of the game, the prob­a­bil­ity of each hy­poth­e­sis of the form “Car in Door X and Host Opens Door Y” has a prob­a­bil­ity of 16, as shown. So far, so good; ev­ery­thing is still perfectly cor­rect.

Now we se­lect a door; say we choose Door 2. The host then opens ei­ther Door 1 or Door 3, to re­veal a goat. Let’s sup­pose he opens Door 1; our di­a­gram now looks like this:

Figure 14

But this shows equal prob­a­bil­ities of the car be­ing be­hind Door 2 and Door 3!

Figure 15

Did you catch the mis­take?

Here’s the cor­rect ver­sion:

As soon as we se­lected Door 2, our di­a­gram should have looked like this:

Figure 16

With Door 2 se­lected, the host no longer has the op­tion of open­ing Door 2; if the car is in Door 1, he must open Door 3, and if the car is in Door 3, he must open Door 1. We thus see that if the car is be­hind Door 3, the host is twice as likely to open Door 1 (namely, 100%) as he is if the car is be­hind Door 2 (50%); his open­ing of Door 1 thus con­sti­tutes some ev­i­dence in fa­vor of the hy­poth­e­sis that the car is be­hind Door 3. So, when the host opens Door 1, our pic­ture looks as fol­lows:

Figure 17

which yields the cor­rect up­dated prob­a­bil­ity dis­tri­bu­tion:

Figure 18

### Draw­ing Two Aces

Here is the state­ment of the prob­lem, from Eliezer’s post:

Sup­pose I have a deck of four cards: The ace of spades, the ace of hearts, and two oth­ers (say, 2C and 2D).

You draw two cards at ran­dom.

(...)

Now sup­pose I ask you “Do you have an ace?”

You say “Yes.”

I then say to you: “Choose one of the aces you’re hold­ing at ran­dom (so if you have only one, pick that one). Is it the ace of spades?”

You re­ply “Yes.”

What is the prob­a­bil­ity that you hold two aces?

(Once again, you may want to think about it, if you haven’t already, be­fore con­tin­u­ing...)

Here’s how our pic­ture method an­swers the ques­tion:

Since the per­son hold­ing the cards has at least one ace, the “hy­pothe­ses” (pos­si­ble card com­bi­na­tions) are the five shown be­low:

Figure 19

Each has a prior prob­a­bil­ity of 15, since there’s no rea­son to sup­pose any of them is more likely than any other.

The “test” that will be run is se­lect­ing an ace at ran­dom from the per­son’s hand, and see­ing if it is the ace of spades. The pos­si­ble re­sults are:

Figure 20

Now we run the test, and get the an­swer “YES”; this puts us in the fol­low­ing situ­a­tion:

Figure 21

The to­tal prior prob­a­bil­ity of this situ­a­tion (the YES an­swer) is (1/​6)+(1/​3)+(1/​3) = 56; thus, since 16 is 15 of 56 (that is, (1/​6)/​(5/​6) = 15), our up­dated prob­a­bil­ity is 15 -- which hap­pens to be the same as the prior prob­a­bil­ity. (I won’t bother dis­play­ing the fi­nal post-up­date pic­ture here.)

What this means is that the test we ran did not provide any ad­di­tional in­for­ma­tion about whether the per­son has both aces be­yond sim­ply know­ing that they have at least one ace; we might in fact say that the re­sult of the test is screened off by the an­swer to the first ques­tion (“Do you have an ace?”).

On the other hand, if we had sim­ply asked “Do you have the ace of spades?”, the di­a­gram would have looked like this:

Figure 22

which, upon re­ceiv­ing the an­swer YES, would have be­come:

Figure 23

The to­tal prob­a­bil­ity mass al­lo­cated to YES is 35, and, within that, the spe­cific situ­a­tion of in­ter­est has prob­a­bil­ity 15; hence the up­dated prob­a­bil­ity would be 13.

So a YES an­swer in this ex­per­i­ment, un­like the other, would provide ev­i­dence that the hand con­tains both aces; for if the hand con­tains both aces, the prob­a­bil­ity of a YES an­swer is 100% -- twice as large as it is in the con­trary case (50%), giv­ing a like­li­hood ra­tio of 2:1. By con­trast, in the other ex­per­i­ment, the prob­a­bil­ity of a YES an­swer is only 50% even in the case where the hand con­tains both aces.

This is what peo­ple who try to ex­plain the differ­ence by ut­ter­ing the opaque phrase “a ran­dom se­lec­tion was in­volved!” are ac­tu­ally talk­ing about: the differ­ence between

Figure 24

and

.

Figure 25

The method ex­plained here is far from the only way of vi­su­al­iz­ing Bayesian up­dates, but I feel that it is among the most in­tu­itive.

(I’d like to thank my sister, Vive-ut-Vi­vas, for help with some of the di­a­grams in this post.)

• This is great. I hope other peo­ple aren’t hes­i­tat­ing to make posts be­cause they are too “el­e­men­tary”. Con­tent on Less Wrong doesn’t need to be ad­vanced; it just needs to be Not Wrong.

• In my defense, we’ve had other el­e­men­tary posts be­fore, and they’ve been found use­ful; plus, I’d re­ally like this to be on­line some­where, and it might as well be here.

It’s quite in­ter­est­ing that peo­ple feel a need to defend them­selves in ad­vance when they think their post is el­e­men­tary, but al­most never feel the same obli­ga­tion when the post is sup­pos­edly too hard, or off-topic, or in­ap­pro­pri­ate for other rea­son. More in­ter­est­ing given that we all have prob­a­bly read about the illu­sion of trans­parency. Still, seems that in­clu­sion of this sort of sig­nal­ling is ir­re­sistible, al­though (as the au­thor’s own defense has stated) the ex­pe­rience tells us that such posts usu­ally meet pos­i­tive re­cep­tion.

As for my part of sig­nal­ling, this com­ment was not meant as a crit­i­cism. How­ever I find it more use­ful if peo­ple defend them­selves only af­ter they are crit­i­cised or oth­er­wise at­tacked.

• It’s con­ceiv­able that peo­ple be­ing ner­vous about posts on el­e­men­tary sub­jects means that they’re more care­ful with el­e­men­tary posts, thus ex­plain­ing some frac­tion of the higher qual­ity.

• It is pos­si­ble. How­ever I am not sure that the el­e­men­tary posts have higher av­er­age qual­ity than other posts, if the com­par­i­son is even pos­si­ble. Rather, what strikes me is that you never read “this is spe­cial­ised and com­pli­cated, but nev­er­the­less I de­cided to post it here, be­cause...”

There still ap­par­ently is a per­cep­tion that it’s a shame to write down some rel­a­tively sim­ple truth, and if one wants to, one must have a damned good rea­son. I can un­der­stand the same mechanism in peer re­viewed jour­nals, where the main aim is to im­press the referee and demon­strate the au­thor’s sta­tus, which in­creases chances to get the ar­ti­cle pub­lished. (If the ar­ti­cle is triv­ial, it at least doesn’t do harm to point out that the au­tor knows it.) Although this prac­tice was crit­i­cised here for many times, it seems that it is re­ally difficult to over­come it. But at least we don’t shun posts be­cause they are el­e­men­tary.

• A piece of ad­vice I heard a long time ago, and which has some­times greatly alle­vi­ated the bore­dom of be­ing stuck in a con­fer­ence ses­sion, is this: If you’re not in­ter­ested in what the lec­turer is talk­ing about, study the lec­ture as a demon­stra­tion of how to give a lec­ture.

By this method even an ex­pert can learn from a skilful ex­po­si­tion of fun­da­men­tals.

• That’s more or less the de­fault vi­su­al­iza­tion; un­for­tu­nately it hasn’t proved par­tic­u­larly helpful to me, or at least not as much as the vi­su­al­iza­tion pre­sented here—hence the need for this post.

The method pre­sented in the post has a dis­crete, se­quen­tial, “flip-the-switch” feel to it which I find very suited to my style of think­ing. If I had known how, I would have pre­sented it as an an­i­ma­tion.

• I don’t have a very ad­vanced ground­ing in math, and I’ve been skip­ping over the tech­ni­cal as­pects of the prob­a­bil­ity dis­cus­sions on this blog. I’ve been read­ing less­wrong by men­tally sub­sti­tut­ing “smart” for “Bayesian”, “chang­ing one’s mind” for “up­dat­ing”, and hav­ing to vaguely trust and be­lieve in­stead of ra­tio­nally un­der­stand­ing.

Now I ab­solutely get it. I’ve got the key to the se­quences. Thank you very very much!

• Sigh. Of course I up­voted this, but...

The first part, the ab­stract part, was a joy to read. But the Monty Hall part started get­ting weaker, and the Two Aces part I didn’t bother read­ing at all. What I’d have done differ­ently if your awe­some idea for a post came to me first: re­move the jar­ring false tan­gent in Monty Hall, make all di­a­grams iden­ti­cal in style to the ones in the first part (col­ors, shapes, fonts, lack of bor­ders), never mix per­centages and frac­tions in the same di­a­gram, use can­cer screen­ing as your first mo­ti­vat­ing ex­am­ple, Monty Hall as the sec­ond ex­am­ple, Two Aces as an ex­er­cise for the read­ers—it’s es­sen­tially a var­i­ant of Monty Hall.

Also, in­di­cate more clearly in the Monty Hall prob­lem state­ment that when­ever the host can open two doors, it chooses each of them with prob­a­bil­ity 50%, rather than (say) always open­ing the lower-num­bered one. Without this as­sump­tion the an­swer could be differ­ent.

Sorry for the crit­i­cisms. It’s just my envy and frus­tra­tion talk­ing. Your post had the po­ten­tial to be so com­pletely awe­some, way bet­ter than Eliezer’s ex­pla­na­tion, but the tiny de­tails broke it.

• this does seem like the type of ar­ti­cle that should be a com­mu­nity effort.. per­haps a wiki en­try?

• 2nd’d. Here I would like to en­courage you (=kom­pon­isto) to do a non-minor edit (al­though those are sel­dom here), to give the post the pol­ish it de­serves.

• I don’t get it re­ally. I mean, I get the method, but not the for­mula. Is this use­ful for any­thing though?

Also, a sim­pler method of ex­plain­ing the Monty Hall prob­lem is to think of it if there were more doors. Lets say there were a mil­lion (thats alot [“a lot” gram­mar nazis] of goats.) You pick one and the host el­limi­nates ev­ery other door ex­cept one. The prob­a­bil­ity you picked the right door is one in a mil­lion, but he had to make sure that the door he left un­opened was the one that had the car in it, un­less you picked the one with a car in it, which is a one in a mil­lion chance.

• It might help to read the se­quences, or just read Jaynes. In par­tic­u­lar, one of the cen­tral ideas of the LW ap­proach to ra­tio­nal­ity is that when one en­coun­ters new ev­i­dence one should up­date one’s be­lief struc­ture based on this new ev­i­dence and your es­ti­mates us­ing Bayes’ the­o­rem. Roughly speak­ing, this is in con­trast to what is some­times de­scribed as “tra­di­tional ra­tio­nal­ism” which doesn’t em­pha­size up­dat­ing on each piece of ev­i­dence but rather on up­dat­ing af­ter one has a lot of clearly rele­vant ev­i­dence.

Edit: Recom­men­da­tion of Map-Ter­ri­tory se­quence seems in­cor­rect. Which se­quence is the one to recom­mend here?

• Up­dat­ing your be­lief based on differ­ent pieces of ev­i­dence is use­ful, but (and its a big but) just be­liev­ing strange things based on im­com­plete ev­i­dence is bad. Also, this ne­glects the fact of time. If you had an in­finite amount of time to an­a­lyze ev­ery pos­si­ble sce­nario, you could get away with this, but oth­er­wise you have to just make quick as­sump­tions. Then, in­stead of test­ing wether these as­sump­tions are cor­rect, you just go with them wher­ever it takes you. If only you could “learn how to learn” and use the Bayesian method on differ­ent meth­ods of learn­ing; eg, test out differ­ent heuris­tics and see which ones give the best re­sults. In the end, you find hu­mans already do this to some ex­tent and “tra­di­tional ra­tio­nal­ism” and sci­ence is based off of the end re­sult of this method. Is this mak­ing any sense? Sure, its use­ful in some ab­stract sense and on var­i­ous math prob­lems, but you can’t pro­gram a com­puter this way, nor can you live your life try­ing to com­pute statis­tics like this in your head.

Other than that, I can see differ­ent places where this would be use­ful.

And so it is writ­ten, “Even if you can­not do the math, know­ing that the math ex­ists tells you that the dance step is pre­cise and has no room in it for your whims.”

• I may not be the best per­son to re­ply to this given that I a) am much closer to be­ing a tra­di­tional ra­tio­nal­ist than a Bayesian and b) be­lieve that the dis­tinc­tion be­tween Bayesian ra­tio­nal­ism and tra­di­tional ra­tio­nal­ism is of­ten ex­ag­ger­ated. I’ll try to do my best.

Up­dat­ing your be­lief based on differ­ent pieces of ev­i­dence is use­ful, but (and its a big but) just be­liev­ing strange things based on in­com­plete ev­i­dence is bad.

So how do you tell if a be­lief is strange? Pre­sum­ably if the ev­i­dence points in one di­rec­tion, one shouldn’t re­gard that be­lief as strange. Can you give an ex­am­ple of a be­lief that should con­sid­ered not a good be­lief to have due to strangeness that one could plau­si­bly have a Bayesian ac­cept like this?

Also, this ne­glects the fact of time. If you had an in­finite amount of time to an­a­lyze ev­ery pos­si­ble sce­nario, you could get away with this, but oth­er­wise you have to just make quick as­sump­tions.

Well yes, and no. The Bayesian starts with some set of prior prob­a­bil­ity es­ti­mates, gen­eral heuris­tics about how the world seems to op­er­ate (re­duc­tion­ism and lo­cal­ity would prob­a­bly be high up on the list). Every­one has to deal with the limits on time and other re­sources. That’s why for ex­am­ple, if some­one claims that hop­ping on one foot cures colon can­cer we don’t gen­er­ally bother test­ing it. That’s true for both the Bayesian and the tra­di­tion­al­ist.

Sure, its use­ful in some ab­stract sense and on var­i­ous math prob­lems, but you can’t pro­gram a com­puter this way, nor can you live your life try­ing to com­pute statis­tics like this in your head

I’m cu­ri­ous as to why you claim that you can’t pro­gram a com­puter this way. For ex­am­ple, au­to­matic Bayesian curve fit­ting has been around for al­most 20 years and is a use­ful ma­chine learn­ing mechanism. Sure, it is much more nar­row than ap­ply­ing Bayesi­anism to un­der­stand­ing re­al­ity as a whole, but un­til we crack the gen­eral AI prob­lem, it isn’t clear to me how you can be sure that that’s a fault of the Bayesian end and not the AI end. If we can un­der­stand how to make gen­eral in­tel­li­gences I see no im­me­di­ate rea­son why one couldn’t make them be good Bayesi­ans.

I agree that in gen­eral, try­ing to gen­er­ally com­pute statis­tics in one’s head is difficult. But I don’t see why that rules out do­ing it for the im­por­tant things. No one is claiming to be a perfect Bayesian. I don’t think for ex­am­ple that any Bayesian when walk­ing into a build­ing tries to es­ti­mate the prob­a­bil­ity that the build­ing will im­me­di­ately col­lapse. Maybe they do if the build­ing is very rick­ety look­ing, but oth­er­wise they just think of it as so tiny as to not bother ex­am­in­ing. But Bayesian up­dat­ing is a use­ful way of think­ing about many classes of sci­en­tific is­sues, as well as gen­eral life is­sues (es­ti­mates for how long it will take to get some­where, es­ti­mates of how many peo­ple will at­tend a party based on the num­ber in­vited and the num­ber who RSVPed for ex­am­ple both can be thought of in some­what Bayesian man­ners). More­over, forc­ing one­self to do a Bayesian calcu­la­tion can help bring into the light many es­ti­mates and premises that were oth­er­wise hid­ing be­hind vague­ness or im­plicit struc­tures.

• (re­duc­tion­ism and non-lo­cal­ity would prob­a­bly be high up on the list).

Guess­ing here you mean lo­cal­ity in­stead of non­lo­cal­ity?

• Yes, fixed thank you.

• So how do you tell if a be­lief is strange? Pre­sum­ably if the ev­i­dence points in one di­rec­tion, one shouldn’t re­gard that be­lief as strange. Can you give an ex­am­ple of a be­lief that should con­sid­ered not a good be­lief to have due to strangeness that one could plau­si­bly have a Bayesian ac­cept like this?

Well for ex­am­ple, if you have a situ­a­tion where the ev­i­dence leads you to be­lieve that some­thing is true, and there is an easy, sim­ple, re­li­able test to prove its not true, why would the bayesian method waste its time? Im­mag­ine you wit­ness some­thing which could be pos­si­ble, but its ex­tremely odd. Like grav­ity not work­ing or some­thing. It could be a hal­lu­ci­na­tions, or a glitch if your talk­ing about a com­puter, and there might be an easy way to prove it is or isn’t. Un­der ei­ther scene­rio, whether its a hal­lu­ci­na­tion or re­al­ity is just weird, it makes an as­sump­tion and then has no rea­son to prove whether this is cor­rect. Ac­tu­ally, that might have been a bad ex­am­ple, but pretty much ev­ery sce­nario you can think of, where mak­ing an as­sump­tion can be a bad thing and you can test the as­sump­tions, would work.

I’m cu­ri­ous as to why you claim that you can’t pro­gram a com­puter this way. For ex­am­ple, au­to­matic Bayesian curve fit­ting has been around for al­most 20 years and is a use­ful ma­chine learn­ing mechanism. Sure, it is much more nar­row than ap­ply­ing Bayesi­anism to un­der­stand­ing re­al­ity as a whole, but un­til we crack the gen­eral AI prob­lem, it isn’t clear to me how you can be sure that that’s a fault of the Bayesian end and not the AI end. If we can un­der­stand how to make gen­eral in­tel­li­gences I see no im­me­di­ate rea­son why one couldn’t make them be good Bayesi­ans.

Well if you can’t pro­gram a vi­able AI out of it, then its not a uni­ver­sal truth to ra­tio­nal­ity. Sure, you might be able to use if its com­pli­mented and pow­ered by other mechanisms, but then its not a un­vir­sal truth, is it. That was my point. If it is an im­por­tant tool, then I have no doubt that once we make AI, it will dis­cover it it­self, or may even have it in its origi­nal pro­gram.

• Well for ex­am­ple, if you have a situ­a­tion where the ev­i­dence leads you to be­lieve that some­thing is true, and there is an easy, sim­ple, re­li­able test to prove its not true, why would the bayesian method waste its time? Im­mag­ine you wit­ness some­thing which could be pos­si­ble, but its ex­tremely odd. Like grav­ity not work­ing or some­thing. It could be a hal­lu­ci­na­tions, or a glitch if your talk­ing about a com­puter, and there might be an easy way to prove it is or isn’t. Un­der ei­ther scene­rio, whether its a hal­lu­ci­na­tion or re­al­ity is just weird, it makes an as­sump­tion and then has no rea­son to prove whether this is cor­rect. Ac­tu­ally, that might have been a bad ex­am­ple, but pretty much ev­ery sce­nario you can think of, where mak­ing an as­sump­tion can be a bad thing and you can test the as­sump­tions, would work.

Firstly, pri­ors are im­por­tant; if some­thing has a low prior prob­a­bil­ity, it’s not gen­er­ally go­ing to get to a high prob­a­bil­ity quickly. Se­condly, not all ev­i­dence has the same strength. Re­mem­ber in par­tic­u­lar that the strength of ev­i­dence is mea­sured by the like­li­hood ra­tio. If you see some­thing that could likely be caused by hal­lu­ci­na­tions, that isn’t nec­es­sar­ily very strong ev­i­dence for it; but hal­lu­ci­na­tions are not to­tally ar­bi­trary, IINM. Still, if you wit­ness ob­jects spon­ta­neously float­ing off the ground, even if you know this is an un­likely hal­lu­ci­na­tion, the prior for some sort of grav­ity failure will be so low that the pos­te­rior will prob­a­bly still be very low. Not that those are the only two al­ter­na­tives, of course.

• Well for ex­am­ple, if you have a situ­a­tion where the ev­i­dence leads you to be­lieve that some­thing is true, and there is an easy, sim­ple, re­li­able test to prove its not true, why would the bayesian method waste its time? Im­mag­ine you wit­ness some­thing which could be pos­si­ble, but its ex­tremely odd. Like grav­ity not work­ing or some­thing. It could be a hal­lu­ci­na­tions, or a glitch if your talk­ing about a com­puter, and there might be an easy way to prove it is or isn’t. Un­der ei­ther scene­rio, whether its a hal­lu­ci­na­tion or re­al­ity is just weird, it makes an as­sump­tion and then has no rea­son to prove whether this is cor­rect. Ac­tu­ally, that might have been a bad ex­am­ple, but pretty much ev­ery sce­nario you can think of, where mak­ing an as­sump­tion can be a bad thing and you can test the as­sump­tions, would work.

If there is an “easy, sim­ple, re­li­able test” to de­ter­mine the claim’s truth within a high con­fi­dence, why do you think a Bayesian wouldn’t make that test?

Well if you can’t pro­gram a vi­able AI out of it, then its not a uni­ver­sal truth to ra­tio­nal­ity.

Can you ex­pand your logic for this? In par­tic­u­lar, it seems like you are us­ing a defi­ni­tion of “uni­ver­sal truth to ra­tio­nal­ity” which needs to be ex­panded out.

• If there is an “easy, sim­ple, re­li­able test” to de­ter­mine the claim’s truth within a high con­fi­dence, why do you think a Bayesian wouldn’t make that test?

Be­cause its not a de­ci­sion mak­ing the­ory, but a one that judges prob­a­bil­ity. The bayesian method will ex­am­ine what it has, and de­cide the prob­a­bil­ity of differ­ent situ­a­tions. Other then that, it doesn’t ac­tu­ally do any­thing. It takes an en­tirely differ­ent sys­tem to ac­tu­ally act on the in­for­ma­tion given. If it is a sim­ple sys­tem and just as­sumes to be cor­rect whichever one has the high­est prob­a­bil­ity, then it isn’t go­ing to bother test­ing it.

• The bayesian method will ex­am­ine what it has, and de­cide the prob­a­bil­ity of differ­ent situ­a­tions. Other then that, it doesn’t ac­tu­ally do any­thing. It takes an en­tirely differ­ent sys­tem to ac­tu­ally act on the in­for­ma­tion given. If it is a sim­ple sys­tem and just as­sumes to be cor­rect whichever one has the high­est prob­a­bil­ity, then it isn’t go­ing to bother test­ing it.

But a Bayesian won’t as­sume which one has the high­est prob­a­bil­ity is cor­rect. That’s the one of the whole points of a Bayesian ap­proach, ev­ery claim is prob­a­bil­is­tic. If one claim is more likely than an­other, the Bayesian isn’t go­ing to lie to it­self and say that the most prob­a­ble claim now has a prob­a­bil­ity of 1. That’s not Bayesi­anism. You seem to be en­gag­ing in what may be a form of the mind pro­jec­tion fal­lacy, in that hu­mans of­ten take what seems to be a high prob­a­bil­ity claim and then treat it like it has a much, much higher prob­a­bil­ity (this is due to a va­ri­ety of cog­ni­tive bi­ases such as con­fir­ma­tion bias and be­lief overkill). A good Bayesian doesn’t do that. I don’t know where you are get­ting this no­tion of a “sim­ple sys­tem” that did that. If it did, it wouldn’t be a Bayesian.

• But a Bayesian wont’ as­sume which one has the high­est prob­a­bil­ity is cor­rect. That’s the one of the whole points of a Bayesian ap­proach, ev­ery claim is prob­a­bil­is­tic. If one claim is more likely than an­other, the Bayesian isn’t go­ing to lie to it­self and say that the most prob­a­ble claim now has a prob­a­bil­ity of 1. That’s not Bayesi­anism. You seem to be en­gag­ing in what may be a form of the mind pro­jec­tion fal­lacy, in that hu­mans of­ten take what seems to be a high prob­a­bil­ity claim and then treat it like it has a much, much higher prob­a­bil­ity (this is due to a va­ri­ety of cog­ni­tive bi­ases such as con­fir­ma­tion bias and be­lief overkill). A good Bayesian doesn’t do that. I don’t know where you are get­ting this no­tion of a “sim­ple sys­tem” that did that. If it did, it wouldn’t be a Bayesian.

I’m not ex­actly sure what you mean by all of this. How does a bayesian sys­tem make de­ci­sions if not by just go­ing on its most prob­a­ble hy­poth­e­sis?

• To make de­ci­sions, you com­bine prob­a­bil­ity es­ti­mates of out­comes with a util­ity func­tion, and max­i­mize ex­pected util­ity. A pos­si­bil­ity with very low prob­a­bil­ity may nev­er­the­less change a de­ci­sion, if that pos­si­bil­ity has a large enough effect on util­ity.

• See the re­ply I made to AlephNeil. Also, this still doesn’t change my sce­nario. If theres a way to test a hy­poth­e­sis, I see no rea­son the bayesian method ever would, even if it seems like com­mon sense to look be­fore you leap.

Any­one know why I can only post com­ments ev­ery 8 min­utes? Is the band­width re­ally that bad?

• Bayesi­anism is only a pre­dic­tor; it gets you from prior prob­a­bil­ities plus ev­i­dence to pos­te­rior prob­a­bil­ities. You can use it to eval­u­ate the like­li­hood of state­ments about the out­comes of ac­tions, but it will only ever give you prob­a­bil­ities, not nor­ma­tive state­ments about what you should or shouldn’t do, or what you should or shouldn’t test. To an­swer those ques­tions, you need to add a de­ci­sion the­ory, which lets you rea­son from a util­ity func­tion plus a pre­dic­tor to a strat­egy, and a util­ity func­tion, which takes a de­scrip­tion of an out­come and as­signs a score in­di­cat­ing how much you like it.

The rate-limit on post­ing isn’t be­cause of band­width, it’s to defend against spam­mers who might oth­er­wise try to use scripts to post on ev­ery thread at once. I be­lieve it goes away with karma, but I don’t know what the thresh­old is.

• Any­one know why I can only post com­ments ev­ery 8 min­utes? Is the band­width re­ally that bad?

You face limits on your rate of post­ing if you’re at or be­low 0 karma, which seems to be the case for you. How you got mod­ded down so much, I’m not so sure of.

• How you got mod­ded down so much, I’m not so sure of.

Bold, un­jus­tified poli­ti­cal claims. Bold, un­jus­tified claims that go against con­sen­sus. Bad spel­ling/​gram­mar. Also a Chris­tian, but those com­ments don’t seem to be nega­tive karma.

• I can at­test the be­ing Chris­tian it­self does not seem to make a nega­tive differ­ence. :D

• Upvoted. That took me a minute to get.

• Yeah, I hadn’t been fol­low­ing Houshalter very closely, and the few that I did see weren’t about poli­tics, and seemed at least some­what rea­son­able. (Maybe I should have checked the post­ing his­tory, but I was just say­ing I’m not sure, not that the op­po­site would be prefer­able.)

• Bold, un­jus­tified poli­ti­cal claims.

What bold un­jus­tified poli­ti­cal claims? You do re­al­ise that ev­ery other per­son on this site I’ve met so far has some kind of ex­treme poli­ti­cal view. I thought I was kind of rea­son­able.

Bold, un­jus­tified claims that go against con­sen­sus.

In other words, I dis­agreed with you. I always look for the rea­sons to doubt some­thing or be­lieve in some­thing else be­fore I just “go along with it”.

What’s wrong with my spel­ling/​gram­mar? I dou­ble check ev­ery­thing be­fore I post it!

Also a Christian

You’re per­se­cut­ing me be­cause of my re­li­gion!?

What­ever. I’ll post again in 8 min­utes I guess.

• Whats wrong with my spel­ling/​gram­mar? I dou­ble check ev­ery­thing be­fore I post it!

In this com­ment:

Whats → What’s

Also, ar­guably a miss­ing comma be­fore “I guess”.

• What bold un­jus­tified poli­ti­cal claims? You do re­al­ise that ev­ery other per­son on this site I’ve met so far has some kind of ex­treme poli­ti­cal view. I thought I was kind of rea­son­able.

Em­pha­sis on ‘un­jus­tified’. Ex­am­ple. This sounds awfully flip­pant and sure of your­self—“This sys­tem wouldn’t work at all”. Why do you sup­pose so many peo­ple, in­clud­ing pro­fes­sional poli­ti­cal sci­en­tists /​ poli­ti­cal philoso­phers /​ philoso­phers of law think that it would work? Do you have an amaz­ing in­sight that they’re all miss­ing? Sure, there are peo­ple with many differ­ent po­si­tions on this is­sue, but un­less you’re ac­tu­ally go­ing to join the de­bate and give solid rea­sons, you weren’t re­ally con­tribut­ing any­thing with this com­ment.

Also, com­ments on poli­ti­cal is­sues are dis­cour­aged, as poli­tics is the mind-kil­ler. Un­less you’re re­ally sure your poli­ti­cal com­ment is ap­pro­pri­ate, hold off on post­ing it. And if you’re re­ally sure your poli­ti­cal com­ment is too im­por­tant not to post, you should check to make sure you’re be­ing ra­tio­nal, as that’s a good sign you’re not.

In other words, I dis­agreed with you. I always look for the rea­sons to doubt some­thing or be­lieve in some­thing else be­fore I just “go along with it”.

Again, em­pha­sis on ‘un­jus­tified’. If peo­ple here be­lieve some­thing, there are usu­ally very good rea­sons for it. Go­ing against that with­out at least at­tempt­ing a jus­tifi­ca­tion is not recom­mended. Here are hun­dreds of peo­ple who have spent years try­ing to un­der­stand how to, in gen­eral, be cor­rect about things, and they have man­aged to reach agree­ment on some is­sues. You should be shaken by that, un­less you know pre­cisely where they’ve all gone wrong, and in that case you should say so. If you’re right, they’ll all change their minds.

Also a Christian

Your[sic] per­se­cut­ing me be­cause of my re­li­gion!?

You’ve in­di­cated you have false be­liefs. That is a point against you. Also if you think the world is flat, the moon is made of green cheese, or 2+2=3, and don’t man­age to fix that when some­one tells you you’re wrong, ra­tio­nal­ists will have a lower opinion of you. If you man­age to con­vince them that 2+2=3, then you win back more points than you’ve lost, but it’s prob­a­bly not worth the try.

• Em­pha­sis on ‘un­jus­tified’. Ex­am­ple. This sounds awfully flip­pant and sure of your­self—“This sys­tem wouldn’t work at all”. Why do you sup­pose so many peo­ple, in­clud­ing pro­fes­sional poli­ti­cal sci­en­tists /​ poli­ti­cal philoso­phers /​ philoso­phers of law think that it would work?

Be­cause they don’t!? I was talk­ing about how the FDA is right, the “wouldn’t work at all” is an un­reg­u­lated drug in­dus­try. If you don’t like my opinion, fine, but lots of peo­ple would agree with me in­clud­ing many of those “poli­ti­cal philoso­phers” you speak so highly of.

If you’re right, they’ll all change their minds.

In my ex­pirience, peo­ple rarely change they’re minds af­ter their sure of some­thing. Thats not to say it doesn’t hap­pen, oth­er­wise why would I try. The point of ar­gu­ment is to try to get both peo­ple on the same ground, then they can both choose for them­selves which is right, even if they don’t pub­li­cly ad­mit “defeat”.

You’ve in­di­cated you have false be­liefs.

What if it’s not a false be­lief? It’s alot differ­ent from “2+2=3” or “the world is flat”. Why? Be­cause you can prove those things cor­rect or in­cor­rect.

If you man­age to con­vince them that 2+2=3, then you win back more points than you’ve lost, but it’s prob­a­bly not worth the try.

Clicky

• What if it’s not a false be­lief? It’s alot differ­ent from “2+2=3” or “the world is flat”. Why? Be­cause you can prove those things cor­rect or in­cor­rect.

The ex­tremely low prior prob­a­bil­ity and the to­tal lack of ev­i­dence al­low us, as Bayesi­ans, to dis­miss it as false. Ta­boo the word “proof”, be­cause it’s not use­ful to us in this con­text.

• Be­cause they don’t!? I was talk­ing about how the FDA is right, the “wouldn’t work at all” is an un­reg­u­lated drug in­dus­try. If you don’t like my opinion, fine, but lots of peo­ple would agree with me in­clud­ing many of those “poli­ti­cal philoso­phers” you speak so highly of.

Speak­ing as some­one who thinks that the gen­eral out­line of your point in that thread is the cor­rect con­clu­sion, the prob­lem is you gave zero ev­i­dence or logic for why you would be cor­rect. Sup­pose some­one says “Hey we do things like X right now, but what if we did Y in­stead?” You can’t just re­spond “Y won’t work.” If you say “Y won’t work be­cause of prob­lems A,B, C” or “X works bet­ter than Y be­cause of prob­lems D,E,F” then you’ve got a dis­cus­sion go­ing. But oth­er­wise, all you have is some­one shout­ing “is not”/​”is too.”

What if it’s not a false be­lief? It’s alot differ­ent from “2+2=3” or “the world is flat”. Why? Be­cause you can prove those things cor­rect or in­cor­rect.

If we’re talk­ing about the re­li­gion mat­ter again, which it seems we are, weren’t you already linked to the Mys­te­ri­ous An­swers se­quence? And I’m pretty sure you were ex­plic­itly given this post. Maybe in­stead of wait­ing 8 min­utes to post be­tween that time read some of the things peo­ple have asked you to read? Or maybe spend a few hours just read­ing the se­quences?

Edit: It is pos­si­ble that you are run­ning into prob­lems with in­fer­en­tial dis­tance.

• In my ex­pirience, peo­ple rarely change they’re minds af­ter their sure of some­thing.

That matches my ex­pe­rience ev­ery­where ex­cept Lw.

If you don’t like my opinion, fine, but lots of peo­ple would agree with me in­clud­ing many of those “poli­ti­cal philoso­phers” you speak so highly of.

Again, I did not say I dis­agreed with you, or that peo­ple down­voted you be­cause they dis­agreed with you. Rather, you’re mak­ing a strong poli­ti­cal claim with­out stat­ing any jus­tifi­ca­tion, and not ac­tu­ally con­tribut­ing any­thing in the pro­cess.

What if it’s not a false be­lief? It’s alot differ­ent from “2+2=3” or “the world is flat”. Why? Be­cause you can prove those things cor­rect or in­cor­rect.

There is strong ev­i­dence that the world is not flat. There is also strong ev­i­dence that the Chris­tian God doesn’t ex­ist, and in fact to an in­differ­ent agent the (very al­gorith­mi­cally com­plex) hy­poth­e­sis that the Chris­tian God ex­ists shouldn’t even be ele­vated to the level of at­ten­tion.

Clicky

False—di­vi­sion by zero. You may want to see How to Con­vince Me 2+2=3.

• Bold, un­jus­tified claims that go against con­sen­sus.

In other words, I dis­agreed with you. I always look for the rea­sons to doubt some­thing or be­lieve in some­thing else be­fore I just “go along with it”.

No. In other words, you’ve made claims that as­sume state­ments against con­sen­sus, of­ten with­out even re­al­iz­ing it or giv­ing any jus­tifi­ca­tion when you do so. As I already ex­plained to you, the gen­eral ap­proach at LW has been hashed out quite a bit. Some peo­ple (such as my­self) dis­agree with a fair bit. For ex­am­ple, I’m much closer to be­ing a tra­di­tional ra­tio­nal­ist than a Bayesian ra­tio­nal­ist and I also as­sign a very low prob­a­bil­ity to a Sin­gu­lar­ity-type event. But I’m aware enough to know when I’m op­er­at­ing un­der non-con­sen­sus views so I’m care­ful to be ex­plicit about what those views are and if nec­es­sary, note why I have them. I’m not the only such ex­am­ple. Ali­corn for ex­am­ple (who also replied to this post) has views on moral­ity that are a dis­tinct minor­ity in LW, but Ali­corn is care­ful when­ever these come up to rea­son care­fully and make her premises ex­plicit. Thus, the com­ments are far more likely to be voted up than down.

Your per­se­cut­ing me be­cause of my re­li­gion!?

Well, for the peo­ple com­plain­ing about gram­mar: “Your” → “You’re”

But no, you’ve only men­tioned your re­li­gious views twice I think, and once in pass­ing. The votes down there were I’m pretty sure be­cause your per­sonal re­li­gious view­point was ut­terly be­sides the point be­ing made about the gen­eral LW con­sen­sus.

• How you got mod­ded down so much, I’m not so sure of.

I’m guess­ing that con­fus­ing “too” and “to”, and “its” and “it’s”, con­tributed.

• By the same rea­son you were in­cor­rect in your re­ply to AlephNeil, perform­ing ex­per­i­ments can in­crease util­ity if what course of ac­tion is op­ti­mal is de­pen­dent on which hy­poth­e­sis is most likely.

• If your util­ity func­tion’s goal is to get the most ac­cu­rate hy­poth­e­sis (not act on it) sure. Other­wise, why waste its time test­ing some­thing that it already be­lieves is true? If your goal is to get the high­est “util­ity” as pos­si­ble, then wast­ing time or re­sources, no mat­ter how small, is in­effi­cient. This means that your mov­ing the blame off the bayesian end and to the “util­ity func­tion”, but its still a prob­lem.

• If your util­ity func­tion’s goal is to get the most ac­cu­rate hy­poth­e­sis (not act on it) sure. Other­wise, why waste its time test­ing some­thing that it already be­lieves is true? If your goal is to get the high­est “util­ity” as pos­si­ble, then wast­ing time or re­sources, no mat­ter how small, is in­effi­cient. This means that your mov­ing the blame off the bayesian end and to the “util­ity func­tion”, but its still a prob­lem.

But you don’t be­lieve it is true; there’s some prob­a­bil­ity as­so­ci­ated with it. Con­sider for ex­am­ple, the fol­low­ing situ­a­tion. Your friend rolls a stan­dard pair of 6 sided dice with­out you see­ing them. If you guess the cor­rect to­tal you get \$1000. Now, it is clear that your best guess is to guess 7 since that is the most com­mon out­come. So you guess 7 and 1/​6th of the time you get it right.

Now, sup­pose you have the slightly differ­ent game where be­fore you make your guess, you may pay your friend \$1 and the friend will tell you the low­est num­ber that ap­peared. You seem to think that for some rea­son a Bayesian wouldn’t do this be­cause they already know that 7 is most likely. But of course they would, be­cause pay­ing the \$1 in­creases their ex­pected pay-off.

In gen­eral, in­creas­ing the ac­cu­racy of your map of the uni­verse is likely to in­crease your util­ity. Some­times it isn’t, and so we don’t bother. Nei­ther a Bayesian ra­tio­nal­ist nor a tra­di­tional ra­tio­nal­ist is go­ing to try to say count all the bricks on the fa­cade of their apart­ment build­ing even though it in­creases the ac­cu­racy of their model. Be­cause this isn’t an in­ter­est­ing piece of the model that is at all likely to tell any­thing use­ful com­pared to other limited forms. If one was an im­mor­tal and re­ally run­ning low on things to do, maybe count­ing that would be a high pri­or­ity.

• Allright, con­sider a situ­a­tion where there is a very very small prob­a­bil­ity that some­thing will work, but it gives in­finite util­ity (or at least ex­tror­di­nar­ily large.) The risk for do­ing it is also re­ally high, but be­cause it is finite, the bayesian util­ity func­tion will eval­u­ate it as ac­cept­able be­cause of the in­finite re­ward in­volved. On pa­per, this works out. If you do it enough times, you suc­ceed and af­ter you sub­tract the to­tal cost from all those other times, you still have in­finity. But in prac­tice most peo­ple con­sider this a very bad course of ac­tion. The risk can be very high, per­haps your life, so even the tra­di­tional ra­tio­nal­ist would avoid do­ing this. Do you see where the prob­lem is? It’s the fact that you only get a finite num­ber of tries in re­al­ity, but the bayesian util­ity func­tion calcu­lates it as though you did it an in­finite num­ber of times and gives you the net util­ity.

• Yes, you aren’t the first per­son to make this ob­ser­va­tion. How­ever, This isn’t a prob­lem with Bayesi­anism so much as with util­i­tar­i­anism giv­ing counter-in­tu­itive re­sults when large num­bers are in­volved. See for ex­am­ple Tor­ture v. dust specks or Pas­cal’s Mug­ging. See es­pe­cially Nyarlathotep’s Deal which is very close to the situ­a­tion you are talk­ing about and shows that the prob­lem seem to more reside in util­i­tar­i­anism than Bayesi­anism. It may very well be that hu­man prefer­ences are just in­con­sis­tent. But this is­sue has very lit­tle to do with Bayesi­anism.

• This isn’t a prob­lem with Bayesi­anism so much as with util­i­tar­i­anism giv­ing counter-in­tu­itive re­sults when large num­bers are in­volved.

Counter-in­tu­itive!? Thats a lit­tle more than just counter-in­tu­itive. Im­mag­ine the CEV uses this func­tion. Doc­tor Evil ap­proaches it and says that an in­finite num­ber of hu­mans will be sac­ri­ficed if it doesn’t let him rule the world. And there are a lot more re­al­is­tic prob­lems like that to. I think the prob­lem comes from the fact that net util­ity of all pos­si­ble wor­lds and ac­tual util­ity are not the same thing. I don’t know how to do it bet­ter, but you might want to think twice be­fore you use this to make trade offs.

• It would help if you read the links peo­ple give you. The situ­a­tion you’ve named is es­sen­tially that in Pas­cal’s Mug­ging.

• It would help if you read the links peo­ple give you. The situ­a­tion you’ve named is es­sen­tially that in Pas­cal’s Mug­ging.

Ac­tu­ally I did. Thats where I got it (af­ter you linked it). And af­ter read­ing all of that, I still can’t find a uni­ver­sal solu­tion to this prob­lem.

• Ah. It seemed like you hadn’t be­cause rather than use the ex­am­ple there you used a very similar case. I don’t know a uni­ver­sal solu­tion ei­ther. But it should be clear that the prob­lem ex­ists for non-Bayesi­ans so the dilemma isn’t a prob­lem with Bayesi­anism.

• My guess at what’s go­ing on here is that you’re in­tu­itively mod­el­ing your­self as hav­ing a bounded util­ity func­tion. In which case (let­ting N de­note an up­per bound on your util­ity), no gam­ble where the prob­a­bil­ity of the “good” out­come is less than −1/​N times the util­ity of the “bad” out­come could ever be worth tak­ing. Or, trans­lated into plain English: there are some risks such that no re­ward could make them worth it—which, you’ll note, is a con­straint on re­wards.

• I’m not sure I un­der­stand. Why put a con­straint on the re­ward, and even if you do, why pick some ar­bi­trary value?

• That’s my ques­tion for you! I was at­tempt­ing to ex­plain the in­tu­ition that gen­er­ated these re­marks of yours:

The risk for do­ing it is also re­ally high, but… the bayesian util­ity func­tion will eval­u­ate it as ac­cept­able be­cause of the [ex­traor­di­nar­ily large] re­ward in­volved. On pa­per, this works out...But in prac­tice most peo­ple con­sider this a very bad course of action

• Other­wise, why waste its time test­ing some­thing that it already be­lieves is true?

Be­cause it might be false. If your util­ity func­tion re­quires you to col­lect green cheese, and so you want to make a plan to go to the moon to col­lect the green cheese, you should know how much you’ll have to spend get­ting to the moon, and what the moon is ac­tu­ally made of. And so it is writ­ten, “If you fail to achieve a cor­rect an­swer, it is fu­tile to protest that you acted with pro­pri­ety.”

• You try to max­i­mize your ex­pected util­ity. Per­haps hav­ing done your calcu­la­tions, you think that ac­tion X has a 56 chance of earn­ing you £1 and a 16 chance of kil­ling you (per­haps some­one’s promised you £1 if you play Rus­sian Roulette).

Pre­sum­ably you don’t base your de­ci­sion en­tirely on the most likely out­come.

• So in this sce­nario you have to de­cide how much your life is worth in money. You can go home and not take any chance of dy­ing or risk a 16 chance to earn X amount of money. Its an ex­ten­sion on the risk/​re­ward prob­lem ba­si­cally, and you have to de­cide how much risk is worth in money be­fore you can com­plete it. Thats a prob­lem, be­cause as far as I know, bayesi­anism doesn’t cover that.

1. It’s not the job of ‘Bayesi­anism’ to tell you what your util­ity func­tion is.

2. This [by which I mean, “the ques­tion of where the agent’s util­ity func­tion comes from”] doesn’t have any­thing to do with the ques­tion of whether Bayesian de­ci­sion-mak­ing takes ac­count of more than just the most prob­a­ble hy­poth­e­sis.

• Is this use­ful for any­thing though?

Only for the stated pur­pose of this web­site—to be “less wrong”! :) Quot­ing from Science Isn’t Strict Enough:

But the Way of Bayes is also much harder to use than Science. It puts a tremen­dous strain on your abil­ity to hear tiny false notes, where Science only de­mands that you no­tice an anvil dropped on your head.

In Science you can make a mis­take or two, and an­other ex­per­i­ment will come by and cor­rect you; at worst you waste a cou­ple of decades.

But if you try to use Bayes even qual­i­ta­tively—if you try to do the thing that Science doesn’t trust you to do, and rea­son ra­tio­nally in the ab­sence of over­whelming ev­i­dence—it is like math, in that a sin­gle er­ror in a hun­dred steps can carry you any­where. It de­mands light­ness, even­ness, pre­ci­sion, perfec­tion­ism.

There’s a good rea­son why Science doesn’t trust sci­en­tists to do this sort of thing, and asks for fur­ther ex­per­i­men­tal proof even af­ter some­one claims they’ve worked out the right an­swer based on hints and logic.

But if you would rather not waste ten years try­ing to prove the wrong the­ory, you’ll need to es­say the vastly more difficult prob­lem: listen­ing to ev­i­dence that doesn’t shout in your ear.

As for the rest of your com­ment: I com­pletely agree! That was ac­tu­ally the ex­pla­na­tion that the OP, kom­pon­isto, gave to me to get Bayesi­anism (edit: I ac­tu­ally mean “the idea that prob­a­bil­ity the­ory can be used to over­ride your in­tu­itions and get to cor­rect an­swers”) to “click” for me (in­so­far as it has “clicked”). But the way that it’s rep­re­sented in the post is re­ally helpful, I think, be­cause it elimi­nates even the need to imag­ine that there are more doors; it ad­dresses the speci­fics of that ac­tual prob­lem, and you can’t ar­gue with the num­bers!

• I don’t get it re­ally. I mean, I get the method, but not the for­mula. Is this use­ful for any­thing though?

Quite a bit! (A quick Google Scholar search turns up about 1500 pa­pers on meth­ods and ap­pli­ca­tions, and there are surely more.)

The for­mula tells you how to change your strength of be­lief in a hy­poth­e­sis in re­sponse to ev­i­dence (this is ‘Bayesian up­dat­ing’, some­times short­ened to just ‘up­dat­ing’). Be­cause the for­mula is a triv­ial con­se­quence of the defi­ni­tion of a con­di­tional prob­a­bil­ity, it holds in any situ­a­tion where you can quan­tify the ev­i­dence and the strength of your be­liefs as prob­a­bil­ities. This is why many of the peo­ple on this web­site treat it as the foun­da­tion of rea­son­ing from ev­i­dence; the for­mula is very gen­eral.

Eliezer Yud­kowsky’s In­tu­itive Ex­pla­na­tion of Bayes’ The­o­rem page goes into this in more de­tail and at a slower pace. It has a few nice Java ap­plets that you can use to play with some of the ideas with spe­cific ex­am­ples, too.

• I don’t get it re­ally. I mean, I get the method, but not the for­mula. Is this use­ful for any­thing though?

There’s a sig­nifi­cant pop­u­la­tion of peo­ple—dis­pro­por­tionately rep­re­sented here—who con­sider Bayesian rea­son­ing to be the­o­ret­i­cally su­pe­rior to the ad hoc meth­ods ha­bit­u­ally used. An in­tro­duc­tory es­say on the sub­ject that many peo­ple here read and agreed with A Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Ex­pla­na­tion.

• Also, a sim­pler method of ex­plain­ing the Monty Hall prob­lem is to think of it if there were more doors. Lets say there were a mil­lion (thats alot [“a lot” gram­mar nazis] of goats.) You pick one and the host el­limi­nates ev­ery other door ex­cept one. The prob­a­bil­ity you picked the right door is one in a mil­lion, but he had to make sure that the door he left un­opened was the one that had the car in it, un­less you picked the one with a car in it, which is a one in a mil­lion chance.

That’s awe­some. I shall use it in the fu­ture. Wish I could multi up­vote.

I think the strug­gle is that peo­ple tend to dis­miss the ex­is­tance of the 3rd door once they see what’s be­hind it. It sort of drops out of the pic­ture as a re­solved thing and then the mind er­ro­neously re­for­mu­lates the situ­a­tion with just the two re­main­ing doors. The scary thing is that peo­ple are gen­er­ally quite eas­ily ma­nipu­lated with these sorts of puz­zles and there are plenty of cir­cum­stances (DNA ev­i­dence given dur­ing jury tri­als comes to mind) when the prob­a­bil­ities be­ing pre­sented are wildly mis­lead­ing as the re­sult of er­ro­neously elimi­nat­ing seg­ments of the prob­lem space be­cause they are “known”.

• One more ap­pli­ca­tion of Bayes I should have men­tioned: Au­mann’s Agree­ment The­o­rem.

• 3 Jun 2010 12:35 UTC
9 points

This is a fan­tas­tic ex­pla­na­tion (which I like bet­ter than the ‘sim­ple’ ex­pla­na­tion re­tired urol­o­gist links to be­low), and I’ll tell you why.

You’ve trans­formed the the­o­rem into a spa­tial rep­re­sen­ta­tion, which is always great—since I rarely use Bayes The­o­rem I have to es­sen­tially ‘re­con­struct’ how to ap­ply it ev­ery time I want to think about it, and I can do that much eas­ier (and with many fewer steps) with a pic­ture like this than with an ex­am­ple like breast can­cer (which is what I would do pre­vi­ously).

Crit­i­cally, you’ve rep­re­sented the WHOLE prob­lem vi­su­ally—all I have to do is pic­ture it in my head and I can ‘read’ di­rectly off of it, I don’t have to think about any other con­cepts or re­mem­ber what cer­tain sym­bols mean. Another plus, you’ve in­cluded the ac­tual num­bers used for max­i­mum trans­parency into what trans­for­ma­tions are ac­tu­ally tak­ing place. It’s a very well done se­ries of di­a­grams.

If I had one (minor) quib­ble, it would be that you should rep­re­sent the prob­a­bil­ities for var­i­ous hy­pothe­ses oc­cur­ing vi­su­ally as well—per­haps us­ing line weights, or split lines like in this di­a­gram.

But very well done, thank you.

(edit: I’d also agree with cousin_it that the first half of the post is the stronger part. The di­a­grams are what make this so great, so stick with them!)

• Won­der­ful. Are you aware of the Tues­day Boy prob­lem? I think it could have been a more im­pres­sive sec­ond ex­am­ple.

“I have two chil­dren. One is a boy born on a Tues­day. What is the prob­a­bil­ity I have two boys?”

(The in­tended in­ter­pre­ta­tion is that I have two chil­dren, and at least one of them is a boy-born-on-a-Tues­day.)

I found it here: Magic num­bers: A meet­ing of math­emag­i­cal tricksters

• I always much pre­fer these stated as ques­tions—you stop some­one and say “Do you have ex­actly two chil­dren? Is at least one of them a boy born on a Tues­day?” and they say “yes”. Other­wise you get into won­der­ing what the prob­a­bil­ity they’d say such a strange thing given var­i­ous fam­ily se­tups might be, which isn’t pre­cisely defined enough...

• Very true. The ar­ti­cle DanielVarga linked to says:

If you have two chil­dren, and one is a boy, then the prob­a­bil­ity of hav­ing two boys is sig­nifi­cantly differ­ent if you sup­ply the ex­tra in­for­ma­tion that the boy was born on a Tues­day. Don’t be­lieve me? We’ll get to the an­swer later.

… which is just wrong: whether it is differ­ent de­pends on how the in­for­ma­tion was ob­tained. If it was:

-- On which day of the week was your youngest boy born ?

-- On a Tues­day.

… then there’s zero new in­for­ma­tion, so the prob­a­bil­ity stays the same, 1/​3rd.

(ETA: ac­tu­ally, to be closer to the origi­nal prob­lem, it should be “Select one of your sons at ran­dom and tell me the day he was born”, but the re­sult is the same.)

• I think the only rea­son­able in­ter­pre­ta­tion of the text is clear since oth­er­wise other stan­dard prob­lems would be am­bigu­ous as well:

“What is prob­a­bil­ity that a per­son’s ran­dom coin toss is tails?”

It does not mat­ter whether you get the in­for­ma­tion from an ex­per­i­menter by ask­ing “Tell me the re­sult of your flip!” or “Did you get tails?”. You just have to stick to the origi­nal text (tails) when you eval­u­ate the an­swer in ei­ther case.

[[EDIT] I think I mis­in­ter­preted your com­ment. I agree that Daniel’s in­tro­duc­tion was am­bigu­ous for the rea­sons you have given.

Still the word­ing “I have two chil­dren, and at least one of them is a boy-born-on-a-Tues­day.” he has given clar­ifies it (and makes it well defined un­der the stan­dard as­sump­tions of in­differ­ence).

• Yes­ter­day I told the prob­lem to a smart non-math-geek friend, and he to­tally couldn’t re­late to this “only rea­son­able in­ter­pre­ta­tion”. He com­pletely un­der­stood the ar­gu­ment lead­ing to 1327, but just couldn’t un­der­stand why do we as­sume that the pre­sen­ter is a ran­domly cho­sen mem­ber of the pop­u­la­tion he claims him­self to be a mem­ber of. That sounded like a com­pletely base­less as­sump­tion to him, that leads to fac­tu­ally in­cor­rect re­sults. He even un­der­stood that as­sum­ing it is our only choice if we want to get a well-defined math prob­lem, and it is the only way to uti­lize all the in­for­ma­tion pre­sented to us in the puz­zle. But all this was not enough to con­vince him that he should as­sume some­thing so stupid.

• I get that as­sum­ing that gen­ders and days of the week are equiprob­a­ble, of all the peo­ple with ex­actly two chil­dren, at least one of whom is a boy born on a Tues­day, 1327 have two boys.

• True, but if you go around ask­ing peo­ple-with-two-chi­dren-at-least-one-of-which-is-a-boy “Select one of your sons at ran­dom, and tell me the day of the week on which he was born”, among those who an­swer “Tues­day”, one-third will have two boys.

(for a suffi­ciently large set of peo­ple-with-two-chi­dren-at-least-one-of-which-is-a-boy who an­swer your ques­tion in­stead of giv­ing you a weird look)

I’m just say­ing that the ar­ti­cle used an im­pre­cise for­mu­la­tion, that could be in­ter­preted in differ­ent ways—es­pe­cially the bit “if you sup­ply the ex­tra in­for­ma­tion that the boy was born on a Tues­day”, which is why ask­ing ques­tions the way you did is bet­ter.

• The Tues­day Boy was loads of fun to think about the first time I came across it—thanks to the par­ent com­ment. I worked through the calcu­la­tions with my 14yo son, on a long metro ride, as a way to test my un­der­stand­ing—he seemed to be fol­low­ing along fine.

The dis­cus­sion in the com­ments on this blog on Az­i­muth has, how­ever, taken the delight to an en­tirely new level. Just wanted to share.

• It seems to me that the stan­dard solu­tions don’t ac­count for the fact that there are a non-triv­ial num­ber of fam­i­lies who are more likely to have a 3rd child, if the first two chil­dren are of the same sex. Some peo­ple have a sex-de­pen­dent stop­ping rule.

P(first two chil­dren differ­ent sexes | you have ex­actly two chil­dren) > P(first two chil­dren differ­ent sexes | you have more than two chil­dren)

The other is­sue with this kind of prob­lem is the am­bi­guity. What was the dis­clo­sure al­gorithm? How did you de­cide which child to give me in­for­ma­tion about? Without that knowl­edge, we are left to spec­u­late.

• This is­sue is also some­times raised in cul­tures where male chil­dren are much more highly prized by par­ents.

Most peo­ple falsely as­sume that such a bias, as it stands, changes gen­der ra­tios for the so­ciety, but its only real effect is that cor­re­spond­ingly larger and rarer fam­i­lies have lots of girls. Such so­cieties typ­i­cally do have weird gen­der ra­tios, but this is mostly due to higher death rates be­fore birth be­cause of se­lec­tive abor­tion, or af­ter birth be­cause some par­ents in such so­cieties feed girls less, teach them less, work them more, and take them to the doc­tor less.

Sup­pose the rules for de­cid­ing to have a child with­out se­lec­tive abor­tion (and so with ba­si­cally 5050 odds of ei­ther gen­der) and no un­fair­ness post-birth were: If you have a boy, stop; if you have no boy but have fewer than N chil­dren, have an­other. In a sce­nario where N > 2, two child fam­i­lies are ei­ther a girl and a boy, or two girls dur­ing a pe­riod when their par­ents still in­tend to have a third. Be­cause that win­dow is rel­a­tively small rel­a­tive to the length of time that fam­i­lies ex­ist to be sam­pled, most two child fam­i­lies (>90%?) would be gen­der bal­anced.

Gen­er­ally, my im­pres­sion is that parental prefer­ences for one or the other sex (or for gen­der bal­ance) are gen­er­ally out of bounds in these kinds of ques­tions be­cause we’re sup­posed to as­sume pla­ton­i­cly perfect fam­ily gen­er­at­ing pro­cesses with ex­act 5050 odds, and no parental bi­ases, and so on. My im­pres­sion is that cul­tural liter­acy is sup­posed to sup­ply the pla­tonic model. If non-pla­tonic as­sump­tions are op­er­at­ing then differ­ent an­swers are ex­pected as differ­ent peo­ple bring in differ­ent ev­i­dence (like prob­a­bil­ities of ly­ing and so forth). If real world fac­tors sneak in later with pla­tonic as­sump­tions al­lowed to stand then its a case of a bad teacher who ex­pects you to guess the pass­word of pre­cisely which ev­i­dence they want to be im­ported, and which ex­cluded.

This is­sue of sig­nal­ing which ev­i­dence to im­port is kind of sub­tle, and peo­ple get it wrong a lot when they try to tell a para­dox. Hav­ing messed them up in the past, I think it’s harder than tel­ling a new joke the first time, and uses similar skills :-)

• Ac­tu­ally, a Bayesian and a fre­quen­tist can have differ­ent an­swers to this prob­lem. It re­sides on what dis­tri­bu­tion you are us­ing to de­cide to tell me that a boy is born on Tues­day. The stan­dard an­swer ig­nores this is­sue.

• I don’t know much about the philos­o­phy of statis­ti­cal in­fer­ence. But I am dead sure that if the Bayesian and the fre­quen­tist re­ally do ask the same ques­tion, then they will get the same an­swer. There is a nice spoiler post where the pos­si­ble in­ter­pre­ta­tions of the puz­zle are clearly spel­led out. Do you sug­gest that some of these in­ter­pre­ta­tions are preferred by ei­ther a fre­quen­tist or a Bayesian?

• Well, es­sen­tially, fo­cus­ing on that coin flip is a very Bayesian thing to do. A fre­quen­tist ap­proach to this prob­lem won’t imag­ine the prior coin flip of­ten. See Eliezer’s post about this here. I agree how­ever that a care­ful fre­quen­tist should get the same re­sults as a Bayesian if they are care­ful in this situ­a­tion. What re­sults one gets de­pends in part on what ex­actly one means by a fre­quen­tist here.

• Just so it’s clear, since it didn’t seem su­per clear to me from the other com­ments, the solu­tion to the Tues­day Boy prob­lem given in that ar­ti­cle is a re­ally clever way to get the an­swer wrong.

The prob­lem is the way they use the Tues­day in­for­ma­tion to con­fuse them­selves. For some rea­son not stated in the prob­lem any­where, they as­sume that both boys can­not be born on Tues­day. I see no jus­tifi­ca­tion for this, as there is no nat­u­ral jus­tifi­ca­tion for this, not even if they were born on the ex­act same day and not just the same day of the week! Twins ex­ist! Us­ing their same bizarre rea­son­ing but adding the ex­tra day they took out I get the cor­rect an­swer of 50% (14/​28), in­stead of the close but in­cor­rect an­swer of 48% (13/​27).

Us­ing proper Bayesian up­dat­ing from the prior prob­a­bil­ities of two chil­dren (25% boys, 50% one each, 25% girls) given the in­for­ma­tion that you have one boy, re­gard­less of when he was born, gets you a 50% chance they’re both boys. Since know­ing only one of the sexes doesn’t give any ex­tra in­for­ma­tion re­gard­ing the prob­a­bil­ity of hav­ing one child of each sex, all of the prob­a­bil­ity for both be­ing girls gets shifted to both be­ing boys.

• No, that’s not right. They don’t as­sume that both boys can’t be born on Tues­day. In­stead, what they are do­ing is point­ing out that al­though there is a sce­nario where both boys are born on Tues­day, they can’t count it twice—of the situ­a­tions with a boy born on Tues­day, there are 6 non-Tues­day/​Tues­day, 6 Tues­day/​non-Tues­day, and only 1, not 2, Tues­day/​Tues­day.

Ac­tu­ally, “one of my chil­dren is a boy born on Tues­day” is am­bigu­ous. If it means “I picked the day Tues­day at ran­dom, and it so hap­pens that one of my chil­dren is a boy born on the day I picked”, then the stated solu­tion is cor­rect. If it means “I picked one of my chil­dren at ran­dom, and it so hap­pens that child is a boy, and it also so hap­pens that child was born on Tues­day”, the stated solu­tion is not cor­rect and the day of the week has no effect on the probability

• No, read it again. It’s con­fus­ing as all getout, which is why they make the mis­take, but EACH child can be born on ANY day of the week. The boy on Tues­day is a red her­ring, he doesn’t fac­tor into the prob­a­bil­ity for what day the sec­ond child can be born on at all. The two boys are not the same boys, they are in­di­vi­d­u­als and their prob­a­bil­ities are in­di­vi­d­ual. Re-la­bel them Boy1 and Boy2 to make it clearer:

Here is the break­down for the Boy1Tu/​Boy2Any op­tion:

Boy1Tu/​Boy2Mon­day Boy1Tu/​Boy2Tues­day Boy1Tu/​Boy2Wed­nes­day Boy1Tu/​Boy2Thurs­day Boy1Tu/​Boy2Fri­day Boy1Tu/​Boy2Satur­day Boy1Tu/​Boy2Sunday

Then the BAny/​Boy1Tu op­tion:

Boy2Mon­day/​Boy1Tu Boy2Tues­day/​Boy1Tu Boy2Wed­nes­day/​Boy1Tu Boy2Thurs­day/​Boy1Tu Boy2Fri­day/​Boy1Tu Boy2Satur­day/​Boy1Tu Boy2Sun­day/​Boy1Tu

Seven op­tions for both. For some rea­son they claim ei­ther BTu/​Tues­day isn’t an op­tion, or Tues­day/​BTu isn’t an op­tion, but I see no rea­son for this. Each boy is an in­di­vi­d­ual, and each boy has a 17 prob­a­bil­ity of be­ing born on a given day. In at­tempt­ing to avoid count­ing ev­i­dence twice you’ve skipped count­ing a piece of ev­i­dence at all! In the origi­nal state­ment, they never said one and ONLY one boy was born on Tues­day, just that one was born on Tues­day. That’s where they screwed up—they’ve de­nied the sec­ond boy the op­tion of be­ing born on Tues­day for no good rea­son.

A key in­sight that should have trig­gered their in­tu­ition that their method was wrong was that they state that if you can find a trait rarer than be­ing born on Tues­day, like say be­ing born on the 27th of Oc­to­ber, then you’ll ap­proach 50% prob­a­bil­ity. That is true be­cause the ac­tual prob­a­bil­ity is 50%.

• Here is the break­down for the Boy1Tu/​Boy2Any op­tion:

Boy1Tu/​Boy2Tuesda

Then the BAny/​Boy1Tu op­tion:

Boy2Tues­day/​Boy1Tu

You’re dou­ble-count­ing the case where both boys are born on Tues­day, just like they said.

A key in­sight that should have trig­gered their in­tu­ition that their method was wrong was that they state that if you can find a trait rarer than be­ing born on Tues­day, like say be­ing born on the 27th of Oc­to­ber, then you’ll ap­proach 50% prob­a­bil­ity.

If you find a trait rarer than be­ing born on Tues­day, the dou­ble-count­ing is a smaller per­centage of the sce­nar­ios, so be­ing closer to 50% is ex­pected.

• I see my mis­take, here’s an up­dated break­down:

Boy1Tu/​Boy2Any

Boy1Tu/​Boy2Mon­day Boy1Tu/​Boy2Tues­day Boy1Tu/​Boy2Wed­nes­day Boy1Tu/​Boy2Thurs­day Boy1Tu/​Boy2Fri­day Boy1Tu/​Boy2Satur­day Boy1Tu/​Boy2Sunday

Then the Boy1Any/​Boy2Tu op­tion:

Boy1Mon­day/​Boy2Tu Boy1Tues­day/​Boy2Tu Boy1Wed­nes­day/​Boy2Tu Boy1Thurs­day/​Boy2Tu Boy1Fri­day/​Boy2Tu Boy1Satur­day/​Boy2Tu Boy1Sun­day/​Boy2Tu

See 7 days for each set? They aren’t in­ter­change­able even though the la­bel “boy” makes it seem like they are.

Do the Bayesian prob­a­bil­ities in­stead to ver­ify, it comes out to 50% even.

• What’s the differ­ence be­tween

Boy1Tu/​Boy2Tuesday

and

Boy1Tues­day/​Boy2Tu

?

• In Boy1Tu/​Boy2Tues­day, the boy referred to as BTu in the origi­nal state­ment is boy 1, in Boy2Tu/​Boy1Tues­day the boy referred to in the origi­nal state­ment is boy2.

That’s why the “born on tues­day” is a red her­ring, and doesn’t add any in­for­ma­tion. How could it?

• This sounds like you are try­ing to di­vide “two boys born on Tues­day” into “two boys born on Tues­day and the per­son is talk­ing about the first boy” and “two boys born on Tues­day and the per­son is talk­ing about the sec­ond boy”.

That doesn’t work be­cause you are now no longer deal­ing with cases of equal prob­a­bil­ity. “Boy 1 Mon­day/​Boy 2 Tues­day”, “Boy 1 Tues­day/​Boy 2 Tues­day”, and “Boy 1 Tues­day/​Boy 1 Mon­day” all have equal prob­a­bil­ity. If you’re cre­at­ing sep­a­rate cases de­pend­ing on which of the boys is be­ing referred to, the first and third of those don’t di­vide into sep­a­rate cases but the sec­ond one does di­vide into sep­a­rate cases, each with half the prob­a­bil­ity of the first and third.

doesn’t add any in­for­ma­tion. How could it?

As I pointed out above, whether it adds in­for­ma­tion (and whether the anal­y­sis is cor­rect) de­pends on ex­actly what you mean by “one is a boy born on Tues­day”. If you picked “boy” and “Tues­day” at ran­dom first, and then no­ticed that one child met that de­scrip­tion, that rules out cases where no child hap­pened to meet the de­scrip­tion. If you picked a child first and then no­ticed he was a boy born on a Tues­day, but if it was a girl born on a Mon­day you would have said “one is a girl born on a Mon­day”, you are cor­rect that no in­for­ma­tion is pro­vided.

• The only rele­vant in­for­ma­tion is that one of the chil­dren is a boy. There is still a 50% chance the sec­ond child is a boy and a 50% chance that the sec­ond child is a girl. Since you already know that one of the chil­dren is a boy, the pos­te­rior prob­a­bil­ity that they are both boys is 50%.

Rephrase it this way:

I have flipped two coins. One of the coins came up heads. What is the prob­a­bil­ity that both are heads?

Now, to see why Tues­day is ir­rele­vant, I’ll re-state it thusly:

I have flipped two coins. One I flipped on a Tues­day and it came up heads. What is the prob­a­bil­ity that both are heads?

The sex of one child has no in­fluence on the sex of the other child, nor does the day on which ei­ther child was born in­fluence the day any other child was born. There is a 17 chance that child 1 was born on each day of the week, and there is a 17 chance that child 2 was born on each day of the week. There is a 149 chance that both chil­dren will be born on any given day (1/​7*1/​7), for a 749 or 17 chance that both chil­dren will be born on the same day. That’s your miss­ing 17 chance that gets re­moved in­ap­pro­pri­ately from the Tues­day/​Tues­day sce­nario.

• I have flipped two coins. One of the coins came up heads. What is the prob­a­bil­ity that both are heads?

13 (you ei­ther got hh, heads/​tails,or tails/​heads). You didn’t tell me THE FIRST came up heads. Thats where you are go­ing wrong. At least one is heads is differ­ent in­for­ma­tion then a spe­cific coin is heads.

This is a pretty well known stats prob­lem, a var­i­ant of Gardern’s boy/​girl para­dox. You’ll prob­a­bly find it an in­tro book, and Jiro is cor­rect. You are still over­count­ing. Boy-boy is a differ­ent case then boy-girl (well, de­pend­ing on what the data col­lec­tion pro­cess is).

If you have two boys (prob­a­bil­ity 14), then the prob­a­bil­ity at least one is born on Tues­day (1-(6/​7)^2). ( 6/​7^2 be­ing the prob­a­bil­ity nei­ther is born on Tues­day). The prob­a­bil­ity of a boy-girl fam­ily is (2*1/​4) then (1/​7) (the 17 for the boy hit­ting on Tues­day).

• Lets add a time de­lay to hope­fully fi­nally illus­trate the point that one coin toss does not in­form the other coin toss.

I have two coins. I flip the first one, and it comes up heads. Now I flip the sec­ond coin. What are the odds it will come up heads?

• No one is sug­gest­ing one flip in­forms the other, rather that when you say “one coin came up heads” you are giv­ing some in­for­ma­tion about both coins.

I have two coins. I flip the first one, and it comes up heads. Now I flip the sec­ond coin. What are the odds it will come up heads?

This is 12, be­cause there are two sce­nar­ios, hh, ht. But its differ­ent in­for­ma­tion then the other ques­tion.

If you say “one coin is heads,” you have hh,ht,th, be­cause it could be that the first flip was tails/​the sec­ond heads (a pos­si­bil­ity you have ex­cluded in the above).

• No, it’s the ex­act same ques­tion, only the la­bels are differ­ent.

The prob­a­bil­ity that any one child is boy is 50%. We have been told that one child is a boy, which only leaves two op­tions—HH and HT. If TH were still available, then so would TT be available be­cause the next flip could be re­vealed to be tails.

Here’s the prob­a­bil­ity in bayesian:

P(BoyBoy) = 0.25 P(Boy) = 0.5 P(Boy|BoyBoy) = 1

P(BoyBoy|Boy) = P(Boy|BoyBoy)*P(BoyBoy)/​P(Boy)

P(BoyBoy|Boy)= (1*0.25) /​ 0.5 = 0.25 /​ 0.5 = 0.5

P(BoyBoy|Boy) = 0.5

It’s ex­actly the same as the coin flip, be­cause the prob­a­bil­ity is 50% - the same as a coin flip. This isn’t the monty hall prob­lem. Know­ing half the prob­lem (that there’s at least one boy) doesn’t change the prob­a­bil­ity of the other boy, it just changes what our pos­si­bil­ities are.

• No, it’s the ex­act same ques­tion, only the la­bels are differ­ent.

No, it isn’t. You should con­sider that you are dis­agree­ing with a pretty stan­dard stats ques­tion, so odds are high you are wrong. With that in mind, you should reread what peo­ple are tel­ling you here.

Now, con­sider “I flip two coins” the pos­si­ble out­comes are hh,ht,th,tt

I hope we can agree on that much.

Now, I give you more in­for­ma­tion and I say “one of the coins is heads,” so we Bayesian up­date by cross­ing out any sce­nario where one coin isn’t heads. There is only 1 (tt)

hh,ht,th

So it should be pretty clear the prob­a­bil­ity I flipped two heads is 13.

Now, your sce­nario, flipped two coins (hh,ht,th,tt), and I give you the in­for­ma­tion “the first coin is heads,” so we cross out ev­ery­thing where the first coin is tails, leav­ing (hh,ht). Now the prob­a­bil­ity you flipped two heads is 12.

I don’t know how to make this any more sim­ple.

I know it’s not the be all end all, but it’s gen­er­ally re­li­able on these types of ques­tions, and it gives P = 12, so I’m not the one dis­agree­ing with the stan­dard re­sult here.

Do the math your­self, it’s pretty clear.

Edit: Read­ing closer, I should say that both an­swers are right, and the prob­a­bil­ity can be ei­ther 12 or 13 de­pend­ing on your as­sump­tions. How­ever, the prob­lem as stated falls best to me in the 12 set of as­sump­tions. You are told one child is a boy and given no other in­for­ma­tion, so the only prob­a­bil­ity left for the sec­ond child is a 50% chance for boy.

Did you ac­tu­ally read it? It does not agree with you. Look un­der the head­ing “sec­ond ques­tion.”

Do the math your­self, it’s pretty clear.

I did the math in the post above, enu­mer­at­ing the pos­si­bil­ities for you to try to help you find your mis­take.

Edit, in re­sponse to the edit:

I should say that both an­swers are right, and the prob­a­bil­ity can be ei­ther 12 or 13 de­pend­ing on your as­sump­tions.

Which is ex­actly analo­gous to what Jiro was say­ing about the Tues­day ques­tion. So we all agree now? Tues­day can raise your prob­a­bil­ity slightly above 50%, as was said all along.

How­ever, the prob­lem as stated falls best to me in the 12 set of as­sump­tions. You are told one child is a boy and given no other in­for­ma­tion, so the only prob­a­bil­ity left for the sec­ond child is a 50% chance for boy.

And you are im­me­di­ately mak­ing the ex­act same mis­take again. You are told ONE child is a boy, you are NOT told the FIRST child is a boy. You do un­der­stand that these are differ­ent?

• The rele­vant quote from the Wiki:

The para­dox arises be­cause the sec­ond as­sump­tion is some­what ar­tifi­cial, and when de­scribing the prob­lem in an ac­tual set­ting things get a bit sticky. Just how do we know that “at least” one is a boy? One de­scrip­tion of the prob­lem states that we look into a win­dow, see only one child and it is a boy. This sounds like the same as­sump­tion. How­ever, this one is equiv­a­lent to “sam­pling” the dis­tri­bu­tion (i.e. re­mov­ing one child from the urn, as­cer­tain­ing that it is a boy, then re­plac­ing). Let’s call the state­ment “the sam­ple is a boy” propo­si­tion “b”. Now we have: P(BB|b) = P(b|BB) P(BB) /​ P(b) = 1 14 /​ 12 = 12. The differ­ence here is the P(b), which is just the prob­a­bil­ity of draw­ing a boy from all pos­si­ble cases (i.e. with­out the “at least”), which is clearly 0.5. The Bayesian anal­y­sis gen­er­al­izes eas­ily to the case in which we re­lax the 5050 pop­u­la­tion as­sump­tion. If we have no in­for­ma­tion about the pop­u­la­tions then we as­sume a “flat prior”, i.e. P(GG) = P(BB) = P(G.B) = 13. In this case the “at least” as­sump­tion pro­duces the re­sult P(BB|B) = 12, and the sam­pling as­sump­tion pro­duces P(BB|b) = 23, a re­sult also deriv­able from the Rule of Suc­ces­sion.

We have no gen­eral pop­u­la­tion in­for­ma­tion here. We have one man with at least one boy.

• I’m not at all sure you un­der­stand that quote. Lets stick with the coin flips:

Do you un­der­stand why these two ques­tions are differ­ent: I tell you- “I flipped two coins, at least one of them came out heads, what is the prob­a­bil­ity that I flipped two heads?” A:1/​3 AND “I flipped two coins, you choose one at ran­dom and look at it, its heads.What is the prob­a­bil­ity I flipped two heads” A: 12

• For the record, I’m sure this is frus­trat­ing as all getout for you, but this whole ar­gu­ment has re­ally clar­ified things for me, even though I still think I’m right about which ques­tion we are an­swer­ing.

Many of my ar­gu­ments in pre­vi­ous posts are wrong (or at least in­com­plete and a bit naive), and it didn’t click un­til the last post or two.

Like I said, I still think I’m right, but not be­cause my prior anal­y­sis was any good. The 13 case was a ma­jor hole in my rea­son­ing. I’m hap­pily wait­ing to see if you’re go­ing to de­stroy my lat­est anal­y­sis, but I think it is pretty solid.

• Yes, and we are deal­ing with the sec­ond ques­tion here.

Is that not what I said be­fore?

We don’t have 1000 fam­i­lies with two chil­dren, from which we’ve se­lected all fam­i­lies that have at least one boy (which gives 13 prob­a­bil­ity). We have one fam­ily with two chil­dren. Then we are told one of the chil­dren is a boy, and given zero other in­for­ma­tion. The prob­a­bil­ity that the sec­ond is a boy is 12, so the prob­a­bil­ity that both are boys is 12.

The pos­si­ble op­tions for the “Boy born on Tues­day” are not Boy/​Girl, Girl/​Boy, Boy/​Boy. That would be the case in the se­lec­tion of 1000 fam­i­lies above.

The pos­si­ble op­tions are Boy (Tu) /​ Girl, Girl /​ Boy (Tu), Boy (Tu) /​ Boy, Boy /​ Boy (Tu).

There are two Boy/​Boy com­bi­na­tions, not one. You don’t have enough in­for­ma­tion to throw one of them out.

This is NOT a case of sam­pling.

• As long as you re­al­ize there is a differ­ence be­tween those two ques­tions, fine. We can dis­agree about what as­sump­tions the word­ing should lead us to, thats ir­rele­vant to the ac­tual statis­tics and can be an agree-to-dis­agree situ­a­tion. Its just im­por­tant to re­al­ize that what the ques­tion means/​how you get the in­for­ma­tion is im­por­tant.

We don’t have 1000 fam­i­lies with two chil­dren, from which we’ve se­lected all fam­i­lies that have at least one boy (which gives 13 prob­a­bil­ity). We have one fam­ily with two chil­dren. Then we are told one of the chil­dren is a boy, and given zero other in­for­ma­tion.

If we have one fam­ily with two chil­dren, of which one is a boy, they are (by defi­ni­tion) a mem­ber of the set “all fam­i­lies that have at least one boy.” So it mat­ters how we got the in­for­ma­tion.

If we got that in­for­ma­tion by grab­bing a kid at ran­dom and look­ing at it (so we have in­for­ma­tion about one spe­cific child), that is sam­pling, and it leads to the 12 prob­a­bil­ity.

If we got that in­for­ma­tion by hav­ing some­one check both kids, and tell us “at least one is a boy” we have differ­ent in­for­ma­tion (its in­for­ma­tion about the set of kids the par­ents have, not in­for­ma­tion about one spe­cific kid).

This is NOT a case of sam­pling.

If it IS sam­pling (if I grab a kid at ran­dom and say “whats your Birth­day?” and it hap­pens to be Tues­day), then the prob­a­bil­ity is 12. (we have in­for­ma­tion about the spe­cific kid’s birth­day).

If in­stead, I ask the par­ents to tell me the birth­day of one of their chil­dren, and the par­ent says ‘I have at least one boy born on Tues­day’, then we get, in­stead, in­for­ma­tion about their set of kids, and the prob­a­bil­ity is the larger num­ber.

Sam­pling is what leads to the an­swer you are sup­port­ing.

• The an­swer I’m sup­port­ing is based on flat pri­ors, not sam­pling. I’m say­ing there are two pos­si­ble Boy/​Boy com­bi­na­tions, not one, and there­fore it takes up half the prob­a­bil­ity space, not 13.

Sam­pling to the “Boy on Tues­day” prob­lem gives roughly 48% (as per the origi­nal ar­ti­cle), not 50%.

We are sim­ply told that the man has a boy who was born on tues­day. We aren’t told how he chose that boy, whether he’s older or younger, etc. There­fore we have four pos­si­bil­ites, like I out­lined above.

Is my anal­y­sis that the pos­si­bil­ities are Boy (Tu) /​Girl, Girl /​ Boy (Tu), Boy (Tu)/​Boy, Boy/​Boy (Tu) cor­rect?

If so, is not the prob­a­bil­ity for some com­bi­na­tion of Boy/​Boy 1/​2? If not, why not? I don’t see it.

BTW, con­trary to my pre­vi­ous posts, hav­ing the in­for­ma­tion about the boy born on Tues­day is crit­i­cal be­cause it al­lows us (and in fact re­quires us) to dis­t­in­guish be­tween the two boys.

That was in fact the point of the origi­nal ar­ti­cle, which I now dis­agree with sig­nifi­cantly less. In fact, I agree with the ma­jor premise that the tues­day in­for­ma­tion pushes the odds of Boy/​Boy closer 50%, I just dis­agree that you can’t rea­son that it pushes it to ex­actly 50%.

• Is my anal­y­sis that the pos­si­bil­ities are Boy (Tu) /​Girl, Girl /​ Boy (Tu), Boy (Tu)/​Boy, Boy/​Boy (Tu) cor­rect?

No. For any day of the week EXCEPT Tues­day, boy and girl are equiv­a­lent. For the case of both chil­dren born on Tues­day you have for girls: Boy(tu)/​Girl(tu),Girl(tu)/​Boy(tu), and for boys: boy(tu)/​boy(tu).

That was in fact the point of the origi­nal ar­ti­cle, which I now dis­agree with sig­nifi­cantly less. In fact, I agree with the ma­jor premise that the tues­day in­for­ma­tion pushes the odds of Boy/​Boy closer 50%, I just dis­agree that you can’t rea­son that it pushes it to ex­actly 50%.

This state­ment leads me to be­lieve you are still con­fused. Do you agree that if I know a fam­ily has two kids, I knock on the door and a boy an­swers and says “I was born on a Tues­day,” that the prob­a­bil­ity of the sec­ond kid be­ing a girl is 1/​2? And in this case, Tues­day is ir­rele­vant? (This the wikipe­dia called “sam­pling”)

Do you agree that if, in­stead, the par­ents give you the in­for­ma­tion “one of my two kids is a boy born on a Tues­day”, that this is a differ­ent sort of in­for­ma­tion, in­for­ma­tion about the set of their chil­dren, and not about a spe­cific child?

• This state­ment leads me to be­lieve you are still con­fused. Do you agree that if I know a fam­ily has two kids, I knock on the door and a boy an­swers and says “I was born on a Tues­day,” that the prob­a­bil­ity of the sec­ond kid be­ing a girl is 1/​2? And in this case, Tues­day is ir­rele­vant? (This the wikipe­dia called “sam­pling”)

I agree with this.

Do you agree that if, in­stead, the par­ents give you the in­for­ma­tion “one of my two kids is a boy born on a Tues­day”, that this is a differ­ent sort of in­for­ma­tion, in­for­ma­tion about the set of their chil­dren, and not about a spe­cific child?

I agree with this if they said some­thing along the lines of “One and only one of them was born on Tues­day”. If not, I don’t see how the Boy(tu)/​Boy(tu) con­figu­ra­tion has the same prob­a­bil­ity as the oth­ers, be­cause it’s twice as likely as the other two con­figu­ra­tions that that is the con­figu­ra­tion they are talk­ing about when they say “One was born on Tues­day”.

Here’s my break­down with 1000 fam­i­lies, to try to make it clear what I mean:

1000 Fam­i­lies with two chil­dren, 750 have boys.

Of the 750, 500 have one boy and one girl. Of these 500, 17, or roughly 71 have a boy born on Tues­day.

Of the 750, 250 have two boys. Of these 250, 27, or roughly 71 have a boy born on Tues­day.

71 = 71, so it’s equally likely that there are two boys as there are a boy and a girl.

Hav­ing two boys dou­bles the prob­a­bil­ity that one boy was born on Tues­day com­pared to hav­ing just one boy.

And I don’t think I’m con­fused about the sam­pling, be­cause I didn’t use the sam­pling rea­son­ing to get my re­sult*, but I’m not su­per con­fi­dent about that so if I am just keep giv­ing me num­bers and hope­fully it will click.

*I mean in the pre­vi­ous post, not speci­fi­cally this post.

• Of these 250, 27, or roughly 71 have a boy born on Tues­day.

This is wrong. With two boys each with a prob­a­bil­ity of 17 to be born on Tues­day, the prob­a­bil­ity of at least one on a Tues­day isn’t 27, its 1-(6/​7)^2

• How can that be? There is a 17 chance that one of the two is born on Tues­day, and there is a 17 chance that the other is born on Tues­day. 17 + 17 is 27.

There is also a 149 chance that both are born on tues­day, but how does that sub­tract from the other two num­bers? It doesn’t change the prob­a­bil­ity that ei­ther of them are born on Tues­day, and both of those prob­a­bil­ities add.

• The prob­lem is that you’re count­ing that 1/​49th chance twice. Once for the first brother and once for the sec­ond.

• I see that now, it took a LOT for me to get it for some rea­son.

• You over­count, the both on Tues­day is over­counted there. Think of it this way- if I have 8 kids do I have a bet­ter than 100% prob­a­bil­ity of hav­ing a kid born on Tues­day?

There is a 1/​7x6/​7 chance the first is born on Tues­day and the sec­ond is born on an­other day. There is a 1/​7x6/​7 chance the sec­ond is born on Tues­day and the first is born on an­other day. And there is a 149 chance that both are born on Tues­day.

All to­gether thats 1349. Alter­na­tively, there is a (6/​7)^2 chance that both are born not-on-Tues­day, so 1-(6/​7)^2 tells you the com­ple­men­tary prob­a­bil­ity.

• Wow.

I’ve seen that same ex­pla­na­tion at least five times and it didn’t click un­til just now. You can’t dis­t­in­guish be­tween the two on tues­day, so you can only count it once for the pair.

Which means the ar­ti­cle I said was wrong was ab­solutely right, and if you were told that, say one boy was born on Jan­uary 17th, the chances of both be­ing born on the same day are 1-(364/​365)^2 (ig­nor­ing leap years), which gives a fi­nal prob­a­bil­ity of roughly 49.46% that both are boys.

ETA: I also think I see where I’m go­ing wrong with the ter­minol­ogy—sam­pling vs not sam­pling, but I’m not 100% there yet.

• “The first coin comes up heads” (in this ver­sion) is not the same thing as “one of the coins comes up heads” (as in the origi­nal ver­sion). This ver­sion is 50%, the other is not.

• How is it differ­ent? In both cases I have two in­de­pen­dent coin flips that have ab­solutely no re­la­tion to each other. How does know­ing which of the two came up heads make any differ­ence at all for the prob­a­bil­ity of the other coin?

If it was the first coin that came up heads, TT and TH are off the table and only HH and HT are pos­si­ble. If the sec­ond coin came up heads then HT and TT would be off the table and only TH and HH are pos­si­ble.

The to­tal prob­a­bil­ity mass of some com­bi­na­tion of T and H (ei­ther HT or TH) starts at 50% for both flips com­bined. Once you know one of them is heads, that prob­a­bil­ity mass for the whole prob­lem is cut in half, be­cause one of your flips is now 100% heads and 0% tails. It doesn’t mat­ter that you don’t know which is which, one flip doesn’t have any in­fluence on the prob­a­bil­ity of the other. Since you already have one heads at 100%, the en­tire prob­a­bil­ity of the re­main­der of the prob­lem rests on the sec­ond coin, which is a 5050 split be­tween heads and tails. If heads, HH is true. If tails, HT is true (or TH, but you don’t get both of them!).

Tell me how know­ing one of the coins is heads changes the prob­a­bil­ity of the sec­ond flip from 50% to 33%. It’s a fair coin, it stays 50%.

• Flip two coins 1000 times, then count how many of those tri­als have at least one head (~750). Count how many of those tri­als have two heads (~250).

Flip two coins 1000 times, then count how many of those tri­als have the first flip be a head (~500). Count how many of those tri­als have two heads (~250).

By the way, these sorts of puz­zles should re­ally be ex­pressed as a ques­tion-and-an­swer di­alogue. Sim­ply vol­un­teer­ing in­for­ma­tion leaves it am­bigu­ous as to what you’ve ac­tu­ally learned (“would this per­son have equally likely said ‘one of my chil­dren is a girl’ if they had both a boy and girl?”).

• Yeah, prob­a­bly the biggest thing I don’t like about this par­tic­u­lar ques­tion is that the an­swer de­pends en­tirely upon un­stated as­sump­tions, but at the same time it clearly illus­trates how im­por­tant it is to be spe­cific.

• . There is still a 50% chance the sec­ond child is a boy and a 50% chance that the sec­ond child is a girl.

No there’s not. The cases where the sec­ond child is a boy and the sec­ond child is a girl are not equal prob­a­bil­ity.

I have flipped two coins. One of the coins came up heads. What is the prob­a­bil­ity that both are heads?

If you picked “heads” be­fore flip­ping the coins, then the prob­a­bil­ity is 13. There are three pos­si­bil­ities: HT, TH, and HH, and all of these pos­si­bil­ities are equally likely.

I have flipped two coins. One I flipped on a Tues­day and it came up heads. What is the prob­a­bil­ity that both are heads?

If you picked “heads” and “Tues­day” be­fore know­ing when you would be flip­ping the coins, and then flipped each coin on a ran­domly-se­lected day, and you just stopped if there weren’t any heads on Tues­day, then the an­swer is the same as the an­swer for boys on Tues­day. If you flipped the coin and then re­al­ized it was Tues­day, the Tues­day doesn’t af­fect the re­sult.

The sex of one child has no in­fluence on the sex of the other child, nor does the day on which ei­ther child was born in­fluence the day any other child was born.

If you picked the sex first be­fore look­ing at the chil­dren, the sex of one child does in­fluence the sex of the other child be­cause it af­fects whether you would con­tinue or say “there aren’t any of the sex I picked” and the sexes in the cases where you would con­tinue are not equally dis­tributed.

• Which boy did I count twice?

Edit:

BAny/​Boy1Tu in the above quote should be Boy2Any/​Boy1Tu.

You could re-la­bel boy1 and boy2 to be cat and dog and it won’t change the prob­a­bil­ities—that would be CatTu/​DogAny.

• Note be­fore you start calcu­lat­ing this: There’s a dis­tinc­tion be­tween the “first” and the “sec­ond” child made in the ar­ti­cle. To avoid the risk of hav­ing to calcu­late all over again, take this into ac­count if you want to com­pare your re­sults to theirs.

I calcu­lated the prob­a­bil­ity with­out know­ing this, so that I just counted BG and GB as one sce­nario, where there’s one girl and one boy. That means that with­out the Tues­day fact the prob­a­bil­ity of an­other boy is 12, not 13.

(I ended up at a pos­te­rior prob­a­bil­ity of 23, by the way.)

• I’m so happy: I’ve just got this one right, be­fore look­ing at the an­swer. It’s damn beau­tiful.

Thanks for shar­ing.

• I’m so happy: I’ve just got this one right, be­fore look­ing at the an­swer. It’s damn beau­tiful.

Same here. It was a perfect test, as I’ve never seen the Tues­day Boy prob­lem be­fore. Took a lit­tle wran­gling to get it all to come out in sane frac­tions, and I was star­ing at the fi­nal re­sult go­ing, “that can’t be right”, but sure enough, it was ex­actly right.

(Funny thing: my origi­nal in­tu­ition about the prob­lem wasn’t that far off. I was sim­ply ig­nor­ing the part about Tues­day, and fo­cus­ing on the prior prob­a­bil­ity that the other child was a boy. It gives a close, but not-quite-right an­swer.)

• It would be nice to have the ar­eas of the blobs rep­re­sent­ing the per­centages.

• When I tried to ex­plain Bayes’ to some fel­low soft­ware en­g­ineers at work, I came up with http://​​moshez.word­press.com/​​2011/​​02/​​06/​​bayes-the­o­rem-for-pro­gram­mers/​​

• Thank you Kom­pon­isto,

I have read many ex­pla­na­tions of Bayesian the­ory, and like you, if I con­cen­trated hard enough I could fol­low the rea­son­ing , but I could never rea­son it out for my­self. Now I can. Your ex­pla­na­tion was perfect for me. It not only en­abled me to “grok” the Monty Hall prob­lem, but Bayesian calcu­la­tions in gen­eral, while be­ing able to re­tain the the­ory.

Thank you again, Ben

• Thank you very much for this. Un­til you put it this way, I could not grasp the Monty Hall prob­lem; I per­sisted in be­liev­ing that there would be a 5050 chance once the door was opened. Thank you for chang­ing my mind.

• I tend to think out Monty Hall like this: The prob­a­bil­ity you have cho­sen the door hid­ing the car is 13. Once one of the other two doors is shown to hide a goat, the prob­a­blity of the third door hid­ing the car must be 23. There­fore you dou­ble your chances to win the car by switch­ing.

• The Monty Hall prob­lem seems like it can be sim­plified: Once you’ve picked the door, you can switch to in­stead se­lect­ing the two al­ter­nate doors. You know that one of the al­ter­nate doors con­tains a goat (since there’s only one car), which is equiv­a­lent to hav­ing Monty open one of those two doors.

The trick is sim­ply in as­sum­ing that Monty is ac­tu­ally in­tro­duc­ing any new in­for­ma­tion.

Not sure if it’s helpful to any­one else, but it just sort of clicked read­ing it this time :)

• Wow, that was great! I already had a fairly good un­der­stand­ing of the The­o­rem, but this helped ce­ment it fur­ther and helped me com­pute a bit faster.

It also gave me a good dose of learn­ing-tin­gles, for which I thank you.

• In figure 10 above, should the sec­ond blob have a value of 72.9% ? I no­ticed that the to­tal of all the per­cents are only adding up to 97% with the cur­rent val­ues. I calcu­lated as fol­lows: New 100% value: 10% + 35% + 3% = 48% H1 : 10% /​ 48% = 20.833% H2: 35% /​ 48% = 72.9166% H3: 3% /​ 48% = 6.25% To­tal: 99.99%

Also, I found this easy to un­der­stand vi­su­ally. Thanks.

• My math­e­mat­ics agrees − 72.9%. Good catch!

• Per­haps a bet­ter ti­tle would be “Bayes’ The­o­rem Illus­trated (My Ways)”

In the first ex­am­ple you use shapes with col­ors of var­i­ous sizes to illus­trate the ideas vi­su­ally. In the sec­ond ex­am­ple, you us­ing plain rec­t­an­gles of ap­prox­i­mately the same size. If I was a vi­sual learner, I don’t know if your post would help me much.

I think you’re on the right track in ex­am­ple one. You might want to use shapes that are eas­ier to es­ti­mate the rel­a­tive ar­eas. It’s hard to tell if one tri­an­gle is twice as big as an­other (as mea­sured by area), but it’s eas­ier to do with rec­t­an­gles of the same height (where you just vary the width). More im­por­tantly, I think it would help to show math with shapes. For ex­am­ple, I would sug­gest that figure 18 has P(door 2)= the or­ange tri­an­gle in figure 17 di­vided by the or­ange tri­an­gle plus the blue tri­an­gle from figure 17 (but where you show the di­vi­sion by shapes). When I teach, I some­times do this with Venn di­a­grams (show di­vi­sion of chunks of cir­cles and rec­t­an­gles to illus­trate con­di­tional prob­a­bil­ity).

• A pre­sen­ta­tion cri­tique: psy­cholog­i­cally, we tend to com­pare the rel­a­tive ar­eas of shapes. Your ovals in Figure 1 are scaled so that their lin­ear di­men­sions (width, for ex­am­ple) are in the ra­tio 2:5:3; how­ever, what we see are ovals whose ar­eas are in ra­tio 4:25:9, which isn’t what you’re try­ing to con­vey. I think this hap­pens for later shapes as well, al­though I didn’t check them all.

• Really? I’d have said the ex­act op­po­site. For ex­am­ple in this post, the phrase “half the origi­nal’s size” means that the lin­ear di­men­sions are halved. This is­sue also come up in the pro­duc­tion of bub­ble charts, where the size of a cir­cle rep­re­sents some value. When I look at a bub­ble chart it is of­ten un­clear whether the data is in­tended to be rep­re­sented by the area or the ra­dius.

It is cer­tainly eas­ier for me to com­pare lin­ear di­men­sions than ar­eas.

• Hence the pop­u­lar­ity of bar charts, where the area and the lin­ear di­men­sion are cou­pled. But the vi­sual im­pact is a func­tion of area, more than length, even if it is hard to quan­tify in the eye—but quan­tifi­ca­tion should be done by the quan­ti­ta­tive num­bers, not by graph­i­cal es­ti­ma­tion.

(How many sen­tences can I start with con­junc­tions? Let me count the ways...)

• Great vi­su­al­iza­tions.

In fact, this (only with­out tri­an­gles, squares,...) is how I’ve been in­tu­itively calcu­lat­ing Bayesian prob­a­bil­ities in “ev­ery­day” life prob­lems since I was young. But you man­aged to make it even clearer for me. Good to see it ap­plied to Monty Hall.

• This is re­ally brilli­ant. Thanks for mak­ing it all seem so easy: for some rea­son I never saw the con­nec­tion be­tween an up­date and rescal­ing like that, but now it seems ob­vi­ous.

I’d like to see this kind of di­a­gram for the Sleep­ing Beauty prob­lem.

• 15 Feb 2013 22:23 UTC
1 point

I find that Monty Hall is eas­ier to un­der­stand with N doors, N > 2.

N doors, one hides a car. You pick a door at ran­dom yield­ing 1/​N prob­a­bil­ity of get­ting car. Host now opens N-2 doors which are not your door, all con­tain­ing goats. The prob­a­bil­ity the other door left has a car is not (N-1)/​N.

Set N to 1000 and peo­ple gen­er­ally agree that switch­ing is good. Set N to 3 and they dis­agree.

• The challenge with Bayes’ illus­tra­tions is to si­mul­ta­neously.show 1) re­la­tions 2) ra­tios. The sug­gested ap­proach works well. I am sug­gest­ing to com­bine Venn di­a­grams and Pie charts:

http://​​or­a­cleaide.word­press.com/​​2012/​​12/​​26/​​a-venn-pie/​​

Happy New Year!

• Thank you Kom­pon­isto! Ap­par­ently, my brain works similar to yours on this mat­ter. Here is a video by Richard Car­rier ex­plain­ing Bayes’ the­o­rem that I also found helpful.

• Why would you need more than plain English to in­tu­itively grasp Monty-Hall-type prob­lems?

Take the origi­nal Monty Hall ‘Dilemma’. Just imag­ine there are two can­di­dates, A and B. A and B both choose the same door. After the mod­er­a­tor picked one door A always stays with his first choice, B always changes his choice to the re­main­ing third door. Now imag­ine you run this ex­per­i­ment 999 times. What will hap­pen? Be­cause A always stays with his ini­tial choice, he will win 333 cars. But where are the re­main­ing 666 cars? Of course B won them!

Or con­duct the ex­per­i­ment with 100 doors. Now let’s say the can­di­date picks door 8. By rule of the game the mod­er­a­tor now has to open 98 of the re­main­ing 99 doors be­hind which there is no car. After­wards there is only one door left be­sides door 8 that the can­di­date has cho­sen. Ob­vi­ously you would change your de­ci­sion now! The same should be the case with only 3 doors!

There re­ally is no prob­lem here. You don’t need to simu­late this. Your chance of pick­ing the car first time is 13 but your chance of choos­ing a door with a goat be­hind it, at the be­gin­ning, is 23. Thus on av­er­age, 23 of times that you are play­ing this game you’ll pick a goat at first go. That also means that 23 of times that you are play­ing this game, and by defi­ni­tion pick a goat, the mod­er­a­tor will have to pick the only re­main­ing goat. Be­cause given the laws of the game the mod­er­a­tor knows where the car is and is only al­lowed to open a door with a goat in it. What does that mean? That on av­er­age, at first go, you pick a goat 23 of the time and hence the mod­er­a­tor is forced to pick the re­main­ing goat 23 of the time. That means 23 of the time there is no goat left, only the car is left be­hind the re­main­ing door. There­fore 23 of the time the re­main­ing door has the car.

I don’t need fancy vi­su­als or even for­mu­las for this. Do you re­ally?

• I can tes­tify that this isn’t any­where near as ob­vi­ous to most peo­ple than it is to you. I, for one, had to have other peo­ple ex­plain it to me the first time I ran into the prob­lem, and even then it took a small while.

• I think the very prob­lem in un­der­stand­ing such is­sues is shown in your re­ply. Peo­ple as­sume too much, they read too much into things. I never said it has been ob­vi­ous to me. I asked why you would need more than plain English to un­der­stand it and gave some ex­am­ples on how to de­scribe the prob­lem in an ab­stract way that might be am­ple to grasp the prob­lem suffi­ciently. If you take things more liter­ally and don’t come up with op­tions that were never men­tioned it would be much eas­ier to un­der­stand. Like call­ing the po­lice in the case of the trol­ley prob­lem or what­ever was never in­tended to be a rule of a par­tic­u­lar game.

• Well, yeah. But if I re­call, I did have a plain English ex­pla­na­tion of it. There was an ar­ti­cle on Wikipe­dia about it, though since this was at least five years ago, the ex­pla­na­tion wasn’t as good as it is in to­day’s ar­ti­cle. It still did a pass­ing job, though, which wasn’t enough for me to get it very quickly.

• Yes­ter­day, when fal­ling asleep, I re­mem­bered that I in­deed used the word ‘ob­vi­ous’ in what I wrote. For­got about it, I wrote the plain-English ex­pla­na­tion from above ear­lier in a com­ment to the ar­ti­cle ‘Pi­geons out­perform hu­mans at the Monty Hall Dilemma’ and just copied it from there.

Any­way, I doubt it is ob­vi­ous to any­one the first time. At least any­one who isn’t a trained Bayesian. But for me it was enough to read some plain-English (Ger­man ac­tu­ally) ex­pla­na­tions about it to come to the con­clu­sion that the right solu­tion is ob­vi­ously right and now also in­tu­itively so.

Maybe the prob­lem is also that most peo­ple are sim­ply skep­ti­cal to ac­cept a given re­sult. That is, is it re­ally ob­vi­ous to me now or have I just ac­cepted that it is the right solu­tion, re­peated many times to be­come in­tu­itively fixed? Is 1 + 1 = 2 re­ally ob­vi­ous? The last page of Rus­sel and White­head’s proof that 1+1=2 could be found on page 378 of the Prin­cipia Math­e­mat­ica. So is it re­ally ob­vi­ous or have we sim­ply all, col­lec­tively, come to ac­cept this ‘ax­iom’ to be right and true?

I haven’t had much time lately to get much fur­ther with my stud­ies, I’m still strug­gling with ba­sic Alge­bra. I have al­most no for­mal ed­u­ca­tion and try to ed­u­cate my­self now. That said, I started to watch a video se­ries lately (The Most IMPORTANT Video You’ll Ever See) and was struck when he said that to roughly figure out the dou­bling time you sim­ply di­vide 70 by the per­centage growth rate. I went to check it my­self if it works and later looked it up. Well, it’s NOT ob­vi­ous why this is the case, at least not for me. Not even now that I have read up on the math­e­mat­i­cal strict for­mula. But I’m sure, as I will think about it more, read more proofs and work with it, I’ll come to re­gard it as ob­vi­ously right. But will it be any more ob­vi­ous than be­fore? I will sim­ply have col­lected some ev­i­dence for its truth value and its con­sis­tency. Things just start to make sense, or we think so be­cause they work and/​or are con­sis­tent.

• But I’m sure, as I will think about it more, read more proofs and work with it, I’ll come to re­gard it as ob­vi­ously right. But will it be any more ob­vi­ous than be­fore?

If you’re in­ter­ested, here is a good ex­pla­na­tion of the deriva­tion of the for­mula. I don’t think it’s ob­vi­ous, any more than the quadratic for­mula is ob­vi­ous: it’s just one of those math­e­mat­i­cal tricks that you learn and be­comes sec­ond na­ture.

• I’m not sure I’m com­pletely happy with that ex­pla­na­tion. They use the re­sult that ln(1+x) is very close to x when x is small. This is due to the Tay­lor se­ries ex­pan­sion of ln(1+x) (edit:or sim­ply on look­ing at the ra­tio of the two and us­ing L’Hospi­tal’s rule), but if one hasn’t had calcu­lus, that claim is go­ing to look like magic.

• Here are more ex­am­ples:

Those ex­pla­na­tions are re­ally great. I’ve missed such in school when won­der­ing WHY things be­have like they do, when I was only shown HOW to use things to get what I want to do. But what do these ex­pla­na­tions re­ally ex­plain. I think they are merely satis­fy­ing our idea that there is more to it than meets the eye. We think some­thing is miss­ing. What such ex­pla­na­tions re­ally do is to show us that the heuris­tics re­ally work and that they are con­sis­tent on more than one level, they are rea­son­able.

• That said, I started to watch a video se­ries lately [...] and was struck when he said that to roughly figure out the dou­bling time you sim­ply di­vide 70 by the per­centage growth rate. I went to check it my­self if it works and later looked it up. Well, it’s NOT ob­vi­ous why this is the case, at least not for me. Not even now that I have read up on the math­e­mat­i­cal strict for­mula.

Well, it’s an ap­prox­i­ma­tion, that’s all. Pi is ap­prox­i­mately equal to 355113 - yeah, there’s good math­e­mat­i­cal rea­sons for choos­ing that par­tic­u­lar frac­tion as an ap­prox­i­ma­tion, but the ac­cu­racy jus­tifies it­self. [ed­ited sen­tence:] You only need one real rev­e­la­tion to not worry about how true Td = 70/​r is: that the dou­bling time is a smooth line—there’s no jaggedy peaks ran­domly in the mid­dle. After that, you can just look how good the fit is and say, “yeah, that works for 0.1 < r < 20 for the ac­cu­racy I need”.

• Although it’s late, I’d like to say that XiXiDu’s ap­proach de­serves more credit and I think it would have helped me back when I didn’t un­der­stand this prob­lem. Eliezer’s Bayes’ The­o­rem post cites the per­centage of doc­tors who get the breast can­cer prob­lem right when it’s pre­sented in differ­ent but math­e­mat­i­cally equiv­a­lent forms. The doc­tors (and I) had an eas­ier time when the prob­lem was pre­sented with quan­tities (100 out of 10,000 women) than with ex­plicit prob­a­bil­ities (1% of women).

Like­wise, think­ing about a large num­ber of tri­als can make the no­tion of prob­a­bil­ity eas­ier to vi­su­al­ize in the Monty Hall prob­lem. That’s be­cause run­ning those tri­als and count­ing your win­nings looks like some­thing. The per­cent chance of win­ning once does not look like any­thing. In­tro­duc­ing the com­peti­tor was also a great touch since now the cars I don’t win are easy to vi­su­al­ize too; that smug bas­tard has them!

Or you know what? Maybe none of that vi­su­al­iza­tion stuff mat­tered. Maybe the key sen­tence is “[Can­di­date] A always stays with his first choice”. If you com­mit to a cer­tain door then you might as well wear a blind­fold from that point for­ward. Then Monty can open all 3 doors if he likes and it won’t bring your chances any closer to 12.

• Are you se­ri­ous? Are you buy­ing this? Ok—let me make this easy: There NEVER WAS a 33% chance. Ever. The 1-in-3 choice is a ruse. No mat­ter what door you choose, Monty has at least one door with a goat be­hind it, and he opens it. At that point, you are pre­sented with a 1-in-2 choice. The prior choice is com­pletely ir­rele­vant at this point! You have a 50% chance of be­ing right, just as you would ex­pect. Your first choice did ab­solutely noth­ing to in­fluence the out­come! This ar­gu­ment re­minds me of the time I bet \$100 on black at a roulette table be­cause it had come up red for like 20 con­sec­u­tive times, and of course it came up red again and I lost my \$\$. A guy at the table said to me “you re­ally think the lit­tle ball re­mem­bers what it pre­vi­ously did and avoids the red slots??”. Don’t fo­cus on the first choice, just look at the sec­ond—there’s two doors and you have to choose one (the one you already picked, or the other one). You got a 50% chance.

• Think about it this way. Let’s say you pre­com­mit be­fore we play Monty’s game that you won’t switch. Then you win 1/​3rd of the time, ex­actly when you picked the cor­rect door first, yes?

Now, sup­pose you pre­com­mit to switch­ing. Un­der what cir­cum­stances will you win? You’ll win if you didn’t pick the cor­rect door to start with. That means you have a 2/​3rd chance of win­ning since you win when­ever your first door wasn’t the cor­rect choice.

Your com­par­i­son to the roulette wheel doesn’t work: The roulette wheel has no mem­ory, but in this case, the car isn’t re­al­lo­cated be­tween the two re­main­ing doors, it was cho­sen be­fore the pro­cess started.

• Your anal­ogy doesn’t hold, be­cause each spin of the roulette wheel is a sep­a­rate trial, while choos­ing a door and then hav­ing the op­tion to choose an­other are causally linked.

If you’ve re­ally thought about XiXiDu’s analo­gies and they haven’t helped, here’s an­other; this is the one that made it ob­vi­ous to me.

Omega trans­mutes a sin­gle grain of sand in a sand­bag into a di­a­mond, then pours the sand equally into three buck­ets. You choose one bucket for your­self. Omega then pours the sand from one of his two buck­ets into the other one, throws away the empty bucket, and offers to let you trade buck­ets.

Each bucket analo­gizes to a door that you may choose; the sand analo­gizes to prob­a­bil­ity mass. Seen this way, it’s clear that what you want is to get as much sand (prob­a­bil­ity mass) as pos­si­ble, and Omega’s bucket has more sand in it. Monty’s un­opened door doesn’t in­herit any­thing tan­gible from the opened door, but it does in­herit the opened door’s prob­a­bil­ity mass.

• That works bet­ter for you? That’s deeply sur­pris­ing. Us­ing en­tities like Omega and trans­mu­ta­tion seems to make things more ab­stract and much harder to un­der­stand what the heck is go­ing on. I must need to mas­sively up­date my no­tions about what sort of de­scrip­tors can make things clear to peo­ple.

• I use en­tities out­side hu­man ex­pe­rience in thought ex­per­i­ments for the sake of pre­vent­ing Clever Hu­mans from try­ing to game the anal­ogy with their in­fer­ences.

“If Monty ‘re­placed’ a grain of sand with a di­a­mond then the di­a­mond might be near the top, so I choose the first bucket.”

“Monty wants to keep the di­a­mond for him­self, so if he’s offer­ing to trade with me, he prob­a­bly thinks I have it and wants to get it back.”

It might seem para­dox­i­cal, but us­ing ‘trans­mute at ran­dom’ in­stead of ‘re­place’, or ‘Omega’ in­stead of ‘Monty Hall’, ac­tu­ally sim­plifies the prob­lem for me by es­tab­lish­ing that all rele­vant facts to the prob­lem have already been in­cluded. That never seems to hap­pen in the real world, so the world of the anal­ogy is use­fully un­real.

• I re­ally like this tech­nique.

• I’m not keen on this anal­ogy be­cause you’re com­par­ing the effect of the new in­for­ma­tion to an agent freely choos­ing to pour sand in a par­tic­u­lar way. A con­fused per­son won’t un­der­stand why Omega couldn’t de­cide to dis­tribute sand some other way—e.g. equally be­tween the two re­main­ing buck­ets.

Any­way, I think JoshuaZ’s ex­pla­na­tion is the clear­est I’ve ever seen.

• “Your anal­ogy doesn’t hold, be­cause each spin of the roulette wheel is a sep­a­rate trial, while choos­ing a door and then hav­ing the op­tion to choose an­other are causally linked.”

No, they are not causally linked. It does not mat­ter what door you choose, you don’t in­fluence the out­come in any way at all. Ul­ti­mately, you have to choose be­tween two doors. In fact, you don’t “choose” a door at first at all. Be­cause there is always at least one goat be­hind a door you didn’t choose, you can­not in­fluence the next ac­tion, which is for Monty to open a door with a goat. At that point it’s a choice be­tween two doors.

• At this point you’ve had this ex­plained to you mul­ti­ple times. May I sug­gest that if you don’t get it at this point, maybe be a bit of an em­piri­cist and write a com­puter pro­gram to re­peat the game many times and see what frac­tion switch­ing wins? Or if you don’t have the skill to do that (in which case learn­ing to pro­gram should be on your list of things to learn how to do. It is very helpful and forces cer­tain forms of care­ful think­ing) play the game out with a friend in real life.

• play the game out with a friend in real life.

If log­i­cal wants to play for real money I vol­un­teer my ser­vices.

• If—and I mean do mean if, I wouldn’t want to spoil the em­piri­cal test—log­i­cal doesn’t un­der­stand the situ­a­tion well enough to pre­dict the cor­rect out­come, there’s a good chance he won’t be able to pro­gram it into a com­puter cor­rectly re­gard­less of his pro­gram­ming skill. He’ll pro­gram the com­puter to perform his mis­in­ter­pre­ta­tion of the prob­lem, and it will re­turn the re­sult he ex­pects.

On the other hand, if he’s right about the Monty Hall prob­lem and he pro­grams it cor­rectly… it will still re­turn the re­sult he ex­pects.

• He could try one of many already-writ­ten pro­grams if he lacks the skill to write one.

• Sure, but then the ques­tion be­comes whether the other pro­gram­mer got the pro­gram right...

My point is that if you don’t un­der­stand a situ­a­tion, you can’t re­li­ably write a good com­puter simu­la­tion of it. So if log­i­cal be­lieves that (to use your first link) James Tauber is wrong about the Monty Hall prob­lem, he has no rea­son to be­lieve Tauber can pro­gram a good simu­la­tion of it. And even if he can read Python code, and has no prob­lem with Tauber’s im­ple­men­ta­tion, log­i­cal might well con­clude that there was just some glitch in the code that he didn’t no­tice—which hap­pens to pro­gram­mers re­gret­tably of­ten.

I think im­ple­ment­ing the game with a friend is the bet­ter op­tion here, for ease of im­ple­men­ta­tion and strength of ev­i­dence. That’s all :)

• The thing you might be over­look­ing is that Monty does not open a door at ran­dom, he opens a door guaran­teed to con­tain a goat. When I first heard this prob­lem, I didn’t get it un­til that was ex­plic­itly pointed out to me.

If Monty opens a door at ran­dom (and the door could con­tain a car), then there is no causal link and there­fore the prob­a­bil­ity would be as you de­scribe.

• This prob­lem is not so difficult to solve if we use a bino­mial tree to try to tackle it. Not only we will come to the same (cor­rect) math­e­mat­i­cal an­swer (which is brilli­antly ex­posed in the first post in this thread) but log­i­cally is more palat­able.

I will ex­posed the log­i­cally, se­man­ti­cally de­rived an­swer straight away and then I will jump in to the bino­mial tree for the proof-out.

The prob­a­bil­ity of the situ­a­tion ex­posed here, which for the sake of be­ing brief I’m go­ing to put it as “con­tes­tant choose a door, a goat is re­vealed in a differ­ent door, then con­tes­tant switches from the origi­nal choice to win the car”, is the same prob­a­bil­ity as be­ing wrong in an sce­nario where the con­tes­tant only needs to choose one door and that’s it. This is 66%. The prob­a­bil­ity of the path the con­tes­tant needs to go through to win a car IF he/​she always switches is ex­actly the same prob­a­bil­ity as be­ing wrong in the first place.

Why?

There are two world-states right at be­gin­ning of the situ­a­tion. This is be­ing right, which means hav­ing cho­sen the door with the car be­hind with a prob­a­bil­ity of 33% and be­ing wrong, which means choos­ing a door with a goat be­hind with a prob­a­bil­ity of 66%. Be­ing wrong is the only world-state that will put the con­tes­tant in the right path IF and only IF he/​she then switches the door. Do­ing so guaran­tees (at a 100%) the con­tes­tant to choose the door with the car be­hind. That’s why is so counter-in­tu­itive be­cause be­ing wrong in the first place will in­crease (dou­ble!) the prob­a­bil­ity of win­ning the car given the path that the host is offer­ing. There­fore, if the path the con­tes­tant needs to take (be­ing wrong) for then to switch (switch­ing to only one door, the one left, there is no prob­a­bil­ity here as there is only one choice left) has a prob­a­bil­ity of 66% the an­swer to this prob­lem is 66%!

The bino­mial tree will illus­trate this bet­ter:

World-state #1 will never, ever, give the con­tes­tant a car if he/​she switches his/​her first choice. The fun­da­men­tal thing here and where it’s so easy to get con­fused is that the prob­lem is es­sen­tially defin­ing paths, not states.

Bino­mial trees are ex­cel­lent tools for real-life op­tions such as this ex­am­ple. How­ever they are mostly used to price op­tions and other fi­nan­cial in­stru­ments with some op­tion­al­ity em­bed­ded in them. The fork­ing paths can get re­ally com­plex. In this case it worked very well be­cause there are only two cy­cles and the sec­ond cy­cle of the sec­ond world-state has only one op­tion for win­ning the car. Many cy­cles and prob­a­bil­ities within the cy­cles and world-states will com­pound the com­plex­ity of these struc­tures. Re­gard­less, I find them very pow­er­ful tools to vi­su­ally rep­re­sent these kind of prob­lems .The key here is that this is a branch­ing-type prob­a­bil­is­tic prob­lem and the world-states are mu­tu­ally ex­clu­sive with their own prob­a­bil­ity dis­tri­bu­tions. Some­thing that the clas­sic prob­a­bil­is­tic anal­y­sis fails to rep­re­sent and make the an­a­lyst be­ing aware to, as it’s so very easy to see the prob­lem as one world, one situ­a­tion. They are not.

• Thanks for post­ing this. Your ex­pla­na­tions are fas­ci­nat­ing and helpful. That said, I do have one quib­ble. I was mis­led by the Two Aces prob­lem be­cause I didn’t know that the two un­known cards (2C and 2D) were pre­cluded from also be­ing aces or aces of spaces. It might be bet­ter to edit the post to make that clear.

• I have a small is­sue with the way you pre­sented the Monty python prob­lem. In my opinion, the setup could be a lit­tle clearer. The Bayesian model you pre­sented holds true iff you make an as­sump­tion about the door you picked; ei­ther goat (bet­ter) or car (less wrong). If you pick a door at ran­dom with no pre­sup­po­si­tions (I be­lieve this is the state most peo­ple are in), then you have no ba­sis to de­cide to switch or not switch, and have a truly 50% chance ei­ther way. If in­stead you in­tro­duce the as­sump­tion of goat, when the host opens up the other goat, you know you had a 23 chance to pick a goat. With both goats known or pre­sumed, the last door must be the car with an er­ror rate of 13.

• As far as I can see, one-third of the time the first door you picked had the car. What hap­pens af­ter­ward can­not change that one-third. The only way it could change your one-third cre­dence is if some­times Monty Hall did one thing and some­times an­other, de­pend­ing on whether you picked the car.

• While the over­all prob­a­bil­ities for the game will never change, the con­tes­tant’s per­cep­tion of the cur­rent state of the game will cause them to af­fect their win rate. To elab­o­rate on what I was say­ing, imag­ine the fol­low­ing in­ter­nal monologue of a con­tes­tant:

I’ve elimi­nated one goat. Two doors left. One is a goat, one is a car. No way to tell which is which, so I’ll just ran­domly pick one.

IMO, this is prob­a­bly what most con­tes­tants be­lieve when faced with the fi­nal choice. Ob­vi­ously, there is a way to have a greater suc­cess rate, but this per­son is eval­u­at­ing in a vac­uum. If con­tes­tants were aware of the ac­tual prob­a­bil­ities in­volved, I think we would see less “ag­o­niz­ing” mo­ments as the con­tes­tants de­cide if they should switch or not. By ran­domly pick­ing door A or B, ir­re­spec­tive of the en­tirety game, you’ve lost your marginal ad­van­tage and low­ered your win rate. That be­ing said, if they still “ran­domly” pick switch ev­ery their win rate will be the ex­pected, ac­tual prob­a­bil­ity.

Edit: The same be­havi­our can be seen in Deal or No Deal. If for some in­sane rea­son, they go all the way to the fi­nal two cases, the cor­rect choice is to switch. I don’t know ex­actly how many cases you have to choose from, but the odds are greatly against you that you picked the 1k case. If the case is still on the board, the chick is hold­ing it. Yet, peo­ple make the choice to switch based en­tirely on the fact that there are two cases and 1 has 1k and the other has 0.01. They think they have a 5050 shot, so they make their odds es­sen­tially 5050 by ran­domly choos­ing. In other words, they might as well as have flipped a coin to make the de­ci­sion be­tween the two cases.

• Ac­tu­ally, I just re­al­ized… there is no rea­son to swap on Deal or No Deal. The rea­son why you swap in Monty Hall is that Monty knows which door has the goats and there is no chance he will open a door to re­veal a car. But in Deal or No Deal the cases that get opened are cho­sen by the con­tes­tant with no knowl­edge of what is in­side them. It’s like if the con­tes­tant got to pick which of the two re­main­ing doors to open in­stead of Monty, there is a 13 chance the con­tes­tant would open the door with the car leav­ing her with only goats to choose from. The fact the the con­tes­tant got lucky and didn’t open the door with the car wouldn’t tell her any­thing about which of the two re­main­ing doors the car is re­ally be­hind.

ETA: Ba­si­cally Deal or No Deal is just a re­ally bor­ing game.

• Ba­si­cally Deal or No Deal is just a re­ally bor­ing game.

Well, it’s ex­cit­ing for those who like high-stakes ran­dom­ness. And there are ex­pected util­ity con­sid­er­a­tions at ev­ery op­por­tu­nity for a deal (I don’t re­mem­ber if there’s a con­sis­tent best choice based on the typ­i­cal deal).

• And there are ex­pected util­ity con­sid­er­a­tions at ev­ery op­por­tu­nity for a deal (I don’t re­mem­ber if there’s a con­sis­tent best choice based on the typ­i­cal deal).

• Maybe it could be in­ter­est­ing if you treat it as a psy­chol­ogy game—try­ing to pre­dict, based on the per­son’s ap­pear­ance, body lan­guage, and state­ments, whether they will con­form to ex­pected-util­ity or not?

• If for some in­sane rea­son, they go all the way to the fi­nal two cases

The way the deals work go­ing down to the fi­nal two cases can end up the best strat­egy. Ba­si­cally they weight the deals to en­courage higher rat­ings. As long as there is a big money case in play they won’t offer the con­tes­tant the full av­er­age of the cases- pre­sum­ably view­ers like to watch peo­ple play for the big money so the show wants these con­tes­tants to keep go­ing. If all the big money cases get used up the banker im­me­di­ately offers the con­tes­tant way more than they are worth to get them off the stage and make way for a new con­tes­tant.

(This was my con­clu­sion af­ter watch­ing un­til I thought I had figured it out. I could be read­ing this into a more com­plex or more ran­dom pat­tern.)

• Ac­tu­ally, peo­ple are much more likely to not switch if they think they are both equal. I’ve just finished read­ing Ja­son Rosen­house’s ex­cel­lent book on the Monty Hall Prob­lem and there’s a strong ten­dency for peo­ple not to switch. Ap­par­ently peo­ple are wor­ried that they’ll feel bad if they switch and that ac­tion causes them to lose. (The book does a very good job dis­cussing a lot about both the prob­lem and psych stud­ies about how peo­ple re­act to it or differ­ent ver­sions. I strongly recom­mend it.)

• On the ac­tual show, some­times Monty Hall did one thing and some­times an­other, so far as I am told. We’re not talk­ing about ac­tual be­hav­ior of con­tes­tants in an ac­tual con­test, we’re talk­ing about op­ti­mal be­hav­ior of con­tes­tants in a fic­tional con­test.

Edit: I’m sorry, I re­ally don’t know what you’re ar­gu­ing. Am I mak­ing sense?

• You are mak­ing perfect sense; it’s me that is not. I had thought to clar­ify the is­sue for peo­ple that might still not “get it” af­ter read­ing the ar­ti­cle. In­stead, I’ve only mud­died the wa­ters.

• Well the Monty Hall prob­lem as stated never oc­curred on Let’s Make a Deal. He’s even on record say­ing he won’t let a con­tes­tant switch doors af­ter pick­ing, in re­sponse to this prob­lem.

• Robin’s de­scrip­tion is cor­rect. I’m not sure what you’re say­ing.

ETA: this thread has got­ten ridicu­lous. I’m delet­ing the rest of my com­ments on it. The best source for info on Monty Hall is youtube. He does ev­ery­thing. One thing that makes it rather differ­ent is that it is usu­ally not clear how many good and bad prizes there are.

• I’m re­ally shocked by the re­ac­tions of the math­e­mat­i­ci­ans. I re­mem­ber solv­ing that prob­lem in like the third week of my In­tro to Com­puter Science Class. And be­fore that I had heard of it and thought through why it was worth switch­ing. I didn’t re­al­ize it caused so much con­fu­sion as re­cently as 20 years ago.

• The prob­lem causes a lot of con­fu­sion. There are stud­ies which show that this is in fact cross-cul­tural. It seems to deeply con­flict with a lot of heuris­tics hu­mans use for work­ing out prob­a­bil­ity. See Don­ald Gran­berg, “Cross-Cul­tural Com­par­i­son of Re­sponses to the Monty Hall Dilemma” So­cial Be­hav­ior and Per­son­al­ity, (1999), 27:4 p 431-448. There are other rele­vant refer­ences in Ja­son Rosen­house’s book “The Monty Hall Prob­lem.” The prob­lem clashes with many com­mon heuris­tics. It isn’t that sur­pris­ing that some math­e­mat­i­ci­ans have had trou­ble with it. (Although I do think it is sur­pris­ing that some of the math­e­mat­i­ci­ans who have had trou­ble were peo­ple like Er­dos who was un­am­bigu­ously first-class)

• Erdos

Wow! I looked this up and it turns out it’s de­scribed in a book I read a long time ago, The Man Who Loved Only Num­bers (do a “Search In­side This Book” for “Monty Hall”). Edit: In this book, the phrase “Book proof” refers to a max­i­mally el­e­gant proof, seen as be­ing in “God’s Book of Proofs”.

I en­coun­tered the prob­lem for the first time in a col­lec­tion of vos Sa­vant’s Pa­rade pieces. It was un­in­tu­itive of course, but most strik­ing for me was the ut­ter un­con­vin­ci­bil­ity of some of the peo­ple who wrote to her.

• the ut­ter unconvincibility

Yes, my fal­lback if my in­tu­ition on a prob­a­bil­ity prob­lem seems to fail me is always to code a quick simu­la­tion—so far, it’s always taken on about a minute to code and run. That any­one both­ered to write her a let­ter, even way back in the 70′s, is mind-bog­gling.

• Yeah it’s re­mark­able isn’t it?

I sup­pose the thing about the Monty-Hall prob­lem which makes it ‘difficult’ is that there is an­other agent with more in­for­ma­tion than you, who gives you a sys­tem­at­i­cally ‘bi­ased’ ac­count of their in­for­ma­tion. (There’s an el­e­ment of ‘de­ceit­ful­ness’ in other words.)

An anal­ogy: Sup­pose you had a coin which you knew was ei­ther 23 bi­ased to­wards heads or 23 bi­ased to­wards tails, and the bias is ac­tu­ally to­wards heads. Say there have been 100 coin tosses, and you don’t know any of the out­comes but some­one else (“Monty”) knows them all. Then they can feed you ‘bi­ased in­for­ma­tion’ by choos­ing a sam­ple of the coin tosses in which most out­comes were tails. The analo­gous con­fu­sion would be to ig­nore this pos­si­bil­ity and as­sume that Monty is ‘hon­estly’ tel­ling you ev­ery­thing he knows.

• Ex­pert con­fi­dence. I read vos Sa­vants book with all the let­ters she got and like how the prob­lem seems to re­ally be a test for the men­tal clar­ity and po­lite­ness of the ac­tors in­volved.

Any­one knows of prob­lems that get similarly vi­o­lent re­ac­tions from ex­perts?

• From Wikipe­dia:

Monty Hall did open a wrong door to build ex­cite­ment, but offered a known lesser prize—such as \$100 cash—rather than a choice to switch doors. As Monty Hall wrote to Selvin: And if you ever get on my show, the rules hold fast for you—no trad­ing boxes af­ter the se­lec­tion. (Hall 1975)

The cita­tion is from a let­ter from Monty him­self, available on­line here.

I’m not sure how the ar­ti­cle you linked to is rele­vant. It does de­scribe an in­stance of Monty Hall ac­tu­ally perform­ing the ex­per­i­ment, but it was in his home, not on the show.

• Was Mr. Hall cheat­ing? Not ac­cord­ing to the rules of the show, be­cause he did have the op­tion of not offer­ing the switch, and he usu­ally did not offer it.

ex­actly as Robin said.

• thomblake’s re­mark was rele­vant too, though—from what I said, you might imag­ine that Monty Hall let peo­ple switch on the show. All the clar­ifi­ca­tions are rele­vant and good.

• Yes, you might imag­ine that, and you’d prob­a­bly be right. Thom’s quote is ev­i­dence against that claim, but very weak.

• Aaargh! And I had up­voted that, be­liev­ing a ran­dom In­ter­net com­ment over a re­li­able offline source! That’s a lit­tle em­bar­rass­ing.

The ar­ti­cle is awe­some, by the way. Thanks!

• My God. I never both­ered read­ing your ex­pla­na­tion be­cause you spent so much time whin­ing about how ev­ery­one else had failed you. I sug­gest you just en­roll in uni­ver­sity and learn the math like the rest of us.

• Are you se­ri­ous? Are you buy­ing this? Ok—let me make this easy: There NEVER WAS a 33% chance. Ever. The 1-in-3 choice is a ruse. No mat­ter what door you choose, Monty has at least one door with a goat be­hind it, and he opens it. At that point, you are pre­sented with a 1-in-2 choice. The prior choice is com­pletely ir­rele­vant at this point! You have a 50% chance of be­ing right, just as you would ex­pect. Your first choice did ab­solutely noth­ing to in­fluence the out­come! This ar­gu­ment re­minds me of the time I bet \$100 on black at a roulette table be­cause it had come up red for like 20 con­sec­u­tive times, and of course it came up red again and I lost my \$\$. A guy at the table said to me “you re­ally think the lit­tle ball re­mem­bers what it pre­vi­ously did and avoids the red slots??”. Don’t fo­cus on the first choice, just look at the sec­ond—there’s two doors and you have to choose one (the one you already picked, or the other one). You got a 50% chance. (by the way—sorry if I posted this twice?? Or in the wrong place?)

• You got a 50% chance.

No, you don’t. Switch­ing gives you the right door 2 out of 3 times. Long be­fore read­ing this ar­ti­cle, I was con­vinced by a pro­gram some­body wrote that ac­tu­ally simu­lates it by count­ing up how many times you would win or lose in that situ­a­tion… and it comes out that you win by switch­ing, 2 out of 3 times.

So, the in­ter­est­ing ques­tion at that point is, why does it work 2 out of 3 times?

And so now, you have an op­por­tu­nity to learn an­other rea­son why your in­tu­ition about prob­a­bil­ities is wrong. It’s not just the lack of “mem­ory” that makes prob­a­bil­ities weird. ;-)