# The Prediction Hierarchy

The sub­stance of this post is de­rived from a con­ver­sa­tion in the com­ment thread which I have de­cided to pro­mote. Teal;deer: if you have to rely on a calcu­la­tion you may have got­ten wrong for your pre­dic­tion, your ex­pec­ta­tion for the case when your calcu­la­tion is wrong should use a sim­pler calcu­la­tion, such as refer­ence class fore­cast­ing.

Edit 2010-01-19: Toby Ord men­tions in the com­ments Prob­ing the Im­prob­a­ble: Method­olog­i­cal Challenges for Risks with Low Prob­a­bil­ities and High Stakes (PDF) by Toby Ord, Ra­faela Hiller­brand, and An­ders Sand­berg of the Fu­ture of Hu­man­ity In­sti­tute, Univer­sity of Oxford. It uses a similar math­e­mat­i­cal ar­gu­ment, but is much more sub­stan­tive than this.

A lot­tery has a jack­pot of a mil­lion dol­lars. A ticket costs one dol­lar. Odds of a given ticket win­ning are ap­prox­i­mately one in forty mil­lion. If your util­ity is lin­ear in dol­lars, should you bet?

The ob­vi­ous (and cor­rect) an­swer is “no”. The clever (and in­cor­rect) an­swer is “yes”, as fol­lows:

Ac­cord­ing to your calcu­la­tions, “this ticket will not win the lot­tery” is true with prob­a­bil­ity 99.9999975%. But can you re­ally be sure that you can calcu­late any­thing to that good odds? Surely you couldn’t ex­pect to make forty mil­lion pre­dic­tions of which you were that con­fi­dent and only be wrong once. Ra­tion­ally, you ought to as­cribe a lower con­fi­dence to the state­ment: 99.99%, for ex­am­ple. But this means a 0.01% chance of win­ning the lot­tery, cor­re­spond­ing to an ex­pected value of a hun­dred dol­lars. There­fore, you should buy the ticket.

The logic is not ob­vi­ously wrong, but where is the er­ror?

First, let us write out the calcu­la­tion alge­braically. Let E(L) be the ex­pected value of play­ing the lot­tery. Let p(L) be your calcu­lated prob­a­bil­ity that the lot­tery will pay off. Let p(C) be your prob­a­bil­ity that your calcu­la­tions are cor­rect. Fi­nally, let j rep­re­sent the value of the jack­pot and let t rep­re­sent the price of the ticket. The ob­vi­ous way to write the clever the­ory is:

E(L) = max(p(L), 1-p(C)) * j—t

This doesn’t sound quite right, though—surely you should as­cribe a higher con­fi­dence when you calcu­late a higher prob­a­bil­ity. That said, when p(L) is much less than p(C), it shouldn’t make a large differ­ence. The straight­for­ward way to ac­count for this is to take p(C) as the prob­a­bil­ity that p(L) is cor­rect, and write the fol­low­ing:

E(L) = [ p(C)*p(L) + 1-p(C) ] * j—t

which can be re­ar­ranged as:

E(L) = p(C) * [p(L)*j—t] + (1-p(C)) * [j—t]

I be­lieve this ex­poses the prob­lem with the clever ar­gu­ment quite ex­plic­itly. Why, if your calcu­la­tions are in­cor­rect (prob­a­bil­ity 1-p(C)), should you as­sume that you are cer­tain to win the lot­tery? If your calcu­la­tions are in­cor­rect, they should tell you al­most noth­ing about whether you will win the lot­tery or not. So what do you do?

What ap­pears to me the el­e­gant solu­tion is to use a less com­plex calcu­la­tion—or a se­ries of less com­plex calcu­a­tions—to act as your backup hy­poth­e­sis. In a tricky en­g­ineer­ing prob­lem (say, calcu­lat­ing the effec­tive­ness of a heat sink), your pri­mary pre­dic­tion might come out of a finite el­e­ment fluid dy­nam­ics calcu­la­tor with p(C) = 0.99 and nar­row er­ror bars, but you would also re­fer to the re­sult of a sim­ple alge­braic model with p(C) = 0.9999 and much wider er­ror bars. And then you would back­stop the lot with your back­ground knowl­edge about heat sinks in gen­eral, writ­ten with wide enough er­ror bars to call p(C) = 1 - ep­silon.

In this case, though, the calcu­la­tion was sim­ple, so our backup pre­dic­tion is just the back­ground knowl­edge. Say that, know­ing noth­ing about a lot­tery but “it’s a lot­tery”, we would have an ex­pected pay­off e. Then we write:

E(L) = p(C) * [p(L)*j—t] + (1-p(C)) * e

I don’t know about you, but for me, e is ap­prox­i­mately equal to -t. And jus­tice is re­stored.

We are ad­vised that, when solv­ing hard prob­lems, we should solve mul­ti­ple prob­lems at once. This is rel­a­tively triv­ial, but I can point out a cou­ple other rel­a­tively triv­ial ex­am­ples where it shows up well:

Sup­pose the lot­tery ap­pears to be marginally prof­itable: should you bet on it? Not un­less you are con­fi­dent in your num­bers.

Sup­pose we con­sider the LHC. Should we (have) switch(ed) it on? Once you’ve checked that it is safe, yes. As a high-en­ergy physics ex­per­i­ment, the backup com­par­i­son would be to things like nu­clear en­ergy, which have only small chances of dev­as­ta­tion on the plane­tary scale. If your calcu­la­tions were to in­di­cate that the LHC is com­pletely safe, even if your P(C) were as low as three or four nines (99.9%, 99.99%), your ac­tual es­ti­mate of the safety of turn­ing it on should be no lower than six or seven nines, and prob­a­bly higher. (In point of fact, given the num­ber of physi­cists an­a­lyz­ing the ques­tion, P(C) is much higher. Three cheers for in­ter­sub­jec­tive ver­ifi­ca­tion.)

Sup­pose we con­sider our Christ­mas shop­ping? When you’re es­ti­mat­ing your time to finish your shop­ping, your calcu­la­tions are not very re­li­able. There­fore your an­swer is strongly dom­i­nated by the sim­pler, much more re­li­able refer­ence class pre­dic­tion.

But what are the odds that this ticket won’t win the lot­tery? …how many nines do I type, again?

• You may wish to check out the pa­per we wrote at the FHI on the prob­lem of tak­ing into ac­count mis­takes in one’s own ar­gu­ment. The math­e­mat­i­cal re­sult is the same as the one here, but the proof is more com­pel­ling. Also, we demon­strate that when ap­plied to the LHC, the re­sult is very differ­ent to the above anal­y­sis.

http://​​www.fhi.ox.ac.uk/​​__data/​​as­sets/​​pdf_file/​​0006/​​4020/​​prob­ing-the-im­prob­a­ble.pdf

• I haven’t read the pa­per through, but the similar­ity in alge­bra can­not be de­nied. I have added a refer­ence to the post.

• Thanks, in­ter­est­ing read. Could you ex­pand more on the points of similar­ity and differ­ence be­tween your ar­gu­ment and RobinZ’s? They cur­rently seem very dis­parate ap­proaches to me.

• Why, when you con­sider the case where you calcu­lated the odds of win­ning the lot­tery in­cor­rectly, do you in­crease rather than de­crease the odds?

In any case, with a lot­tery, you do know the odds of win­ning; they’re stated on the ticket.

At the mo­ment, I’m calcu­lat­ing my ex­pected value, not the odds, but there are a num­ber of rea­sons to think that jack­pot /​ stated-odds is op­ti­mistic:

1. The lot­tery may be a fraud.

2. The lot­tery may go bust.

3. I may lose the ticket.

4. I may have to split the pot.

In gen­eral, the rigor­ous ap­proach would be to rewrite ev­ery­thing as prob­a­bil­ity dis­tri­bu­tions.

Be­sides: if you want to as­sume the av­er­age lot­tery ticket is more valuable that I would—e = −0.5*t, say—that’s your right. I make no jus­tifi­ca­tion for my pri­ors.

• You quoted this from some­where:

Ac­cord­ing to your calcu­la­tions, “this ticket will not win the lot­tery” is true with prob­a­bil­ity 99.9999975%. But can you re­ally be sure that you can calcu­late any­thing to that good odds? Surely you couldn’t ex­pect to make forty mil­lion pre­dic­tions of which you were that con­fi­dent and only be wrong once. Ra­tion­ally, you ought to as­cribe a lower con­fi­dence to the state­ment: 99.99%, for ex­am­ple.

This says that ra­tio­nally, you should as­sign a much higher ex­pected value to the ticket. But all 4 fac­tors you just listed are ones which would make the ex­pected value of the ticket lower.

• Oh, I see what you mean. That wasn’t a quote, ac­tu­ally—it was es­sen­tially an ar­tic­u­la­tion of ci­pher­goth’s clever (but in­cor­rect) ar­gu­ment. The pur­pose of this post was to ex­plain my method for re­but­ting it.

• It’s just a restate­ment of the Pas­cal’s Mug­ging prob­lem, but with the lot­tery in place of the mug­ging.

• I’m still am­biva­lent about Pas­cal’s Mug­ging, how­ever—my in­stinct is to re­fuse to pay, but I don’t feel I can suffi­ciently jus­tify that re­sponse.

The lot­tery, as an or­di­nary situ­a­tion, is far more tractable.

• Can’t you ap­ply a similar ar­gu­ment? In­stead of con­sid­er­ing P(mug­ger’s state­ment is true), you con­sider P(you have the fain­test idea what’s go­ing on).

• My in­stinc­tive prob­a­bil­ity mea­sure­ment for such a state­ment is not so small as 1/​3^^^^3. My best re­tort at the mo­ment is purely prag­matic: never ac­cept such a mug­ging, be­cause oth­er­wise you will be mugged.

• In­deed, the prob­a­bil­ity that we don’t know what’s go­ing on is non-neg­ligible. What I’m sug­gest­ing is that we don’t have to as­sign a non-neg­ligible prob­a­bil­ity to the spe­cific hy­poth­e­sis “this mug­ger is speak­ing the literal truth”—in­stead we avoid over­con­fi­dence by try­ing to con­sider all of the hy­pothe­ses that might hide be­hind the gen­eral as­ser­tion “our grasp on this situ­a­tion is much less than we think” and try to use broader refer­ence classes to see what the out­comes of var­i­ous strate­gies might be in those in­stances, us­ing the strat­egy you out­line for the lot­tery.

• Not to en­gage in need­less turn­about, but how does that trans­late into math?

• In­stead of think­ing of the propo­si­tion H = the mug­ger is hon­est, and try­ing to calcu­late E(U|AH)P(H) + E(U|A~H)P(~H) where A is an ac­tion such as hand­ing over your wallet and U is util­ity, you con­sider the hy­pothe­ses you’re re­ally ap­ply­ing, T, that your gen­eral the­ory about mug­gers is suffi­cient to un­der­stand the situ­a­tion, and ~T, that you just don’t have a han­dle on the situ­a­tion. Then in­stead of di­rectly us­ing the mug­ger’s stated util­ity for the value you try to ap­peal to a sim­pler and more gen­eral the­ory to find a value for E(U|A~T). The more gen­eral “an offer from a stranger” refer­ence class should suffice; you buy only a tiny minor­ity of the things that you’re offered. Beyond that you have the “don’t know” refer­ence class, but that has to have ex­pected zero util­ity.

This ar­gu­ment doesn’t ap­ply to any pos­si­bil­ity that you are in a po­si­tion to prop­erly think about. You are in a po­si­tion to as­sess the prob­a­bil­ity that Miriam Ach­aba wishes to en­trust you with 25M USD, so you’re best ad­vised not to reach for the “other” column on that one.

• Agreed—the trick is that be­ing wrong “only once” is de­cep­tive. I may be wrong more than once on a one-in-forty-mil­lion chance. But I may also be wrong zero times in 100 mil­lion tries, on a prob­lem as fre­quent and well-un­der­stood as the lot­tery, and I’m hes­i­tant to say that any read­ing prob­lems I may have would bias the test to­ward more lu­cra­tive mis­takes.

• Let’s rephrase, then. Sup­pose for a mo­ment that you are 100% con­fi­dent a lot­tery ticket costs \$1, you can buy it, it pays \$10^6 on a win, etc etc and that you are read­ing the ticket right now and be­lieve it says the prob­a­bil­ity the ticket will win is 1/​(4x10^6). Should you be­lieve the ticket is +EV?

The wrong calcu­la­tion: Yes, be­cause you es­ti­mate you’ll mis­read the ticket (or it’s ly­ing, etc etc) 1 in a mil­lion times, which makes the EV 10^6 x (10^-6 + (1-10^-6) x 1/​(4x10^6)) = 1 + ~0.25.

The right calcu­la­tion: No, be­cause you’ll mis­read the ticket 1 in a mil­lion times, which makes the EV 10^6(10^-6 x *P + (1-10^-6) x 1/​(4x10^6)) = P + ~0.25 where P is what­ever prob­a­bil­ity of win­ning with 1 ticket you as­sign to an ar­bi­trary lot­tery that costs \$1 and pays \$10^6 where you in­cor­rectly read the prob­a­bil­ity off the back of the ticket as be­ing 10^-6 (or it’s ly­ing, etc etc). If your pri­ors say P ~= 1 then they need ad­just­ing; if they say P ~= 10^-7 to 10^-6 then they prob­a­bly don’t need ad­just­ing. And then the EV is ~= 0.25 again.

AFAICT this is the same as in the post, but I’m not cer­tain I un­der­stand pre­cisely where your ques­tion is.

Edit: ha, I put 10^-5 to 10^-6 (which is of course silly) in­stead of 10^-7 to 10^-6, but RobinZ put ~0 anyway

• I must con­fer on you the high­est form of praise among as­piring ra­tio­nal­ists:

“Damn it, why didn’t I think of that?”

• Thank you!

A lot of credit has to go to ci­pher­goth and Wei_Dai, ac­tu­ally—I was just try­ing to run a stack trace on my in­stan­ta­neous re­jec­tion of the origi­nal lot­tery ar­gu­ment; they’re the ones who made me quan­tify it.

• Se­conded. In fact I’m al­most tempted to de­clare that this su­per­sedes my post

• Great work! That said, I’m sus­pi­cious of this for be­ing too con­ve­nient—I’m given cause to worry by the way I like the an­swers it gives too much. It al­most seems to make the stan­dard cau­tion against over­con­fi­dence dis­ap­pear from our calcu­la­tions al­to­gether, es­pe­cially in the cases where it’s hard­est to think about. And it gets us into “refer­ence class ten­nis” again.

• That’s a good point—stan­dard cau­tions against over­con­fi­dence should re­duce p(C), much like time pres­sure, ex­haus­tion, and com­plex­ity of ar­gu­ment.

• So, upon learn­ing that my calcu­la­tions were wrong, am I cor­rect in say­ing that my new prob­a­bil­ity es­ti­mate—be­fore do­ing any fur­ther calcu­la­tions—should be­come what­ever my prior prob­a­bil­ity was be­fore I did the calcu­la­tion?

• Let me be more pre­cise: be­fore you see any­thing wrong with your calcu­la­tions, you have no real rea­son to ex­pect lo­cat­ing an er­ror in them to give you ev­i­dence of any­thing spe­cific. There­fore, when do­ing your ini­tial post-calcu­la­tions, the prior prob­a­bil­ity is ap­pro­pri­ate.

After you find an er­ror in your calcu­la­tions, you can usu­ally fix the er­ror in your calcu­la­tions.

• Not quite. It de­pends on your be­liefs about how the calcu­la­tion could go wrong and how much this would change the re­sult. If you are very con­fi­dent in all parts ex­cept a minor cor­rect­ing term, and are sim­ply told that there is an er­ror in the calcu­la­tion, then you can still have some kind of rough con­fi­dence in the re­sult (you can see how to spell this out in maths). If you know the ex­act part of the calcu­la­tion that was mis­taken, then the situ­a­tion is slightly differ­ent, but still not iden­ti­cal to re­vert­ing to your prior.

• I be­lieve that the anal­y­sis of this prob­lem can be made more math­e­mat­i­cally rigor­ous than is done in this post. Not only will a for­mal anal­y­sis help us avoid prob­lem’s in our rea­son­ing, but it will clearly illus­trate what as­sump­tions have been made (so we can ques­tion their le­gi­t­i­macy).

Let’s as­sume (as is done im­plic­itly in the post) that you know with 100% cer­tainty that the only two pos­si­ble pay­outs are \$1 mil­lion and \$0. Then:

ex­pected earn­ings = p(\$1 mil­lion pay­out) \$1 mil­lion + p(\$0 pay­out) \$0 - (ticket price)

= p(\$1 mil­lion pay­out) * \$1 mil­lion - (ticket price)

= p(\$1 mil­lion pay­out|cor­rectly com­puted odds) p(cor­rectly com­puted odds) * \$1 mil­lion

• p(\$1 mil­lion pay­out|in­cor­rectly com­puted odds) p(in­cor­rectly com­puted odds) * \$1 mil­lion

• (ticket price)

= (1/​40,000,000) p(cor­rectly com­puted odds) * \$1 mil­lion

• p(\$1 mil­lion pay­out|in­cor­rectly com­puted odds) (1 - p(cor­rectly com­puted odds)) * \$1 mil­lion

• (ticket price)

We note now that we can write:

p(\$1 mil­lion pay­out|in­cor­rectly com­puted odds) (1 - p(cor­rectly com­puted odds)) \$1 mil­lion = p(\$1 mil­lion pay­out|in­cor­rectly com­puted odds) \$1 mil­lion (1 - p(cor­rectly com­puted odds)) = (p(\$1 mil­lion pay­out|in­cor­rectly com­puted odds) \$1 mil­lion + p(\$0 pay­out|in­cor­rectly com­puted odds) \$0) (1 - p(cor­rectly com­puted odds)) = (ex­pected pay­out given in­cor­rectly com­puted odds) (1 - p(cor­rectly com­puted odds))

Hence, our re­sult­ing equa­tion is:

ex­pected earn­ings = (1/​40,000,000) p(cor­rectly com­puted odds) * \$1 mil­lion

• (ex­pected pay­out given in­cor­rectly com­puted odds) (1 - p(cor­rectly com­puted odds))

• (ticket price)

Now, un­der the fairly rea­son­able (but not quite true) as­sump­tion (which seems to be im­plic­itly made by the au­thor) that

(ex­pected pay­out given in­cor­rectly com­puted odds) = (ex­pected pay­out given that we know noth­ing ex­cept that we are deal­ing with a lotto that costs (ticket price) to play)

we can con­vert to the no­ta­tion of the ar­ti­cle, which gives us:

E(L) = p(C) p(L) j + (1 - p(C)) * (e + t) - t

Here I have in­ter­preted e as the ex­pected value given that we are deal­ing with a lotto that we know noth­ing else about (rather than ex­pected earn­ings un­der those cir­cum­stances). The au­thor de­scribes e as an “ex­pected pay­off” but I don’t think that is re­ally quite what was meant (un­less “pay­off” re­turns to to­tal net pay­off in­clud­ing the ticket price).

We can now re­ar­range this for­mula:

E(L) = p(C) p(L) j + (1 - p(C)) e + (1 - p(C)) t—t = p(C) p(L) j + (1 - p(C)) e + (1 - p(C)) t—t = p(C) p(L) j + (1 - p(C)) e—p(C) t = p(C) ( p(L) j—t) + (1 - p(C)) e

which fi­nally gets us to the au­thor’s ter­mi­nal for­mula.

What is the point of do­ing this care­ful, for­mal anal­y­sis? Well, we now see where the au­thor’s for­mula comes from ex­plic­itly, it is proven rigor­ously, and we are fully aware of what as­sump­tions were made. The as­sump­tions are:

1. You know with 100% cer­tainty that the only two pos­si­ble pay­outs are \$1 mil­lion and \$0

and

1. ex­pected pay­out given in­cor­rectly com­puted odds = ex­pected pay­out given that we know noth­ing ex­cept that we are deal­ing with a lotto that costs the given ticket price to play

The first as­sump­tion is rea­son­able as­sum­ing that lotto is not fraud­u­lent, you don’t have prob­lems read­ing the rules, it is not pos­si­ble for mul­ti­ple peo­ple to claim the pay­out, etc.

The sec­ond as­sump­tion, how­ever, is harder to jus­tify. There are many ways that a calcu­la­tion of odds could go wrong (putting a dec­i­mal point in the wrong place, mak­ing a mul­ti­pli­ca­tion er­ror, un­know­ingly mi­s­un­der­stand­ing the laws of prob­a­bil­ity, ac­tu­ally be­ing in­sane, etc.) If we could re­ally enu­mer­ate all of them, un­der­stand how they effect our com­puted pay­out prob­a­bil­ity, and es­ti­mate the prob­a­bil­ity of each oc­cur­ring, then we could com­pute this miss­ing fac­tor ex­actly. As things stand though, it is prob­a­bly un­ten­able. It should not be ex­pected though that er­rors that make the pay­out prob­a­bil­ity ar­tifi­cially larger will bal­ance those that make it ar­tifi­cially smaller. Mis­plac­ing a dec­i­mal point, for ex­am­ple, will al­most cer­tainly be no­ticed if it leads to a per­centage greater than 100%, but not if it leads to one that is less than that (cre­at­ing an asym­me­try).

• The sec­ond as­sump­tion, how­ever, is harder to jus­tify. There are many ways that a calcu­la­tion of odds could go wrong (putting a dec­i­mal point in the wrong place, mak­ing a mul­ti­pli­ca­tion er­ror, un­know­ingly mi­s­un­der­stand­ing the laws of prob­a­bil­ity, ac­tu­ally be­ing in­sane, etc.) If we could re­ally enu­mer­ate all of them, un­der­stand how they effect our com­puted pay­out prob­a­bil­ity, and es­ti­mate the prob­a­bil­ity of each oc­cur­ring, then we could com­pute this miss­ing fac­tor ex­actly. As things stand though, it is prob­a­bly un­ten­able. It should not be ex­pected though that er­rors that make the pay­out prob­a­bil­ity ar­tifi­cially larger will bal­ance those that make it ar­tifi­cially smaller. Mis­plac­ing a dec­i­mal point, for ex­am­ple, will al­most cer­tainly be no­ticed if it leads to a per­centage greater than 100%, but not if it leads to one that is less than that (cre­at­ing an asym­me­try).

This is a valid point, and one I missed in my writeup. (Toby_Ord said some­thing similar, but that was in re­sponse to a spe­cific ques­tion.)

It is prob­a­bly a use­ful skill to rec­og­nize asym­me­tries in the pos­si­ble di­rec­tion of er­ror, such as that which you pointed out. I can see two ways to han­dle this:

a. Ad­di­tional terms in the deriva­tion, such as P(dec­i­mal-point er­ror) and P(sign er­ror), with the e term re­stricted to the unan­ti­ci­pated-er­ror case.
b. Mod­ifi­ca­tion of e.

• Surely you couldn’t ex­pect to make forty mil­lion pre­dic­tions of which you were that con­fi­dent and only be wrong once.

If it were 40 mil­lion pre­dic­tions about lot­ter­ies of that size I could.

• Don’t be so over­con­fi­dent, ap­par­ently in 1997 New York ran a pro­mo­tion that dou­bled the pay­outs of a sin­gle game on ex­actly 4 days in ex­actly 1 month, and payed out \$1.40 to \$1 on av­er­age.

• That’s a differ­ent game though, where the odds of win­ning aren’t one in 40 mil­lion.

• My grasp of statis­tics is atro­cious, some­thing I hope to im­prove this year with an open uni­ver­sity maths course, so apolo­gies if this is a dumb ques­tion:

Do the figures change if you take “play­ing the lot­tery” as over the whole of your lifes­pan? I mean, most of the peo­ple I know who play the lot­tery make a com­mit­ment to play reg­u­larly. Is the calcu­la­tion af­fected in any mean­ingful way? At least the costs of play­ing the lot­tery weekly over say 20 years be­come much less triv­ial in ap­pear­ance

• If by ‘do the figures change’ you mean ‘does it ever be­come a good bet’ then no.

• Your odds of win­ning once go up as you in­crease the num­ber of tick­ets you buy (# of tick­ets pur­chased * Chance of win­ning per ticket). The ex­pected value of a given ticket re­mains the same. All you are do­ing is fo­cus­ing more money away from other pos­si­bil­ities. If you buy 5 tick­ets a week for your en­tire life, and the odds of win­ning are 1 in 100 mil­lion, then you have a 0.000169 chance of win­ning the lot­tery, but you could have spent your 16 thou­sand on a new TV or a va­ca­tion.

• It comes out to about the right num­ber in this case, but your math is wrong. The ex­pected num­ber of times you win in n tri­als at prob­a­bil­ity p equals np, but the prob­a­bil­ity of win­ning at least once is slightly less at 1-(1-p)^n.

• Yes, thanks for the cor­rec­tion.

• As mat­tnew­port and Lu­casSloan point out, it doesn’t change the ac­tual num­bers—a bad bet mul­ti­plied a thou­sand­fold is still a bad bet—but it does change the wrong num­bers: buy­ing a thou­sand tick­ets for a 0.01% chance of a mil­lion dol­lars is a los­ing bet again.* More ev­i­dence that the ig­no­rance ar­gu­ment fails.

* How I calcu­late this (changes in ital­ics):

Ac­cord­ing to your calcu­la­tions, “none of these thou­sand tick­ets will win the lot­tery” is true with prob­a­bil­ity 99.9975000312185%. But can you re­ally be sure that you can calcu­late any­thing to that good odds? Surely you couldn’t ex­pect to make forty thou­sand pre­dic­tions of which you were that con­fi­dent and only be wrong once. Ra­tion­ally, you ought to as­cribe a lower con­fi­dence to the state­ment: 99.99%, for ex­am­ple. But this means a 0.01% chance of win­ning the lot­tery, cor­re­spond­ing to an ex­pected value of a hun­dred dol­lars. There­fore, … these thou­sand tick­ets still lose, be­cause you spend a thou­sand to win a hun­dred.

• You some­how make an as­sump­tion that mak­ing a fault of calcu­lat­ing the util­ity of the lot­tery ticket to low is more likely than mak­ing a fault of calcu­lat­ing it to high.

In prin­ci­ple those two sorts of pos­si­ble er­rors should bal­ance each other out.

• I make no such point. If you read the post, nowhere do I as­sume any spe­cific re­la­tion be­tween [p(L)*jt] and e—my point is speci­fi­cally that you should use some­thing with no de­pen­dence on your calcu­la­tion (and strictly more re­li­able) to draw con­clu­sions from when you’re not sure.

• A non-math­e­mat­i­cal rule of thumb for the same situ­a­tion might be the idea that if you can’t be very cer­tain of the nom­i­nal odds, then you can’t be very cer­tain of ac­tu­ally re­ceiv­ing the pay­off ei­ther.