Probabilities Small Enough To Ignore: An attack on Pascal’s Mugging

Sum­mary: the prob­lem with Pas­cal’s Mug­ging ar­gu­ments is that, in­tu­itively, some prob­a­bil­ities are just too small to care about. There might be a prin­ci­pled rea­son for ig­nor­ing some prob­a­bil­ities, namely that they vi­o­late an im­plicit as­sump­tion be­hind ex­pected util­ity the­ory. This sug­gests a pos­si­ble ap­proach for for­mally defin­ing a “prob­a­bil­ity small enough to ig­nore”, though there’s still a bit of ar­bi­trari­ness in it.

This post is about find­ing a way to re­solve the para­dox in­her­ent in Pas­cal’s Mug­ging. Note that I’m not talk­ing about the bas­tardized ver­sion of Pas­cal’s Mug­ging that’s got­ten pop­u­lar of late, where it’s used to re­fer to any ar­gu­ment in­volv­ing low prob­a­bil­ities and huge stakes (e.g. low chance of thwart­ing un­safe AI vs. as­tro­nom­i­cal stakes). Nei­ther am I talk­ing speci­fi­cally about the “mug­ging” illus­tra­tion, where a “mug­ger” shows up to threaten you.
Rather I’m talk­ing about the gen­eral de­ci­sion-the­o­retic prob­lem, where it makes no differ­ence how low of a prob­a­bil­ity you put on some deal pay­ing off, be­cause one can always choose a hu­mon­gous enough pay­off to make “make this deal” be the dom­i­nat­ing op­tion. This is a prob­lem that needs to be solved in or­der to build e.g. an AI sys­tem that uses ex­pected util­ity and will be­have in a rea­son­able man­ner.
In­tu­ition: how Pas­cal’s Mug­ging breaks im­plicit as­sump­tions in ex­pected util­ity theory
In­tu­itively, the prob­lem with Pas­cal’s Mug­ging type ar­gu­ments is that some prob­a­bil­ities are just too low to care about. And we need a way to look at just the prob­a­bil­ity part com­po­nent in the ex­pected util­ity calcu­la­tion and ig­nore the util­ity com­po­nent, since the core of PM is that the util­ity can always be ar­bi­trar­ily in­creased to over­whelm the low prob­a­bil­ity.
Let’s look at the con­cept of ex­pected util­ity a bit. If you have a 10% chance of get­ting a dol­lar each time when you make a deal, and this has an ex­pected value of 0.1, then this is just a differ­ent way of say­ing that if you took the deal ten times, then you would on av­er­age have 1 dol­lar at the end of that deal.
More gen­er­ally, it means that if you had the op­por­tu­nity to make ten differ­ent deals that all had the same ex­pected value, then af­ter mak­ing all of those, you would on av­er­age end up with one dol­lar. This is the jus­tifi­ca­tion for why it makes sense to fol­low ex­pected value even for unique non-re­peat­ing events: be­cause even if that par­tic­u­lar event wouldn’t re­peat, if your gen­eral strat­egy is to ac­cept other bets with the same EV, then you will end up with the same out­come as if you’d taken the same re­peat­ing bet many times. And even though you only get the dol­lar af­ter ten deals on av­er­age, if you re­peat the tri­als suffi­ciently many times, your prob­a­bil­ity of hav­ing the av­er­age pay­out will ap­proach one.
Now con­sider a Pas­cal’s Mug­ging sce­nario. Say some­one offers to cre­ate 10^100 happy lives in ex­change for some­thing, and you as­sign them a 0.000000000000000000001 prob­a­bil­ity to them be­ing ca­pa­ble and will­ing to carry through their promise. Naively, this has an over­whelm­ingly pos­i­tive ex­pected value.
But is it re­ally a benefi­cial trade? Sup­pose that you could make one deal like this per sec­ond, and you ex­pect to live for 60 more years, for about 1,9 billion trades in to­tal. Then, there would be a prob­a­bil­ity of 0,999999999998 that the deal would never once have paid off for you. Which sug­gests that the EU calcu­la­tion’s im­plicit as­sump­tion—that you can re­peat this of­ten enough for the util­ity to con­verge to the ex­pected value—would be vi­o­lated.
Our first attempt
This sug­gests an ini­tial way of defin­ing a “prob­a­bil­ity small enough to be ig­nored”:
1. Define a “prob­a­bil­ity small enough to be ig­nored” (PSET, or by slight re­ar­rang­ing of let­ters, PEST) such that, over your life­time, the ex­pected times that the event hap­pens will be less than one.
2. Ig­nore deals where the prob­a­bil­ity com­po­nent of the EU calcu­la­tion in­volves a PEST.
Look­ing at the first at­tempt in detail
To calcu­late PEST, we need to know how of­ten we might be offered a deal with such a prob­a­bil­ity. E.g. a 10% chance for some­thing might be a PEST if we only lived for a short enough time that we could make a deal with a 10% chance once. So, a more pre­cise defi­ni­tion of a PEST might be that it’s a prob­a­bil­ity such that
(amount of deals that you can make in your life that have this prob­a­bil­ity) * (PEST) < 1
But defin­ing “one” as the min­i­mum times we should ex­pect the event to hap­pen for the prob­a­bil­ity to not be a PEST feels a lit­tle ar­bi­trary. In­tu­itively, it feels like the thresh­old should de­pend on our de­gree of risk aver­sion: maybe if we’re risk averse, we want to re­duce the ex­pected amount of times some­thing hap­pens dur­ing our lives to (say) 0,001 be­fore we’re ready to ig­nore it. But part of our mo­ti­va­tion was that we wanted a way to ig­nore the util­ity part of the calcu­la­tion: bring­ing in our de­gree of risk aver­sion seems like it might in­tro­duce the util­ity again.
What if re­defined risk aver­sion/​neu­tral­ity/​prefer­ence (at least in this con­text) as how low one would be will­ing to let the “ex­pected amount of times this might hap­pen” fall be­fore con­sid­er­ing a prob­a­bil­ity a PEST?
Let’s use this idea to define an Ex­pected Life­time Utility:
ELU(S,L,R) = the ELU of a strat­egy S over a life­time L is the ex­pected util­ity you would get if you could make L deals in your life, and were only will­ing to ac­cept deals with a min­i­mum prob­a­bil­ity P of at least S, tak­ing into ac­count your risk aver­sion R and as­sum­ing that each deal will pay off ap­prox­i­mately P*L times.
ELU example
Sup­pose that we a have a world where we can take three kinds of ac­tions.
- Ac­tion A takes 1 unit of time and has an ex­pected util­ity of 2 and prob­a­bil­ity 13 of pay­ing off on any one oc­ca­sion.
- Ac­tion B takes 3 units of time and has an ex­pected util­ity of 10^(Gra­ham’s num­ber) and prob­a­bil­ity 1/​100000000000000 of pay­ing off one any one oc­ca­sion.
- Ac­tion C takes 5 units of time and has an ex­pected util­ity of 20 and prob­a­bil­ity 1100 of pay­ing off on an one oc­ca­sion.
As­sum­ing that the world’s life­time is fixed at L = 1000 and R = 1:
ELU(“always choose A”): we ex­pect A to pay off on ((1000 /​ 1) * 13) = 333 in­di­vi­d­ual oc­ca­sions, so with R = 1, we deem it ac­cept­able to con­sider the util­ity of A. The ELU of this strat­egy be­comes (1000 /​ 1) * 2 = 2000.
ELU(“always choose B”): we ex­pect B to pay off on ((1000 /​ 3) * 1/​100000000000000) = 0.00000000000333 oc­ca­sions, so with R = 1, we con­sider the ex­pected util­ity of B to be 0. The ELU of this strat­egy thus be­comes ((1000 /​ 3) * 0) = 0.
ELU(“always choose C”): we ex­pect C to pay off on ((1000 /​ 5) * 1100) = 2 in­di­vi­d­ual oc­ca­sions, so with R = 1, we con­sider the ex­pected util­ity of C to be ((1000 /​ 5) * 20) = 4000.
Thus, “always choose C” is the best strat­egy.
Defin­ing R
Is R some­thing to­tally ar­bi­trary, or can we de­ter­mine some more ob­jec­tive crite­ria for it?
Here’s where I’m stuck. Thoughts are wel­come. I do know that while set­ting R = 1 was a con­ve­nient ex­am­ple, it’s most likely too high, be­cause it would sug­gest things like not us­ing seat belts.
Gen­eral thoughts on this approach
An in­ter­est­ing thing about this ap­proach is that the thresh­old for a PEST be­comes de­pen­dent on one’s ex­pected life­time. This is sur­pris­ing at first, but ac­tu­ally makes some in­tu­itive sense. If you’re liv­ing in a dan­ger­ous en­vi­ron­ment where you might be kil­led any­time soon, you won’t be very in­ter­ested in spec­u­la­tive low-prob­a­bil­ity op­tions; rather you want to fo­cus on mak­ing sure you sur­vive now. Whereas if you live in a mod­ern-day Western so­ciety, you may be will­ing to in­vest some amount of effort in weird low-prob­a­bil­ity high-pay­off deals, like cry­on­ics.
On the other hand, whereas in­vest­ing in that low-prob­a­bil­ity, high-util­ity op­tion might not be good for you in­di­vi­d­u­ally, it could still be a good evolu­tion­ary strat­egy for your genes. You your­self might be very likely to die, but some­one else car­ry­ing the risk-tak­ing genes might hit big and be very suc­cess­ful in spread­ing their genes. So it seems like our defi­ni­tion of L, life­time length, should vary based on what we want: are we look­ing to im­ple­ment this strat­egy just in our­selves, our whole species, or some­thing else? Ex­actly what are we max­i­miz­ing over?