St. Petersburg Mugging Implies You Have Bounded Utility

This post de­scribes an in­finite gam­ble that, un­der some rea­son­able as­sump­tions, will mo­ti­vate peo­ple who act to max­i­mize an un­bounded util­ity func­tion to send me all their money. In other words, if you un­der­stand this post and it doesn’t mo­ti­vate you to send me all your money, then you have a bounded util­ity func­tion, or per­haps even upon re­flec­tion you are not choos­ing your ac­tions to max­i­mize ex­pected util­ity, or per­haps you found a flaw in this post.

Briefly, we do this with The St. Peters­burg Para­dox, con­verted to a mug­ging along the lines of Pas­cal’s Mug­ging. I then tweaked it to ex­tract all of the money in­stead of just a fixed sum.

I have always won­dered if any ac­tual pay­ments have re­sulted from Pas­cal’s Mug­ging, so I in­tend to track pay­ments re­ceived for this vari­a­tion. If any­one does have un­bounded util­ity and wants to prove me wrong by send­ing money, send it with Pay­pal to tim at fun­gible dot com. An­no­tate the trans­fer with the phrase “St. Peters­burg Mug­ging”, and I’ll edit this ar­ti­cle pe­ri­od­i­cally to say how much money I re­ceived. In or­der to avoid con­fus­ing the ex­per­i­ment, and to ex­er­cise my spite, I promise I will not spend the money on any­thing you will find es­pe­cially valuable. SIAI would be bet­ter char­ity, if you want to do char­ity, but don’t send that money to me.

Here’s the hy­po­thet­i­cal (that is, false) offer to per­sons with un­bounded util­ity:

  • Let’s call your util­ity func­tion “UTILITY”. We as­sume it takes a state of the uni­verse as an ar­gu­ment.

  • Define DUT to be UTILITY(the pre­sent situ­a­tion plus you re­ceiv­ing $1000)-UTILITY(the pre­sent situ­a­tion). Here DUT stands for Differ­ence in UTility. We as­sume DUT is pos­i­tive.

  • You have un­bounded util­ity, so for each non­nega­tive N there is a uni­verse UN(N) such that UTILITY(UN(N)) is at least DUT * 2**N. Here UN stands for “uni­verse”.

  • The phrase “I am a god” is defined to mean that I am able to change the uni­verse to any state I choose. I may not be a god af­ter I make the change.

  • The offer is: For ev­ery dol­lar you send me, I will flip a coin. If it comes out Tails, or I am not a god, I will do noth­ing. If it comes out Heads and I am a god, I will flip the coin re­peat­edly un­til I see it come up Heads again. Let T be the num­ber of times it was Tails. I will then change the uni­verse to UN(T).

If I am ly­ing and the offer is real, and I am a god, what util­ity will you re­ceive from send­ing me a dol­lar? Well, the prob­a­bil­ity of me see­ing N Tails fol­lowed by a Head is (1/​2)**(N + 1), and your util­ity for the re­sult­ing uni­verse is UTILITY(UN(N)) >= DUT * 2**N, so your ex­pected util­ity if I see N tails is (1/​2)**(N + 1) * UTILITY(UN(N)) >= (1/​2)**(N + 1) * DUT * 2 ** N = DUT/​2. There are in­finitely many pos­si­ble val­ues for N, so your to­tal ex­pected util­ity is pos­i­tive in­finity * DUT/​2, which is pos­i­tive in­finity.

I hope we agree that it is un­likely that I am a god, but it’s con­sis­tent with what you have ob­served so far, so un­less you were born with cer­tain knowl­edge that I am not a god, you have to as­sign pos­i­tive prob­a­bil­ity to it. Similarly, the prob­a­bil­ity that I’m ly­ing and the above offer is real is also pos­i­tive. The product of two pos­i­tive num­bers is pos­i­tive. Com­bin­ing this with the re­sult from the pre­vi­ous para­graph, your ex­pected util­ity from send­ing me a dol­lar is in­finitely pos­i­tive.

If you send me one dol­lar, there will prob­a­bly be no re­sult. Per­haps I am a god, and the above offer is real, but I didn’t do any­thing be­yond flip­ping the first coin be­cause it came out Tails. In that case, noth­ing hap­pens. Your ex­pected util­ity for the next dol­lar is also in­finitely pos­i­tive, so you should send the next dol­lar too. By in­duc­tion you should send me all your dol­lars.

If you don’t send money be­cause you have bounded util­ity, that’s my de­sired out­come. If you do feel mo­ti­vated to send me money, well, I sup­pose I lost the ar­gu­ment. Re­mem­ber to send all of it, and re­mem­ber that you can always send me more later.

As of 7 June 2011, no­body has sent me any money for this.

ETA: Some in­ter­est­ing is­sues keep com­ing up. I’ll put them here to de­crease the re­dun­dancy:

  • Yes, you can jus­tify not giv­ing me money be­cause I might be a god by claiming that there are lots of other un­likely gods that have a bet­ter claim on your re­sources. My pur­pose in writ­ing this post is to find a good rea­son not to be jerked around by un­likely gods in gen­eral. Find­ing a rea­son to be jerked around by some other un­likely god is miss­ing the point.

  • I for­got to men­tion that if I am a god, I can stop time while I flip coins, so we aren’t re­source-con­strained on the num­ber of times I can flip the coin.

  • Yes, you can say that your prior prob­a­bil­ity of me be­ing a god is zero. If you want to go that way, can you say what that prior prob­a­bil­ity dis­tri­bu­tion looks like in gen­eral? I’m ac­tu­ally more wor­ried about mak­ing a Friendly AI that gets jerked around by an un­likely god that we did not plan for, so hav­ing a spe­cial case about me be­ing god doesn’t solve an in­ter­est­ing por­tion of the prob­lem. For what it’s worth, I be­lieve the Univer­sal Prior would give pos­i­tive small prob­a­bil­ity to many sce­nar­ios that have a god, since uni­verses with a god are not in­cred­ibly much more com­plex than uni­verses that don’t have a god.