# Expected utility without the independence axiom

John von Neu­mann and Oskar Mor­gen­stern de­vel­oped a sys­tem of four ax­ioms that they claimed any ra­tio­nal de­ci­sion maker must fol­low. The ma­jor con­se­quence of these ax­ioms is that when faced with a de­ci­sion, you should always act solely to in­crease your ex­pected util­ity. All four ax­ioms have been at­tacked at var­i­ous times and from var­i­ous di­rec­tions; but three of them are very solid. The fourth—in­de­pen­dence—is the most con­tro­ver­sial.

To un­der­stand the ax­ioms, let A, B and C be lot­ter­ies—pro­cesses that re­sult in differ­ent out­comes, pos­i­tive or nega­tive, with a cer­tain prob­a­bil­ity of each. For 0<p<1, the mixed lot­tery pA + (1-p)B im­plies that you have p chances of be­ing in lot­tery A, and (1-p) chances of be­ing in lot­tery B. Then writ­ing A>B means that you pre­fer lot­tery A to lot­tery B, A<B is the re­verse and A=B means that you are in­differ­ent be­tween the two. Then the von Neu­mann-Mor­gen­stern ax­ioms are:

• (Com­plete­ness) For ev­ery A and B ei­ther A<B, A>B or A=B.

• (Tran­si­tivity) For ev­ery A, B and C with A>B and B>C, then A>C.

• (Con­ti­nu­ity) For ev­ery A>B>C then there ex­ist a prob­a­bil­ity p with B=pA + (1-p)C.

• (In­de­pen­dence) For ev­ery A, B and C with A>B, and for ev­ery 0<t≤1, then tA + (1-t)C > tB + (1-t)C.

In this post, I’ll try and prove that even with­out the In­de­pen­dence ax­iom, you should con­tinue to use ex­pected util­ity in most situ­a­tions. This re­quires some mild ex­tra con­di­tions, of course. The prob­lem is that al­though these con­di­tions are con­sid­er­ably weaker than In­de­pen­dence, they are harder to phrase. So please bear with me here.

The whole in­sight in this post rests on the fact that a lot­tery that has 99.999% chance of giv­ing you £1 is very close to be­ing a lot­tery that gives you £1 with cer­tainty. I want to ex­press this fact by look­ing at the nar­row­ness of the prob­a­bil­ity dis­tri­bu­tion, us­ing the stan­dard de­vi­a­tion. How­ever, this nar­row­ness is not an in­trin­sic prop­erty of the dis­tri­bu­tion, but of our util­ity func­tion. Even in the ex­am­ple above, if I de­cide that re­ceiv­ing £1 gives me a util­ity of one, while re­ceiv­ing zero gives me a util­ity of minus ten billion, then I no longer have a nar­row dis­tri­bu­tion, but a wide one. So, un­like the tra­di­tional set-up, we have to as­sume a util­ity func­tion as be­ing given. Once this is cho­sen, this al­lows us to talk about the mean and stan­dard de­vi­a­tion of a lot­tery.

Then if you define c(μ) as the lot­tery giv­ing you a cer­tain re­turn of μ, you can use the fol­low­ing ax­iom in­stead of in­de­pen­dence:

• (Stan­dard de­vi­a­tion bound) For all ε>0, there ex­ists a δ>0 such that for all μ>0, then any lot­tery B with mean μ and stan­dard de­vi­a­tion less that μδ has B>c((1-ε)μ).

This seems com­pli­cated, but all that it says, in math­e­mat­i­cal terms, is that if we have a prob­a­bil­ity dis­tri­bu­tion that is “nar­row enough” around its mean μ, then we should value it are be­ing very close to a cer­tain re­turn of μ. The nar­row­ness is ex­pressed in terms of its stan­dard de­vi­a­tion—a lot­tery with zero SD is a guaran­teed re­turn of μ, and as the SD gets larger, the dis­tri­bu­tion gets wider, and the chances of get­ting val­ues far away from μ in­creases. So risk, in other words, scales (ap­prox­i­mately) with the SD.

We also need to make sure that we are not risk lov­ing—if we are in­vet­er­ate gam­blers for the point of be­ing gam­blers, our be­havi­our may be a lot more com­pli­cated.

• (Not risk lov­ing) If A has mean μ>0, then A≤c(μ).

I.e. we don’t love a worse rate of re­turn just be­cause of the risk. This ax­iom can and maybe should be weak­ened, but it’s a good ap­prox­i­ma­tion for the mo­ment—most peo­ple are not risk lov­ing with huge risks.

As­sume you are go­ing to be have to choose n differ­ent times whether to ac­cept in­de­pen­dent lot­ter­ies with fixed mean β>0, and all with SD less than a fixed up­per-bound K. Then if you are not risk lov­ing and n is large enough, you must ac­cept an ar­bi­trar­ily large pro­por­tion of the lot­ter­ies.

Proof: From now on, I’ll use a differ­ent con­ven­tion for adding and scal­ing lot­ter­ies. Treat­ing them as ran­dom vari­ables, A+B will mean the lot­tery con­sist­ing of A and B to­gether, while xA will mean the same lot­tery as A, but with all re­turns (pos­i­tive or nega­tive) scaled by x.

Let X1, X2, … , Xn be these n in­de­pen­dent lot­ter­ies, with means β and var­i­ances vj. The since the stan­dard de­vi­a­tions are less than K, the var­i­ances must be less than K2.

Let Y = X1 + X2 + … + Xn. The mean of Y is nβ. The var­i­ance of Y is the sum of the vj, which is less than nK2. Hence the SD of Y is less than K√(n). Now pick an ε>0, and the re­sult­ing δ>0 from the stan­dard de­vi­a­tion bound ax­iom. For large enough n, nβδ must be larger than K√(n); hence, for large enough n, Y > c((1-ε)nβ). Now, if we were to re­fuse more that εn of the lot­ter­ies, we would be left with a dis­tri­bu­tion with mean ≤ (1-ε)nβ, which, since we are not risk lov­ing, is worse than c((1-ε)nβ), which is worse than Y. Hence we must ac­cept more than a pro­por­tion (1-ε) of the lot­ter­ies on offer.

This only ap­plies to lot­ter­ies that share the same mean, but we can gen­er­al­ise the re­sult as:

As­sume you are go­ing to be have to choose n differ­ent times whether to ac­cept in­de­pen­dent lot­ter­ies all with means greater than a fixed β>0, and all with SD less than a fixed up­per-bound K. Then if you are not risk lov­ing and n is large enough, you must ac­cept lot­ter­ies whose means rep­re­sent an ar­bi­trar­ily large pro­por­tion of the to­tal mean of all lot­ter­ies on offer.

Proof: The same proof works as be­fore, with nβ now be­ing a lower bound on the true mean μ of Y. Thus we get Y > c((1-ε)μ), and we must ac­cept lot­ter­ies whose to­tal mean is greater than (1-ε)μ.

Anal­y­sis: Since we re­jected in­de­pen­dence, we must now con­sider the lot­ter­ies when taken as a whole, rather than just see­ing them in­di­vi­d­u­ally. When con­sid­ered as a whole, “rea­son­able” lot­ter­ies are more tightly bunched around their to­tal mean than they are in­di­vi­d­u­ally. Hence the more lot­ter­ies we con­sider, the more we should treat them as if only their mean mat­tered. So if we are not risk lov­ing, and ex­pect to meet many lot­ter­ies with bounded SD in our lives, we should fol­low ex­pected util­ity. Deprived of in­de­pen­dence, ex­pected util­ity sneaks in via ag­gre­ga­tion.

Note: This restates the first half of my pre­vi­ous post—a post so con­fus­ingly writ­ten it should be staked through the heart and left to die on a cross­road at noon.

Edit: Rewrote a part to em­pha­sis the fact that a util­ity func­tion needs to be cho­sen in ad­vance—thanks to Peter de Blanc and Nick Hay for bring­ing this up.

• Folks, please write at least short re­views on tech­ni­cal ar­ti­cles: if some­one parsed the math, whether it ap­pears sen­si­ble, whether the mes­sage ap­pears in­ter­est­ing, and what ex­actly this mes­sage con­sists in. Also, this ar­ti­cle lacks refer­ences: is the stuff it de­scribes stan­dard, how does it re­late to the field?

• The re­sult is my own work, but the rea­son­ing is not par­tic­u­larly com­plex, and might well have been done be­fore.

It’s kind of a poor man’s ver­sion of the cen­tral limit the­o­rem, for differ­ing dis­tri­bu­tions.

By this I mean that it’s known that if you take the mean of iden­ti­cal in­de­pen­dent dis­tri­bu­tions, it will tend to a nar­row spike as the num­ber of dis­tri­bu­tions in­crease. This post shows that similar things hap­pen with non-iden­ti­cal dis­tri­bu­tions, if we bound the var­i­ances.

And please do point out any er­rors that any­one finds!

• The math looks valid—I be­lieve the con­tent is origi­nal to Stu­art_Arm­strong, at­tempt­ing to show a novel set of prefer­ences which im­ply ex­pected-value calcu­la­tion in (sufi­ciently) iter­ated cases but not in iso­lated cases.

Edit: For ex­am­ple, an agent whose de­ci­sion-mak­ing crite­ria satisfy Stu­art_Arm­strong’s crite­ria might re­fuse to bet $1 for a 50% chance of win­ning$2.50 and 50% chance of los­ing his ini­tial dol­lar if it were a one-off gam­ble, but would be will­ing to make 50 such bets in a row if the odds of win­ning each were in­de­pen­dent. In both cases, the ex­pected value is pos­i­tive, but only in the lat­ter case is the prob­a­ble vari­a­tion from the ex­pected value small enough to over­come the risk aver­sion.

• This ar­ti­cle had an in­ter­est­ing ti­tle so I scanned it—but it lacked an ab­stract, a con­clu­sion, had lots of maths in it—and I haven’t liked most of Stu­art’s other ar­ti­cles—so I gave up on it early.

• The ar­ti­cle at­tempts to show that you don’t need the in­de­pen­dence ax­iom to jus­tify us­ing ex­pected util­ity. So I re­placed the in­de­pen­dence ax­iom with an­other ax­iom that ba­si­cally says that very thin dis­tri­bu­tion is pretty much the same as a guaran­teed re­turn.

Then I showed that if you had a lot of “rea­son­able” lot­ter­ies and put them to­gether, you should be­have ap­prox­i­mately ac­cord­ing to ex­pected util­ity.

There’s a lot of maths in it be­cause the re­sult is novel, and there­fore has to be firmly jus­tified. I hope to ex­plore non-in­de­pen­dent lot­ter­ies in fu­ture posts, so the foun­da­tions need to be solid.

• Since we re­jected in­de­pen­dence, we must now con­sider the lot­ter­ies when taken as a whole, rather than just see­ing them in­di­vi­d­u­ally. When con­sid­ered as a whole, “rea­son­able” lot­ter­ies are more tightly bunched around their to­tal mean than they are in­di­vi­d­u­ally. Hence the more lot­ter­ies we con­sider, the more we should treat them as if only their mean mat­tered.

You are ab­solutely cor­rect, and it pains me be­cause this is­sue should have been set­tled a long time ago.

When Eliezer Yud­kowsky first brought up the break­down of in­de­pen­dence in hu­mans, way, way back dur­ing the dis­cus­sion of the Allais Para­dox, the poster “Gray Area” ex­plained why peo­ple aren’t be­ing money-pumped, even though they vi­o­late in­de­pen­dence. He/​she came to the same con­clu­sion in the quote above.

Here’s what Gray Area said back then:

Fi­nally, the ‘money pump’ ar­gu­ment fails be­cause you are chang­ing the rules of the game. The origi­nal ques­tion was, I as­sume, ask­ing whether you would play the game once, whereas you would pre­sum­ably iter­ate the money pump un­til the pen­nies turn into mil­lions. The prob­lem, though, is if you asked peo­ple to make the origi­nal choices a mil­lion times, they would, cor­rectly, max­i­mize ex­pec­ta­tions. Be­cause when you are talk­ing about a mil­lion tries, ex­pec­ta­tions are the ap­pro­pri­ate frame­work. When you are talk­ing about 1 try, they are not. [bold added]

I didn’t see any­one even re­ply to Gray Area any­where in that se­ries, or any­time since.

So I bring up es­sen­tially the same point when­ever Eliezer uses the Allais re­sult, always con­clud­ing with a zinger like: If get­ting lot­tery tick­ets is be­ing ex­ploited, I don’t want to be em­pow­ered.

Please, folks, stop equat­ing a hy­po­thet­i­cal money pump with the ac­tual sce­nario.

• The Allais Para­dox is not about risk aver­sion or lack thereof; it’s about peo­ple’s de­ci­sions be­ing in­con­sis­tent. There are definitely situ­a­tions in which you would want to choose a 50% chance of $1M over a 10% chance of$10M. How­ever, if you would do so, you should also then choose a 5% chance of $1M over a 1% chance of$10M, be­cause the rel­a­tive risk is the same. See Eliezer’s fol­lowup post, Zut Allais.

Turn­ing a per­son into a money pump also isn’t about play­ing the same gam­ble a zillion times (as any good in­vestor will tell you, if you play the gam­ble a zillion times, all the risk dis­ap­pears and you’re left with only ex­pected re­turn, which leaves you with a differ­ent prob­lem). The money pump works thusly: I sell you gam­ble A for $5. You then trade with me gam­ble A for gam­ble B. You then sell me back gam­ble B for$4. I then sell you gam­ble A for $5… wash, rinse, re­peat. Nowhere in the cy­cle is ei­ther gam­ble ac­tu­ally paid out. • Are you sure you’re re­spond­ing to the right per­son here? 1) I wasn’t claiming that Allais is about risk aver­sion. 2) I was claiming it doesn’t show an in­con­sis­tency (and IMO suc­ceeded). 3) I did read Zut Allais, and the other Allais ar­ti­cle with the other ridicu­lous French pun, and it wasn’t re­spon­sive to the point that Gray Area raised. (You may note that a strap­ping lad named “Silas” even noted this at the time.) How­ever, if you would do so, you should also then choose 4) You can­not sub­stan­ti­ate the charge that you should do the lat­ter if you did the former, since no nega­tive con­se­quence ac­tu­ally re­sults from vi­o­lat­ing that “should” in the one-shot case. You know, the one peo­ple were ac­tu­ally tested on. ETA: (I think the sec­ond para­graph was just added in tomm­c­cabe’s post.) Turn­ing a per­son into a money pump also isn’t about play­ing the same gam­ble a zillion times. My point never hinged on it be­ing oth­er­wise. The money pump works thusly: I sell you gam­ble A for$5. You then trade with me gam­ble A for gam­ble B. You then sell me back gam­ble B for $4. I then sell you gam­ble A for$5… wash, rinse, re­peat. Nowhere in the cy­cle is ei­ther gam­ble ac­tu­ally paid out.

Okay, and where in the Allais ex­per­i­ment did it per­mit any of those ex­changes to hap­pen? Right, nowhere.

Believe it or not, when I say, “I pre­fer B to A”, it doesn’t mean “I hereby legally obli­gate my­self to re­deem on de­mand any B for an A”, yet your money pump re­quires that.

• The prob­lem is that you’re los­ing money do­ing it once. You would agree that c(0) > c(-2), yes? If they are will­ing to trade A for B in a one-shot game, they shouldn’t be will­ing to pay more for A than for B in a one-shot—you don’t trade the more valuable item for the less valuable. That their prefer­ences may re­verse in the iter­ated situ­a­tion has no bear­ing on the Allais prob­lem.

Edit: The text above fol­low­ing the ques­tion mark is in­cor­rect. See my later com­ment quot­ing Eliezer for the cor­rect state­ment.

• The prob­lem is that you’re los­ing money do­ing it once.

Again, if sud­denly be­ing offered the choice of 1A/​1B then 2A/​2B as de­scribed here, but be­ing “in­con­sis­tent”, is what you call “los­ing money”, then I don’t want to gain money!

If they are will­ing to trade A for B in a one-shot game, they shouldn’t be will­ing to pay more for A than for B in a one-shot

But that’s not what’s hap­pen­ing the para­dox. They’re (do­ing some­thing iso­mor­phic to) prefer­ring A to B once and then p*B to p*A once. At no point do they “pay” more for B than A while prefer­ring A to B. At no point does any­one make or offer the money-pump­ing trades with the sub­jects, nor have they obli­gated them­selves to do so!

• Con­sider Eliezer’s fi­nal re­marks in The Allais Para­dox (I link purely for the con­ve­nience of those com­ing in in the mid­dle):

Sup­pose that at 12:00PM I roll a hun­dred-sided die. If the die shows a num­ber greater than 34, the game ter­mi­nates. Other­wise, at 12:05PM I con­sult a switch with two set­tings, A and B. If the set­ting is A, I pay you $24,000. If the set­ting is B, I roll a 34-sided die and pay you$27,000 un­less the die shows “34”, in which case I pay you noth­ing.

Let’s say you pre­fer 1A over 1B, and 2B over 2A, and you would pay a sin­gle penny to in­dulge each prefer­ence. The switch starts in state A. Be­fore 12:00PM, you pay me a penny to throw the switch to B. The die comes up 12. After 12:00PM and be­fore 12:05PM, you pay me a penny to throw the switch to A.

I have taken your two cents on the sub­ject.

You’re right in­so­far as Eliezer in­vokes the Ax­iom of In­de­pen­dence when he re­solves the Allais Para­dox us­ing ex­pected value; I do not yet see any way in which Stu­art_Arm­strong’s crite­ria rule out the prefer­ences (1A > 1B)u(2A < 2B). How­ever, in the sce­nario Eliezer de­scribes, an agent with those prefer­ences ei­ther loses one cent or two cents rel­a­tive to the agent with (1A > 1B)u(2A > 2B).

• Your prefer­ences be­tween A and B might rea­son­ably change if you ac­tu­ally re­ceive the money from ei­ther gam­ble, so that you have more money in your bank ac­count now than you did be­fore. How­ever, that’s not what’s hap­pen­ing; the ex­per­i­menter can use you as a money pump with­out ever ac­tu­ally pay­ing out on ei­ther gam­ble.

• Yes, I know that a money pump doesn’t in­volve do­ing the gam­ble it­self. You don’t have to re­peat your­self, but ap­par­ently, I do have to re­peat my­self when I say:

The money pump does re­quire that the ex­per­i­menter make ac­tual futher trades with you, not just imag­ine hy­po­thet­i­cal ones. The sub­jects didn’t make these trades, and if they saw many more lot­tery tick­ets po­ten­tially com­ing into play, so as to smooth out re­turns, they would quickly re­vert to stan­dard EU max­i­miza­tion, as pre­dicted by Arm­strongs’s deriva­tion.

• “Po­ten­tially com­ing into play, so as to smooth out re­turns” re­quires that there be the pos­si­bil­ity of the sub­ject ac­tu­ally tak­ing more than one gam­ble, which never hap­pens. If you mean that peo­ple might get sus­pi­cious af­ter the tenth time the ex­per­i­menter takes their money and gives them noth­ing in re­turn, and there­after stop do­ing it, I agree with you; how­ever, all this proves is that mak­ing the origi­nal trade was stupid, and that peo­ple are able to learn to not make stupid de­ci­sions given suffi­cient rep­e­ti­tion.

• “Po­ten­tially com­ing into play, so as to smooth out re­turns” re­quires that there be the pos­si­bil­ity of the sub­ject ac­tu­ally tak­ing more than one gam­ble, which never hap­pens.

The pos­si­bil­ity has to hap­pen, if you’re cy­cling all these tick­ets through the sub­ject’s hands. What, are they fake tick­ets that can’t ac­tu­ally be used now?

There are fac­tors that come into play when you get to do lots of runs, but aren’t pre­sent with only one run. A sub­ject’s choice in a one-shot sce­nario does not im­ply that they’ll make the money-los­ing trades you de­scribe. They might, but you would have to ac­tu­ally test it out. They don’t be­come ir­ra­tional un­til such a thing ac­tu­ally hap­pens.

• “What, are they fake tick­ets that can’t ac­tu­ally be used now?”

No, they’re just the same tick­ets. There’s only ever one of each. If I sell you a choco­late bar, trade the choco­late bar for a bag of Sk­it­tles, buy the bag of Sk­it­tles, and re­peat ten thou­sand times, this does not mean I have ten thou­sand of each; I’m just re-us­ing the same ones.

“They might, but you would have to ac­tu­ally test it out. They don’t be­come ir­ra­tional un­til such a thing ac­tu­ally hap­pens.”

We did test it out, and yes, peo­ple did act as money pumps. See The Con­struc­tion of Prefer­ence by Sarah Licht­en­stein and Paul Slovic.

• You can also listen to an in­ter­view with one of Sarah Licht­en­stein’s sub­jects who re­fused to make his prefer­ences con­sis­tent even af­ter the money-pump as­pect was ex­plained:

http://​​www.de­ci­sion­re­search.org/​​pub­li­ca­tions/​​books/​​con­struc­tion-prefer­ence/​​listen.html

• You can also listen to an in­ter­view with one of Sarah Licht­en­stein’s sub­jects who re­fused to make his prefer­ences con­sis­tent even af­ter the money-pump as­pect was ex­plained:

http://​​www.de­ci­sion­re­search.org/​​pub­li­ca­tions/​​books/​​con­struc­tion-prefer­ence/​​listen.html

That is an in­cred­ible in­ter­view.

Ad­mit­ting that the set of prefer­ences is in­con­sis­tent, but re­fus­ing to fix it is not so bad a con­clu­sion—maybe he’d just make it worse (eg, by rais­ing the bid on B to 550). At times he seems to ad­mit that the over­all pat­tern is ir­ra­tional (“It shows my rea­son­ing pro­cess isn’t too good”). At other times, he doesn’t ad­mit the prob­lem, but I think you’re too harsh on him in fram­ing it as re­fusal.

I may be mi­s­un­der­stand­ing, but he seems to say that the game doesn’t al­low him to bid higher than 400 on B. If he val­ues B higher than 400 (yes, an ab­surd mis­take), but sells it for 401, merely be­cause he wasn’t al­lowed to value it higher, then that seems to me to be the biggest mis­take. It fits the book’s ti­tle, though.

Maybe he just means that his sense of math is that the cap should be 400, which would be the lone ex­am­ple of math helping him. He seems torn be­tween au­thor­ity figures, the “ra­tio­nal­ity” of non-cir­cu­lar prefer­ences and the un­named math of ex­pected val­ues. I’m some­what sur­prised that he doesn’t see them as the same or­a­cle. Maybe he was scarred by child­hood math teach­ers, and a lone psy­chol­o­gist can’t match that in­timi­da­tion?

• That sounds to me as though he is us­ing ex­pected util­ity to come up with his num­bers, but doesn’t un­der­stand ex­pected util­ity, so when asked which he prefers he uses some other emo­tional sys­tem.

• “1) I wasn’t claiming that Allais is about risk aver­sion.”

The differ­ence be­tween your prefer­ences over choos­ing lot­tery A vs. lot­tery B when both are performed a mil­lion times, and your prefer­ences over choos­ing A vs. B when both are performed once, is a mea­sure­ment of your risk aver­sion; this is what Gray Area was talk­ing about, is it not?

“Believe it or not, when I say, “I pre­fer B to A”, it doesn’t mean “I hereby legally obli­gate my­self to re­deem on de­mand any B for an A”″

Then you must be us­ing a differ­ent (and, I might add, quite un­usual) defi­ni­tion of the word “prefer­ence”. To quote dic­tio­nary.com:

pre⋅fer /​prɪˈfɜr/​ [pri-fur] –verb (used with ob­ject), -ferred, -fer⋅ring.

1. to set or hold be­fore or above other per­sons or things in es­ti­ma­tion; like bet­ter; choose rather than: to pre­fer beef to chicken.

What does it mean to say that you pre­fer B to A, if you wouldn’t trade B for A if the trade is offered? Could I say that I pre­fer tor­ture to candy, even if I always choose candy when the choice is offered to me?

Typo: Did you mean “pre­fer A to B”?

• I pre­fer B to A does not im­ply I pre­fer 10B to 10A, or even I pre­fer 2B to 2A. Ex­pected util­ity != ex­pected re­turn.

I agree pretty much com­pletely with Silas. If you want to prove that peo­ple are money pumps, you need to ac­tu­ally get a ran­dom sam­ple of peo­ple and then ac­tu­ally pump money out of them. You can’t just take a sin­gle-shot hy­po­thet­i­cal and ex­trap­o­late to other hy­po­thet­i­cals when the whole is­sue is how peo­ple deal with the vari­abil­ity of re­turns.

• Strictly speak­ing, Eliezer’s for­mu­la­tion of the Allais Para­dox is not the one that has been ex­per­i­men­tally tested. I be­lieve a similar money pump can be im­ple­mented for the canon­i­cal ver­sion, how­ever—and Zut Allais! shows that peo­ple can be turned into money pumps in other situ­a­tions.

• “I pre­fer B to A does not im­ply I pre­fer 10B to 10A, or even I pre­fer 2B to 2A. Ex­pected util­ity != ex­pected re­turn.”

Of course, but, as I’ve said (I think?) five times now, you never ac­tu­ally get 2B or 2A at any point dur­ing the money-pump­ing pro­cess. You go from A, to B, to noth­ing, to A, to B… etc.

For ex­am­ples of Ve­gas gam­blers ac­tu­ally hav­ing money pumped out of them, see The Con­struc­tion of Prefer­ence by Sarah Licht­en­stein and Paul Slovic.

• The differ­ence be­tween your prefer­ences over choos­ing lot­tery A vs. lot­tery B when both are performed a mil­lion times, and your prefer­ences over choos­ing A vs. B when both are performed once, is a mea­sure­ment of your risk aver­sion; this is what Gray Area was talk­ing about, is it not?

No, it’s not, and the prob­lem as­serted by Allais para­dox is that the util­ity func­tion is in­con­sis­tent, no mat­ter what the risk prefer­ence.

Then you must be us­ing a differ­ent (and, I might add, quite un­usual) defi­ni­tion of the word “prefer­ence”. To quote dic­tio­nary.com:

1. to set or hold be­fore or above other per­sons or things in es­ti­ma­tion; like bet­ter; choose rather than: to pre­fer beef to chicken.

I don’t see any­thing in there that about how many times the choice has to hap­pen, which is the very is­sue at stake.

If there’s any un­usu­al­ness, it’s definitely on your side. When you buy a choco­late bar for a dol­lar, that “prefer­ence of a choco­late bar to a dol­lar” does not some­how mean that you are will­ing to trade ev­ery dol­lar you have for a choco­late bar, nor have you legally obli­gated your­self to re­deem choco­late bars for dol­lars on de­mand (as a money pump would re­quire), nor does any­one ex­pect that you will trade the rest of your dol­lars this way.

It’s called diminish­ing marginal util­ity. In fact, it’s called marginal anal­y­sis in gen­eral.

What does it mean to say that you pre­fer B to A, if you wouldn’t trade B for A if the trade is offered?

It means you would trade B for A on the next op­por­tu­nity to do so, not that you would in­definitely do it for­ever, as the money pump re­quires.

• “When you buy a choco­late bar for a dol­lar, that “prefer­ence of a choco­late bar to a dol­lar” does not some­how mean that you are will­ing to trade ev­ery dol­lar you have for a choco­late bar, nor have you legally obli­gated your­self to re­deem choco­late bars for dol­lars on de­mand (as a money pump would re­quire), nor does any­one ex­pect that you will trade the rest of your dol­lars this way.”

Un­der nor­mal cir­cum­stances, this is true, be­cause the situ­a­tion has changed af­ter I bought the choco­late bar: I now have an ad­di­tional choco­late bar, or (more likely) an ad­di­tional bar’s worth of choco­late in my stom­ach. My prefer­ences change, be­cause the situ­a­tion has changed.

How­ever, af­ter you have bought A, and swapped A for B, and sold B, you have not gained any­thing (such as a choco­late bar, or a full stom­ach), and you have not lost any­thing (such as a dol­lar); you are in pre­cisely the same po­si­tion that you were be­fore. Hence, con­sis­tency dic­tates that you should make the same de­ci­sion as you did be­fore. If, af­ter buy­ing the choco­late bar, it fell down a well, and an­other dol­lar was added to my bank ac­count be­cause of the choco­late bar in­surance I bought, then yes, I should keep buy­ing choco­late bars for­ever if I want to be con­sis­tent (as­sum­ing that there is no cost to my time, which there es­sen­tially isn’t in this case).

• And some­thing about your state has like­wise change af­ter the swaps you de­scribed, just like when I have bought the first choco­late bar.

Jeez, where’s Ali­corn when you need her? We need some­one to make a point about how, “Just be­cause a woman sleeps with you once, doesn’t meen she’s in­con­sis­tent by …” and then show the map­ping to the logic be­ing used here.

ETA: For­get the po­si­tion I im­puted to Ali­corn for the mo­ment. I’m mak­ing the point: how is this bizarre ex­trap­o­la­tion of prefer­ences any differ­ent from a very un­for­tu­nate overex­trap­o­la­tion of­ten used by men?

• Jeez, where’s Ali­corn when you need her? We need some­one to make a point about how, “Just be­cause a woman sleeps with you once, doesn’t meen she’s in­con­sis­tent by …” and then show the map­ping to the logic be­ing used here.

What, ex­actly, are you try­ing to ac­com­plish here? Your last in­ter­ac­tion with Ali­corn made it pretty clear that pro­ject­ing non-se­quitur sex­ual refer­ences onto her was un­wel­come. Are you trol­ling?

• The last in­ter­ac­tion wasn’t a “sex­ual refer­ence”, even by Ali­corns defi­ni­tion. I was try­ing to point out that her phras­ing was a refer­ence to Lau­raABJ’s im­plied be­liefs about when a woman is re­ject­ing a man not nec­es­sar­ily in a sex­ual con­text.

I’d be in­ter­ested to know why the fol­low-up kept get­ting mod­ded down. As far as I can tell, peo­ple just didn’t un­der­stand.

And I don’t know how this is non-se­quitur or pro­ject­ing sex­ual refer­ences. Peo­ple here are draw­ing ab­surd in­fer­ences about some­one’s prefer­ences from one-time choices. It looks to me like the same kind of ques­tion­able rea­son­ing used in the con­text I men­tioned, and the same kind of thing Ali­corn en­joys re­fut­ing.

Sorry for hav­ing an in­suffi­ciently re­fined red-flag de­tec­tor, and for what­ever offense I may have caused. Just make sure your offense is be­cause of the topic, not be­cause you just re­al­ized what your overex­trap­o­la­tion looks like in other con­texts.

• Just to raise the most ob­vi­ous pos­si­ble ob­jec­tion to your phras­ing: there was noth­ing to pre­vent you from mak­ing what­ever metaphor you sug­gested Ali­corn could have em­ployed. It is gen­er­ally poor man­ners to in­voke un­in­volved peo­ple as sup­port­ers of your ar­gu­ments with­out their per­mis­sion, and in this situ­a­tion, if Ali­corn were in­ter­ested in be­com­ing in­volved in this thread, she could have posted her­self.

• Thanks, that make much more sense.

• The sex­ual refer­ences in par­tic­u­lar are a sub­set of a broad class of things from SilasBarta that I do not wel­come. That class of things is “any­thing in­volv­ing me and SilasBarta di­rectly in­ter­act­ing ever again”. Just so no one in­ter­prets that last in­ter­ac­tion too finely.

• It would prob­a­bly be best to make your point in your own voice and not to put words in Ali­corn’s mouth (how­ever in­di­rectly), since you know that she will not in­ter­act di­rectly with you to cor­rect any mis­ap­pre­hen­sions about her views you may have.

ETA: Whoops, I see RobinZ got there first.

• Your point about Ali­corn not be­ing likely to cor­rect Silas is no less apt than mine about not drag­ging neu­tral par­ties into an ar­gu­ment—in fact, it is scarcely less gen­eral.

• And some­thing about your state has like­wise change af­ter the swaps you de­scribed, just like when I have bought the first choco­late bar.

Yes, but hav­ing made the swaps seems highly ques­tion­able as a a di­men­sion of your state that af­fects your prefer­ences.

• It’s highly-ques­tion­able as a rele­vant state di­men­sion be­cause … you need it to be to make the re­sults come out right?

• the poster “Gray Area” ex­plained why peo­ple aren’t be­ing money-pumped, even though they vi­o­late in­de­pen­dence.

I ac­tu­ally think that (for some ex­am­ples) it’s ac­tu­ally sim­pler than that. The Allais para­dox as­sumes that the pro­posal of the bet it­self has no effect on the util­ity of the pro­posee. In re­al­ity, if I took a 5% chance at $100M, in­stead of a 100% chance at$4M, there’s a 95% chance I’d be kick­ing my­self ev­ery time I opened my wallet for the rest of my life. Thus, tak­ing the bet and los­ing is sig­nifi­cantly worse than never hav­ing the bet pro­posed at all. If this is fac­tored in cor­rectly, EY’s origi­nal for­mu­la­tion of the Allais Para­dox is no longer func­tional: I pre­fer cer­tainty, be­cause los­ing when cer­tainty was an op­tion car­ries lower util­ity than never hav­ing bet.

This is more about how you calcu­late out­comes than it is about in­de­pen­dence di­rectly. If los­ing when you could have had a guaran­teed (or nearly-guaran­teed) win car­ries nega­tive util­ity, and if you can only play once, it does not seem like it con­tra­dicts in­de­pen­dence.

• Glad this for­mu­la­tion is use­ful! I do in­deed think that peo­ple of­ten be­have like you de­scribe, with­out gen­er­ally los­ing huge sums of cash.

How­ever, the con­clu­sion of my post is that it is ira­tional to de­vi­ate from ex­pected util­ity for small sums. Agre­gat­ing ev­ery small de­ci­sion you make will give you ex­pected util­ity.

• I think the post is say­ing “if your prefer­ences are some­what cou­pled to the prefer­ences of an ex­pec­ta­tion max­i­mizer, then in some limit, your prefer­ences match that ex­pec­ta­tion max­i­mizer.”

But so what? Why should your prefer­ences have any re­la­tion to a real-val­ued func­tion of the world? If you satisfy all the ax­ioms, your prefer­ences are ex­actly ex­pec­ta­tion-max­i­miz­ing for a func­tion that vN and M tell you how to build. But if the whole point is to drop one of the ax­ioms, why should you still ex­pect such a func­tion to be rele­vant?

(this has been said el­se­where on the thread, but not too ten­ta­tively, and not at the top level.)

• The re­sults are on the “ex­pected” part of ex­pected util­ity, not on the “util­ity” part. In­de­pen­dence is over­strong; re­plac­ing it with the some­what cou­pling to an ex­pec­ta­tion max­i­mizer is much weaker. And yet in the limit it mimics the ex­pec­ta­tion re­quire­ment, which is very use­ful re­sult.

(drop­ping in­de­pen­dence com­pletely leaves you flailing all over the place)

• Is the new ax­iom suffi­cient to show that the agent can­not be money-pumped?

• It’s enough to show that an agent can­not be re­peat­edly money-pumped. The more op­por­tu­ni­ties for money pump­ing, the less chances there are of it suc­ceed­ing.

Con­trast house­hold ap­pli­cance in­surance ver­sus health in­surance. Both are a one-shot money-pump, as you get less than your ex­pected util­ity out of then. An agent fol­low­ing these ax­ioms will prob­a­bly health-in­sure, but will not ap­pli­ance in­sure.

• Can you write out the math on that? To me it looks like the Allais Para­dox or a sim­ple var­i­ant would still go through. It is easy for the ex­pected var­i­ance of a bet to in­crease as a re­sult of learn­ing ad­di­tional in­for­ma­tion—in fact the Allais Para­dox de­scribes ex­actly this. So you could pre­fer A to B when they are bun­dled with var­i­ance-re­duc­ing most prob­a­ble out­come C, and then af­ter C is ruled out by fur­ther ev­i­dence, pre­fer B to A. Thus you’d pay a penny at the start to get A rather than B if not-C, and then af­ter learn­ing not-C, pay an­other penny to get B rather than A.

• I’ll try and do the maths. This is some­what com­plex with­out in­de­pen­dence, as you have to es­ti­mate what the to­tal re­sults of fol­low­ing a cer­tain strat­egy is, over all the bets you are likely to face. Ob­vi­ously you can’t money pump me if I know you are go­ing to do it; I just com­bine all the bets and see it’s a money pump, and so don’t fol­low it.

So if you tried to money pump me re­peat­edly, I’d es­ti­mate it was likely that I’d be money pumped, and ad­just my strat­egy ac­cord­ingly.

• I be­lieve SilasBarta has cor­rectly (if that is the word) noted that it does not—it is perfectly pos­si­ble for an agent to satisfy the new ax­ioms and fall vic­tim to the Allais Para­dox.

Edit: cor­rec­tion—he does not state this.

• That sounds more like the ex­act op­po­site of my po­si­tion.

• I apol­o­gize. In the course of con­ver­sa­tion with you, I came to that con­clu­sion, but you re­ject that po­si­tion.

• To sum­ma­rize my point: if you fol­low the new ax­ioms, you will act differ­ently in one-shot vs. mas­sive-shot sce­nar­ios. Act­ing like the former in the lat­ter will cause you to be money-pumped, but per the ax­ioms, you never ac­tu­ally do it. So you can fol­low the new ax­ioms, and still not get money-pumped.

• Your ax­iom talks about ex­pected util­ity, but you have not defined that term yet.

• The post as­sumes a knowl­edge of ba­sic statis­tics through­out—in such a con­text, the mean­ing of “ex­pected util­ity” is trans­par­ent.

• Sorry, I meant the defi­ni­tion of util­ity.

[edit: this should have been a re­ply to Stu­art Arm­strong’s com­ment be­low RobinZ’s.]

• Utility is the thing you want to max­i­mize in your de­ci­sion-mak­ing.

• A de­ci­sion-maker in gen­eral isn’t nec­es­sar­ily max­i­miz­ing any­thing. Von Neu­mann and Mor­gen­stern showed that if you satisfy ax­ioms 1 through 4, then you do in fact take ac­tions which max­i­mize ex­pected util­ity for some util­ity func­tion. But this post is ig­nor­ing ax­iom 4 and as­sum­ing only ax­ioms 1 through 3. In that case, why should we ex­pect there to be a util­ity func­tion?

• Thanks for bring­ing this up, and I’ve change my post to re­flect your com­ments. Un­for­tu­nately, I have to de­cree a util­ity func­tion ahead of time for this to make any sense, as I can change the mean and SD of any dis­tri­bu­tion by just chang­ing my util­ity func­tion.

I have a new post up that ar­gues that where small sums are con­cerned, you have to have a util­ity func­tion lin­ear in cash.

• ? This is just the stan­dard defi­ni­tion. The mean of the ran­dom vari­able, when it is ex­pressed in terms of utils.

Should this be speci­fied in the post, or is it com­mon knowl­edge on this list?

• The Von-Neu­mann Mor­gen­stern ax­ioms talk just about prefer­ence over lot­ter­ies, which are sim­ply prob­a­bil­ity dis­tri­bu­tions over out­comes. That is you have an un­struc­tured set O of out­comes, and you have a to­tal pre­order­ing over Dist(O) the set of prob­a­bil­ity dis­tri­bu­tions over O. They do not talk about a util­ity func­tion. This is quite el­e­gant, be­cause to make de­ci­sions you must have prefer­ences over dis­tri­bu­tions over out­comes, but you don’t need to as­sume that O has a cer­tain struc­ture, e.g. that of the re­als.

The ex­pected util­ity the­o­rem says that prefer­ences which satisfy the first four ax­ioms are ex­actly those which can be rep­re­sented by:

A ⇐ B iff E[U;A] ⇐ E[U;B]

for some util­ity func­tion U: O → R, where

E[U;A] = \sum{o} A(o) U(o)

How­ever, U is only defined up to pos­i­tive af­fine trans­for­ma­tion i.e. aU+b will work equally well for any a>0. In par­tic­u­lar, you can am­plify the stan­dard de­vi­a­tion as much as you like by re­defin­ing U.

Your ax­ioms re­quire you to pick a par­tic­u­lar rep­re­sen­ta­tion of U for them to make sense. How do you choose this U? Even with a mechanism for choos­ing U, e.g. as­sume bounded non­triv­ial prefer­ences and pick the unique U such that \sup{x} U(x) = 1 and \inf{x} U(x) = 0, this is still less el­e­gant than talk­ing di­rectly about lot­ter­ies.

Can you re­define your ax­ioms to talk only about lot­ter­ies over out­comes?

• Can you re­define your ax­ioms to talk only about lot­ter­ies over out­comes?

Alas no. I’ve changed my post to ex­plain the difficul­ties as I can change the mean and SD of any dis­tri­bu­tion by just chang­ing my util­ity func­tion.

I have a new post up that ar­gues that where small sums are con­cerned, you have to have a util­ity func­tion lin­ear in cash.

• You started out by as­sum­ing a prefer­ence re­la­tion on lot­ter­ies with var­i­ous prop­er­ties. The com­plete­ness, tran­si­tivity, and con­ti­nu­ity ax­ioms talk about this prefer­ence re­la­tion. Your “stan­dard de­vi­a­tion bound” ax­iom, how­ever, talks about a util­ity func­tion. What util­ity func­tion?

• It’s a good re­sult, but I won­der if the stan­dard de­vi­a­tion is the best pa­ram­e­ter. Loss-averse agents re­act differ­ently to asym­met­ri­cal dis­tri­bu­tions al­low­ing large losses than those al­low­ing large gains.

Edit: For ex­am­ple, the mean of an ex­po­nen­tial dis­tri­bu­tion f(x;t) = L e^(-Lx) has mean and stan­dard de­vi­a­tion 1/​L, but a loss-averse agent is likely to pre­fer it to the nor­mal dis­tri­bu­tion N(1/​L, 1/​L^2), which has the same mean and stan­dard de­vi­a­tion.

• Once you aban­ndon in­de­pen­dence, the pos­si­bil­ities are lit­ter­aly in­finite—and not just eas­ily con­trol­lable in­fini­ties, ei­ther. I worked with SD as that’s the sim­plest model I could use; but skew­ness, kur­to­sis or, Bayes help us, the higher mo­ments, are also valid choices.

You just have to be care­ful that your choice of units is con­sis­tent; the SD and the mean are in the same unit, the var­i­ance is in units squared, the skew­ness and kur­to­sis are unitless, the k-th mo­ment is in units to the power k, etc...

• That’s true—and it oc­curred to me af­ter I posted the com­ment that your crite­ria don’t define the de­ci­sion sys­tem any­way, so even us­ing some other method you might still be able to prove that it meets your con­di­tions.

• See also semi­var­i­ance in the con­text of in­vest­ment (and bet­ting in gen­eral). NB: “semi­var­i­ance” has a differ­ent mean­ing in the con­text of spa­tial statis­tics.

• “The mean of Y is nβ. The var­i­ance of Y is the sum of the vj, which is less than nK2.” Been a while for me, but doesn’t this re­quire the lot­ter­ies to be un­cor­re­lated? If so, that should be listed with your ax­ioms.

• It re­quires the lot­ter­ies to be in­de­pen­dent), which im­plies un­cor­re­lated. Stu­art_Arm­strong speci­fied in­de­pen­dence.

• Ugh, color me stupid—I as­sumed the “in­de­pen­dence” we were re­lax­ing was prob­a­bil­ity-re­lated. Thanks RobinZ.

• You know, I didn’t even re­al­ise I’d used “in­de­pen­dence” both ways! Most of the time, it’s only worth point­ing out the fact if the ran­dom vari­ables are not in­de­pen­dent.

• No prob­lem. (Don’t you love it when peo­ple use the same sym­bol for mul­ti­ple things in the same work? I know as a me­chan­i­cal en­g­ineer, I got so much joy from re­mem­ber­ing which “h” is the heat trans­fer co­effi­cient and which is the height!)

• “The var­i­ance of Y is the sum of the vj, which is less than nK2.” You need to spec­ify that the lot­ter­ies are un­cor­re­lated for this to be true.

• He speci­fied “in­de­pen­dent”—un­cor­re­lated is im­plied.