Generalizing Foundations of Decision Theory II

As promised in the pre­vi­ous post, I de­velop my for­mal­ism for jus­tify­ing as many of the de­ci­sion-the­o­retic ax­ioms as pos­si­ble with gen­er­al­ized dutch-book ar­gu­ments. (I’ll use the term “gen­er­al­ized dutch-book” to re­fer to ar­gu­ments with a fam­ily re­sem­blance to dutch-book or money-pump.) The even­tual goal is to re­lax these as­sump­tions in a way which ad­dresses bounded pro­cess­ing power, but for now the goal is to get as much of clas­si­cal de­ci­sion the­ory as pos­si­ble jus­tified by a gen­er­al­ized dutch-book.

I made some pre­dic­tions about what I would and wouldn’t be able to jus­tify with gen­er­al­ized dutch-book. Th­ese turned out to be only par­tially cor­rect. I was able to get a ver­sion of Jeffrey’s ev­i­den­tial de­ci­sion the­ory, but with non­stan­dard util­ity and prob­a­bil­ity func­tions and with less as­sump­tions about the struc­ture of the boolean alge­bra. Non­stan­dard prob­a­bil­ity and util­ity func­tions are not a very big gen­er­al­iza­tion in them­selves. This makes it a lit­tle less in­ter­est­ing, sup­port­ing Scott’s com­ment. I still have some hope that more in­ter­est­ing things can be de­vel­oped from this go­ing for­ward, though.

I wouldn’t be too sur­prised if this has all been done be­fore and I just haven’t been able to find it. Jus­tify­ing all of the ax­ioms in terms of gen­er­al­ized dutch-book ar­gu­ments seems like an ob­vi­ously de­sir­able thing to do. Yet, I haven’t seen such a com­plete jus­tifi­ca­tion of com­mon ax­ioms via gen­er­al­ized dutch-books; that is part of why I ini­tially as­sumed it would not be pos­si­ble. The clos­est thing I’ve heard about is Stu­art Arm­strong’s post show­ing that im­mu­nity to a gen­er­al­iza­tion of the money-pump ar­gu­ment is nec­es­sary and suffi­cient for the VNM ax­ioms minus con­ti­nu­ity. So, al­though the re­sult here doesn’t give as much of a gen­er­al­iza­tion of de­ci­sion the­ory as I might have hoped for, it’s the nicest jus­tifi­ca­tion of ev­i­den­tial de­ci­sion the­ory that I know of.


The ba­sic rep­re­sen­ta­tion of an agent’s prefer­ence is as fol­lows:

  • The agent has a set of pos­si­ble be­liefs, which is a Boolean alge­bra. Ele­ments of will be called propo­si­tions.

  • The agent has prefer­ences over , which can be re­vealed by ask­ing it to choose from a non-empty set of propo­si­tions . (Th­ese need not be mu­tu­ally ex­clu­sive; we can ask things like “Would you rather be a bank tel­ler or a fem­i­nist bank tel­ler?”) The choices of the agent are rep­re­sented by a choice func­tion , such that .

  • will be ab­bre­vi­ated . will be ab­bre­vi­ated . means that ei­ther or .

I’m rep­re­sent­ing prefer­ences on sets only so that I can ar­gue that this re­duces to bi­nary prefer­ence. Forc­ing the choice func­tion to re­turn at least one item is like as­sum­ing the com­plete­ness of bi­nary prefer­ence. This de­prives me of the chance to ap­ply Vanessa’s money-pump ar­gu­ment for com­plete­ness, but I think “the agent has to make a choice” is a de­cent jus­tifi­ca­tion for com­plete­ness. I al­low the agent to re­turn more than one choice be­cause to do oth­er­wise would be equiv­a­lent to as­sum­ing anti-sym­me­try; this seems too strong. We want an agent to be able to be in­differ­ent be­tween propo­si­tions. (You can imag­ine that the agent would choose ran­domly when con­fronted with such choices, but the choice func­tion en­codes the set of choices they might make.)

##Money and Contracts

In or­der to make the money-pump ar­gu­ment, I’ll need a con­cept of money. I as­sume that money comes from an or­dered field . To make dutch-book ar­gu­ments, I also need con­tracts which pay money con­di­tional on cer­tain propo­si­tions. Th­ese can be rep­re­sented as a pair, . The agent holds a mul­ti­set of con­tracts, since it’s mean­ingful to hold sev­eral copies of the same con­tract. In fact, for one proof I’ll need counts of con­tracts to come from , so that we can have frac­tion­ally many, nega­tively many, and what­ever else pro­vides. (This is a some­what un­for­tu­nate com­pli­ca­tion, but I didn’t find a way to do with­out it.) How­ever, to avoid headaches, I con­sider only mul­ti­sets with finitely many non-zero counts. So, putting these to­gether, we will ex­tend the prefer­ences of the agent to triples giv­ing a propo­si­tion, an amount of money, and a multi-set of con­tracts: . Such triples will be called prospects. I will still use the ab­bre­vi­a­tions , , and . I’ll also write to in­di­cate the ad­di­tion of money to a prospect. Multi-set ad­di­tion will be writ­ten , so for ex­am­ple means that is like ex­cept that the count of con­tract is one greater.

Lemma 1 (con­straints on value of money). We can ex­tend on non-empty sets of propo­si­tions to a func­tion on non-empty sets of prospects, in a way which satis­fies the fol­low­ing re­quire­ments:

  • Like choices over propo­si­tions, choices over prospects re­turn only el­e­ments from the given set, and re­turn at least one such el­e­ment: for a non-empty set of prospects , we have and .

  • For a set of propo­si­tions , let be the set of prospects where . Then we re­quire . In other words, de­ci­sions are the same as be­fore when no money or con­tracts are in­volved.

  • All else be­ing equal, more money is bet­ter: for real num­bers , .

  • For a set of prospects , let be the set de­rived by adding to each prospect in . We re­quire .

  • We re­quire that if the choice set con­tains more than one prospect, sub­tract­ing any pos­i­tive amount of money from a sin­gle item re­moves it from the choice set. For ex­am­ple, if , then for , .

  • For each set of prospects , there is a quan­tity of money such that we can sub­tract it from all prospects in (leav­ing the rest of the same) with­out chang­ing the choice set. That is, . (This is a way of say­ing that a strict prefer­ence im­plies a prefer­ence we’re will­ing to pay for—money has small enough units to be less im­por­tant than any­thing else.)

  • For each set of prospects such that , there is a quan­tity of money such that we can sub­tract it from all the prospects in (leav­ing the rest of the same), all those el­e­ments would be ex­cluded from the new choice set. That is, . (This is a way of say­ing that any prefer­ence can be over­come with enough money.)

  • Con­tracts are val­ued in­de­pen­dently of each other and of the amount of money the agent has; IE, then for all .

Proof. Let be the or­dered field defined by ra­tio­nal-co­effi­cient polyno­mi­als in , con­sid­er­ing in­te­ger pow­ers of and tak­ing and for all .

For a set of prospects , let be the set cho­sen by the fol­low­ing pro­cess: or­der the prospects first by the nega­tive-power el­e­ments of the money alone, dis­card­ing any prospects which are non-max­i­mal in this re­spect, to yield ; Ap­ply to the set of propo­si­tions of prospects in , keep­ing all prospects whose propo­si­tions would be kept by ; fi­nally, or­der the re­main­ing set by money (con­sid­er­ing non-nega­tive pow­ers in the polyno­mi­als this time), keep­ing only the max­i­mal el­e­ments. In other words, we first con­sider the in­finite amounts of money; then, we con­sider what would do; fi­nally, we con­sider the finite and in­finites­i­mal amounts of money. This defi­ni­tion of satis­fies all prop­er­ties above.

Note that the given con­struc­tion of is not nec­es­sar­ily the one the agent will use; for ex­am­ple, most agents typ­i­cally con­sid­ered would be bet­ter-rep­re­sented by tak­ing and as­sign­ing real val­ues to propo­si­tions. Rather, this lemma is just to show that my as­sump­tions about how the agent’s prefer­ences are ex­tended to prospects aren’t re­strict­ing the agent’s na­tive prefer­ences on propo­si­tions at all. Any prefer­ences on propo­si­tions can be ex­tended to prospects in a way which obeys these rules. The same could not be said of other similarly nat­u­ral as­sump­tions, for ex­am­ple as­sum­ing all of the above while re­quiring real-val­ued money. We don’t want to sneak in re­stric­tions on agent’s prefer­ences via as­sump­tions about money; the money should only serve to fa­cil­i­tate dutch-book ar­gu­ments.

Hence­forth, refers to some func­tion which obeys the prop­er­ties from lemma 1.

As for the ques­tion of why these par­tic­u­lar as­sump­tions are made rather than some other set, this is at the level of ask­ing why an agent should care about dutch-book or money-pump ar­gu­ments, and I’ll make lit­tle at­tempt to an­swer that here. The as­sump­tions are made with the ex­press pur­pose of fa­cil­i­tat­ing the proof. One way to look at it is that we’re ask­ing the agent hy­po­thet­i­cal ques­tions, aug­ment­ing the propo­si­tions the agent un­der­stands with to­tally imag­i­nary ob­jects called “money” and “con­tracts”. The pos­si­bil­ity of our as­sump­tions about these be­ing “wrong” is mean­ingless. In­stead, the ques­tion is whether ar­gu­ments em­ploy­ing such hy­po­thet­i­cals to con­clude var­i­ous prin­ci­ples of ra­tio­nal­ity are com­pel­ling.


The agent plays games with the bookie, as fol­lows:

  • The game pro­ceeds in rounds. At each round, the agent is “hold­ing” some prospect. In the ini­tial round, the agent holds a prospect given by the bookie (the “start­ing prospect”).

  • The bookie can move the game to the next round in two ways:

    • The bookie can offer the agent a set of prospects (“choice round”). The set must in­clude the prospect which the agent is cur­rently hold­ing. The agent chooses one of the prospects, ac­cord­ing to . If con­tains sev­eral prospects, we let the bookie choose. The agent holds the cho­sen prospect in the next round.

    • The bookie can test a propo­si­tion (“test round”). This re­fines what the agent is hold­ing ac­cord­ing to whether the test comes out true or false. In ad­di­tion, con­tracts pay out if they are trig­gered by the new propo­si­tion. More speci­fi­cally: if the agent is hold­ing in this round, and the bookie tests , then the agent ei­ther holds in the next round or . is minus any con­tracts such that ; the quan­tity is the sum of the pay­outs of those con­tracts. (The multi-set pay­out of copies of a con­tract is .) Similarly, sub­tracts the con­tracts ac­ti­vated by the other case, whose pay­outs are summed up in . We can never move to a prospect whose propo­si­tion is ; so, some­times there will only be one op­tion.

  • The bookie can end play at any round.

  • I re­strict games to have all offers be­fore all tests, so that games pro­ceed in two phases; in phase one, the bookie pre­sents the agent with a se­quence of choices, while in phase two, the bookie tests propo­si­tions in or­der to pay out con­tracts. I don’t think this re­stric­tion makes any differ­ence ex­cept to sim­plify mat­ters.

(Let­ting the bookie choose when the agent is in­differ­ent is a con­ve­nience; we could con­sider the agent’s re­sponse to be ran­dom, but we wouldn’t want to con­sider an agent im­mune to dutch-book if it some­times es­caped them via ran­dom choice. We’d con­demn the agent if it had a chance of be­ing dutch-booked. So, it’s sim­pler to just say that the bookie chooses.)

Fix­ing a par­tic­u­lar bookie strat­egy and out­comes for each test round, we get a play-out. Sup­pose the agent starts off hold­ing . Call the prospect which the agent ac­tu­ally holds at the end of the play-out . Then an agent ex­pe­riences re­gret if, in ev­ery situ­a­tion, is as bad or worse than it would have if it had stuck with the start­ing prospect. More for­mally, re­gret is defined in two cases:

  • If the last prospect held be­fore any test rounds be­gin is (IE, we hold the start­ing propo­si­tion), then take the start­ing prospect through the same se­quence of test rounds (if any) with the same out­comes to get a com­par­i­son prospect . This is the end-con­di­tion which would have re­sulted if the agent had stuck with the ini­tial prospect, but the bookie had run the same tests and they had come out the same way. If , the agent has re­gret. If , the re­gret is strict.

  • If the last prospect be­fore test rounds doesn’t have propo­si­tion , then in­stead, take the start­ing prospect and gen­er­ate the set of all pos­si­ble re­sults if the agent had stuck with the start­ing prospect, and the bookie had performed the same tests, but the tests could go ei­ther way within the limits of log­i­cal pos­si­bil­ity. In this case, the agent is only said to ex­pe­rience re­gret if is less than or equal to ev­ery el­e­ment of . The re­gret is strict if is strictly less than any el­e­ment of .

A bookie’s strat­egy is a gen­er­al­ized dutch-book if the agent re­grets ev­ery pos­si­ble play-out, and strictly re­grets at least one play-out.

My defi­ni­tion of re­gret is a bit clunky. The in­tu­ition be­hind split­ting up the cases in this way can be un­der­stood by a cou­ple of ex­am­ples.

Sup­pose the agent starts with prospect . The bookie offers the choice of trad­ing this for , which the agent ac­cepts. This is in­ter­preted as pay­ing $10 for a $1 bet on . The bookie then tests for , which comes up true. The agent now holds . Clearly, the agent should re­gret this out­come. If it had stuck with , it would have af­ter a suc­cess­ful test for , which is strictly preferred by my as­sump­tions con­cern­ing money.

On the other hand, sup­pose that the agent starts with and is offered the trade of , which it prefers. The bookie then tests , which can only come out true. The agent now has . In this case, it doesn’t make sense to say that the agent could have had if it had only de­clined the trade. came out true be­cause of the trade. To es­tab­lish re­gret in this case, we would want both and .

Still, I’m not to­tally happy com­par­ing to the start­ing prospect in this way. It would be more nat­u­ral to com­pare the agent to agents with differ­ing prefer­ences: an agent has (strict) re­gret if some other prefer­ences would have given it (strictly) bet­ter out­comes, given the bookie’s strat­egy. We then deem a sys­tem of prefer­ences ir­ra­tional if some other sys­tem of prefer­ences achieves su­pe­rior re­sults even by the judge­ment of . How­ever, that defi­ni­tion wouldn’t work here; book­ies could re­ward ir­ra­tional­ity, mak­ing ev­ery prefer­ence sys­tem dutch-book­able. The defi­ni­tion I use avoids this.

##Ar­gu­ments for Necessity

Here, I prove a num­ber of prop­er­ties on the as­sump­tion that the agent is not sus­cep­ti­ble to gen­er­al­ized dutch-books.

We can show that all prefer­ences re­duce to bi­nary prefer­ence (a ver­sion of In­de­pen­dence of Ir­rele­vant Alter­na­tives):

The­o­rem 1 (Re­duc­tion to Bi­nary Prefer­ence). If is not sus­cep­ti­ble to a gen­er­al­ized dutch-book, then is ex­actly the set of prospects such that for all .

Proof. Sup­pose not. Then ei­ther there is a with a such that , or oth­er­wise, there is a with no such but .

In the first case, the bookie can start the agent with , pre­sent the agent with the set of choices , and make the agent choose . At this point, the bookie can offer , with suffi­ciently small so as to be fa­vor­able. The agent is now strictly worse off than it started, con­tra­dict­ing the as­sump­tion that it has no gen­er­al­ized dutch-books.

In the sec­ond case, start the agent with , and pre­sent the choice of . The agent will choose some­thing other than . Now the bookie can charge to switch back, offer­ing . Again, this should be im­pos­si­ble.

Here’s the clas­sic money-pump:

The­o­rem 2 (Tran­si­tivity). If is not sus­cep­ti­ble to a gen­er­al­ized dutch-book, then when­ever and , .

Proof. Sup­pose not. Then we can find such that and , but . The bookie can start the agent hold­ing and offer the choice to switch to and then . The agent ei­ther strictly prefers these or is in­differ­ent; in ei­ther case, the bookie can get the agent to switch. Since , the bookie can now offer . By the prop­er­ties of money, there ex­ists an small enough to en­sure this trade is fa­vor­able. The agent is now strictly worse off than it started.

Note that bi­nary prefer­ence is com­plete by defi­ni­tion, so we have com­plete, tran­si­tive prefer­ences. Also note that tran­si­tivity of im­plies tran­si­tivity of and of .

Now we prove a ver­sion of Jeffrey’s law of av­er­ag­ing:

The­o­rem 3 (Aver­ag­ing). Sup­pose is not sus­cep­ti­ble to a gen­er­al­ized dutch-book. If , then im­plies .

Proof. Sup­pose not. Then by tran­si­tivity, ei­ther the dis­junc­tion is strictly preferred to both dis­juncts, or both dis­juncts are strictly preferred to it.

In the first case, sup­pose wlog that . Start the agent with . Offer the choice to switch to with small enough to be ap­peal­ing. The agent ac­cepts. Then, test whether . The agent ei­ther ends up with or . In both cases, it ex­pe­riences strict re­gret.

In the sec­ond case, sup­pose again that wlog. Start the agent with and charge to switch to . Then, test for . This cre­ates strict re­gret, since the agent would have ended up in ei­ther or if it did noth­ing, both of which are bet­ter than .

Next, we want to make clas­sic dutch-book ar­gu­ments to es­tab­lish prob­a­bil­ity laws. To do this, how­ever, we need to es­tab­lish some prop­er­ties of . Speci­fi­cally, we can show from tran­si­tivity and com­plete­ness that prospects act like val­ues which ex­tend the or­dered field .

Defi­ni­tion. Sup­pose that is iso­mor­phic with a sub­field of the sur­real num­bers. Fix one such sub­field to iden­tify with . Also fix an ar­bi­trary well-or­der­ing of prospects, , which will act like the sur­real or­der­ing of stages. Then the rel­a­tive value of prospect rel­a­tive to prospect , de­noted , mea­sures the prices at which the agent will ac­cept offers to switch: it is defined as the sur­real num­ber where is the set of mon­e­tary val­ues such that , plus those val­ues where and . is the set of mon­e­tary val­ues such that , plus those val­ues where and . The value (non-rel­a­tive) shall be defined as .

(There are broad con­di­tions un­der which can be em­bed­ded into the sur­real num­bers as re­quired here; in NBG set the­ory, this is always pos­si­ble.)

Lemma 2. If is iso­mor­phic to a sub­field of the sur­real num­bers, then the rel­a­tive value given above is well-defined, and the clo­sure of all prospect val­ues un­der the field op­er­a­tions is an or­dered field. Fur­ther­more, if and only if .

Proof. Due to com­plete­ness, any prospect splits the set of val­ues so far (IE, plus all ) into dis­joint sets worth less than, more than, and (pos­si­bly) the same as the trade of for . Call the first set and the sec­ond . Due to tran­si­tivity, it must be that any el­e­ment from is less than any el­e­ment from . We can con­clude that is a well-formed sur­real num­ber. We can there­fore take the clo­sure un­der field op­er­a­tions to get a new sub­field of the sur­re­als, which will it­self be an or­dered field.

Sup­pose , with and . By the defi­ni­tion of or­der­ing on sur­real num­bers, there must ei­ther be an such that , or an such that . If , let ; oth­er­wise, is for some . So, we have ei­ther or . In ei­ther case, by tran­si­tivity.

Now, sup­pose . Either or . In the first case, must have in its left set; in the sec­ond case, must have in its right set. Either way, .

Or­di­nar­ily, af­ter get­ting a no­tion of value like this we’d like to show that it is unique up to lin­ear trans­for­ma­tions or some­thing similar. This type of re­sult is very un­likely here, since the ar­bi­trary choice of can re­sult in sig­nifi­cant differ­ences in (on top of the already non-unique choice of to rep­re­sent a given ). Nonethe­less, we have the first half of a rep­re­sen­ta­tion the­o­rem. We can use this to define prob­a­bil­ity.

For no­ta­tional con­ve­nience, define the value of a con­tract to be the value of adding that con­tract to the con­tract set of prospect , with . We define prob­a­bil­ity via the value of $1 bets:

Defi­ni­tion. The prob­a­bil­ity of a propo­si­tions is defined as the value $V ([A, $1])$.

The­o­rem 4 (Prob­a­bil­ity Laws). If has no gen­er­al­ized dutch-books,

  1. (Non-Nega­tivity).

  2. if is a tau­tol­ogy (Nor­mal­iza­tion).

  3. If , then (Finite Ad­di­tivity).

  4. If , $Pr(A)>0 (Non-Dog­ma­tism).

Proof. (Non-nega­tivity.) Sup­pose . This means . But since a strict prefer­ence im­plies a prefer­ence we’re will­ing to pay for, for some . A game which offers that trade and then tests is a gen­er­al­ized dutch-book. The agent is worse off by in the case where comes out false, and worse off by if comes out true. So, this can’t hap­pen.

(Nor­mal­iza­tion.) Sup­pose is a tau­tol­ogy and . This is a prefer­ence it’s will­ing to pay for, so $(\top, 0, { [A,$1] } ) \prec (\top, 1-n, \emp­ty­set)$ for some . But this im­plies that the agent can be gen­eral-dutch-booked by start­ing it out with $(\top, 0, { [A,$1] } )$, offer­ing it the trade for , and then test­ing for (which will cer­tainly come true); the agent could have had $1 rather than $1-n. Similarly, if , we can start the agent with for some , and get the agent to trade for , ul­ti­mately leav­ing it with only af­ter test­ing . So .

(Finite Ad­di­tivity.) Sup­pose . Since and are mu­tu­ally ex­clu­sive, the agent must be in­differ­ent be­tween and the pair of con­tracts , . (Hold­ing that pair of con­tracts pays out in ex­actly the same way as hold­ing , so if they aren’t val­ued equally, then the bookie can charge a small amount for switch­ing from one to the other to make a dutch book.) How­ever, the pair of con­tracts must be val­ued as ; oth­er­wise, the amount of money the agent is will­ing to pay for a di­rect offer of the pair would be differ­ent than what the agent would be will­ing to pay if offered first and then . In the case where , the bookie can start the agent out hold­ing both and , pay the agent an amount of money slightly more than but less than (a quan­tity of money whose ex­is­tence is as­sured by or­der-den­sity) to give them up, and then charge in­di­vi­d­u­ally to give them back. In the re­verse case, the bookie would pay for their re­moval in­di­vi­d­u­ally and then charge to give them back jointly. Either way, we have a dutch book.

(Non-Dog­ma­tism.) Sup­pose and . The bookie can start the agent with and offer , which the agent is will­ing to take, and then test . If , the agent is worse off by one dol­lar; oth­er­wise, the out­come is the same as it would have been. So, this is a gen­er­al­ized dutch book.

So, we’ve got our prob­a­bil­ity and util­ity func­tions. We just need to do a lit­tle more work to con­nect them to each other.

Lemma 3 (Ex­change Rate). Prob­a­bil­ities are ex­change rates be­tween money and con­di­tional bets: .

Proof. Re­mem­ber that by a prop­erty of money, if and only if for all . This im­plies that de­pends on only and . In par­tic­u­lar, the value is in­de­pen­dent of the num­ber of copies of the con­tract already in . Since copies of a con­tract will have equiv­a­lent pay­outs to one copy of a con­tract , it must have the same value to avoid a gen­er­al­ized dutch book. Since the multi-set of con­tracts al­lows the num­ber of copies of a con­tract to be any­thing in , this ar­gu­ment ap­plies for any . Since has mul­ti­plica­tive in­verses, for any but 0 we can take , prov­ing the re­sult. The re­sult also holds for , since the value of a con­tract pay­ing zero must be zero.

In what fol­lows, I will ab­bre­vi­ate as .

The­o­rem 5 (Ex­pected Utility). If propo­si­tions and are mu­tu­ally ex­clu­sive and has no gen­er­al­ized dutch-books, .

Proof. Sup­pose . Then for any such that , , we have . Tak­ing the nega­tive of both sides and ap­ply­ing the ex­change rate lemma, we have . This means there is a price, , which the agent is will­ing to pay in or­der to trade the two con­tracts and for the con­tract . Choose so that , , and . Now, an agent who pays for the trade (start­ing from and be­ing offered ) is strictly worse off in ev­ery out­come af­ter and have been tested, as com­pared with an agent who didn’t. This con­tra­dicts our as­sump­tion that there are no gen­er­al­ized dutch-books.

Now sup­pose . By a similar ap­pli­ca­tion of the lemma, we get and make the agent pay for the trade in the op­po­site di­rec­tion. This also con­tra­dicts our as­sump­tion. So we must have the de­sired re­sult.

This com­pletes the proof that an agent im­mune to gen­er­al­ized dutch books must have prefer­ences which can be rep­re­sented by (non­stan­dard) prob­a­bil­ity and util­ity func­tions obey­ing the law of ex­pected util­ity. This in it­self does not tell us much, be­cause it may be that I can ar­gue all sorts of con­straints from the as­sump­tion that no gen­er­al­ized dutch book ex­ists—the con­di­tion could even be un­satis­fi­able. Next we want to see that prefer­ences be­ing rep­re­sentable as prob­a­bil­ity and util­ity func­tions is suffi­cient.

##Ar­gu­ment for Sufficiency

The­o­rem 6 (Suffi­ciency). If can be rep­re­sented as tak­ing the high­est- op­tion ac­cord­ing to a (pos­si­bly non­stan­dard) on propo­si­tions (ex­cept ) which obeys the law of ex­pected util­ity as me­di­ated by a (pos­si­bly non­stan­dard) func­tion which fol­lows the prob­a­bil­ity laws, then there ex­ists a which is im­mune to gen­er­al­ized dutch books.

Proof. Define a value on prospects, , where gives the mul­ti­set count of the con­tract in . Let choose the sub­set with max­i­mal . This defi­ni­tion of obeys a gen­er­al­ized ex­pected value rule, in which the value of a prospect is the ex­pected value of all pos­si­ble out­comes af­ter ap­ply­ing any com­bi­na­tion of test rounds to the prospect. Now, if is will­ing to trade dur­ing any choice rounds, it must be for a prospect with higher or equal ex­pected value. This makes a gen­er­al­ized dutch-book im­pos­si­ble. There are two cases to con­sider, based on the defi­ni­tion of re­gret. Either the agent has traded for a prospect whose propo­si­tion is the same as the start­ing propo­si­tion, or the agent has traded for one which is differ­ent. In the first case, the prob­a­bil­ities of differ­ent test out­comes all re­main the same. It is not pos­si­ble that each out­come is the same or worse as a re­sult of the trade, with at least one be­ing worse; the ex­pected util­ity would have been lower, and no trade would have been made. In the sec­ond case, the prob­a­bil­ities may change. How­ever, due to non-dog­ma­tism, all pos­si­bil­ities still have pos­i­tive prob­a­bil­ity. It is there­fore not pos­si­ble that ev­ery out­come af­ter trade is the same as or worse than ev­ery pos­si­ble out­come of the prospect be­fore trade, with at least one of the new worse than ev­ery old; this would as­sure a lower ex­pected util­ity, pre­vent­ing trade.

Now, it fol­lows from re­sults in the pre­vi­ous sec­tion that a which is im­mune to gen­er­al­ized dutch-books im­plies a and fol­low­ing the law of ex­pected util­ity, which rep­re­sent the origi­nal prefer­ences . So, com­bined with the­o­rem 6, we’ve got nec­es­sary and suffi­cient con­di­tions for such a rep­re­sen­ta­tion.


I still feel un­com­fortable with cer­tain as­pects of this, es­pe­cially the as­sump­tions which I had to edit as I went to make the ar­gu­ments go through. The most glar­ing case is the long list of as­sump­tions about money. Other bits I feel un­easy about are the M-val­ued mul­ti­sets of con­tracts and the defi­ni­tion of re­gret. It’s not a flaw in the for­mal ar­gu­ment that wor­ries me, but rather, the ques­tion of how much con­trivance we can throw into the setup. With a few differ­ent choices in defi­ni­tions, would I get a sig­nifi­cantly differ­ent de­ci­sion the­ory?

The over­all struc­ture of the ar­gu­ment is that we pro­pose a set of thought ex­per­i­ments to an agent whose choice func­tion (on non-thought-ex­per­i­ments) is . The agent is asked to con­struct a func­tion rep­re­sent­ing what it would do un­der the con­di­tions of the thought ex­per­i­ments. It can con­struct any it wants, so long as it obeys some as­sump­tions we make as part of the thought ex­per­i­ment. How­ever, if it gives us a which vi­o­lates some rule of clas­si­cal de­ci­sion the­ory, we give it back a gen­er­al­ized dutch-book illus­trat­ing a case where seems “in­con­sis­tent”: sub-op­ti­mal by its own stan­dards.

From these thought ex­per­i­ments, the agent is not only sup­posed to ar­rive at a “con­sis­tent” ; it is sup­posed to be mo­ti­vated to re­vise if nec­es­sary to do the job. Pre­sum­ably, this is be­cause “in­con­sis­ten­cies” in the thought ex­per­i­ments are sup­posed to in­di­cate real in­con­sis­ten­cies in (where the sys­tem of prefer­ences it­self judges some other sys­tem to be bet­ter, in some sense). Of course, this isn’t di­rectly true.

Per­haps a bet­ter way of think­ing about it is not that we’re try­ing to con­vince an agent to change its own prefer­ences, but that we’re try­ing to con­vince an agent’s de­signer to give it co­her­ent prefer­ences in the first place. Still, it’s un­clear why the de­signer should pay spe­cial at­ten­tion to this class of hy­po­thet­i­cal sce­nar­ios.

I did the whole ar­gu­ment this way to avoid “struc­tural” as­sump­tions, where the agent’s be­liefs are as­sumed to already con­tain things like fair coins in or­der to define prob­a­bil­ity, and so on. In the end, though, an agent may only find these hy­po­thet­i­cals con­vinc­ing to the ex­tent that they re­sem­ble the struc­ture of situ­a­tions it ac­tu­ally be­lieves in.

Per­haps ar­gu­ments based on money can be seen as mak­ing tran­si­tivity and other de­ci­sion-the­o­retic prop­er­ties “con­ta­gious”: if there is some­thing you care about ap­prox­i­mately in­de­pen­dently of ev­ery­thing else, which has prop­er­ties of money, then you can ar­gue for clean de­ci­sion-the­o­retic prop­er­ties for ev­ery­thing else based on that one thing.

How­ever, some as­pects seem strange even with this kind of think­ing. One of the ba­sic as­pects of money-pump ar­gu­ments, which I’ve faith­fully repli­cated here, is that the agent ap­par­ently must “for­get” pre­vi­ous choices, and de­cide only on the ba­sis of the cur­rent choice in front of it. I think there is a strong case to be made that by re­quiring a choice to be made only as a func­tion of the prospects on offer in a given round, we’re not let­ting the agent fully un­der­stand the game it is play­ing. This leads to ob­jec­tions such as “you can re­fuse fur­ther trans­ac­tions when you no­tice a money-pump” (for a dis­cus­sion, see Money Pumps, Di­achronic and Syn­chronic, Yair Levy).

In any case, these are ques­tions which I hope to an­swer more deeply in the next stage. Now that I have a ver­sion of clas­si­cal DT jus­tified ex­clu­sively by gen­er­al­ized dutch-book, I’ll be think­ing about how to adapt this kind of thing to log­i­cal un­cer­tainty and per­haps other MIRI prob­lems. I won’t promise an­other post in the se­ries; it might not go any­where, or I might de­cide that the ex­pected value is low and I should do other things for a while. Hope­fully this post has some value.