Complete Class: Consequentialist Foundations

The fun­da­men­tals of Bayesian think­ing have been jus­tified in many ways over the years. Most peo­ple here have heard of the VNM ax­ioms and Dutch Book ar­gu­ments. Far fewer, I think, have heard of the Com­plete Class The­o­rems (CCT).

Here, I ex­plain why I think of CCT as a more purely con­se­quen­tial­ist foun­da­tion for de­ci­sion the­ory. I also show how com­plete-class style ar­gu­ments play a role is so­cial choice the­ory, jus­tify­ing util­i­tar­i­anism and a ver­sion of futarchy. This means CCT acts as a bridg­ing anal­ogy be­tween sin­gle-agent de­ci­sions and col­lec­tive de­ci­sions, there­fore shed­ding some light on how a pile of agent-like pieces can come to­gether and act like one agent. To me, this sug­gests a po­ten­tially rich vein of in­tel­lec­tual ore.

I have some ideas about mod­ify­ing CCT to be more in­ter­est­ing for MIRI-style de­ci­sion the­ory, but I’ll only do a lit­tle of that here, mostly ges­tur­ing at the prob­lems with CCT which could mo­ti­vate such mod­ifi­ca­tions.


Background

My Motives

This post is a con­tinu­a­tion of what I started in Gen­er­al­iz­ing Foun­da­tions of De­ci­sion The­ory and Gen­er­al­iz­ing Foun­da­tions of De­ci­sion The­ory II. The core mo­ti­va­tion is to un­der­stand the jus­tifi­ca­tion for ex­ist­ing de­ci­sion the­ory very well, see which as­sump­tions are weak­est, and see what hap­pens when we re­move them.

There is also a sec­ondary mo­ti­va­tion in hu­man (ir)ra­tio­nal­ity: to the ex­tent foun­da­tional ar­gu­ments are real rea­sons why ra­tio­nal be­hav­ior is bet­ter than ir­ra­tional be­hav­ior, one might ex­pect these ar­gu­ments to be helpful in teach­ing or train­ing ra­tio­nal­ity. This is re­lated to my crite­rion of con­se­quen­tial­ism: the ar­gu­ment in fa­vor of Bayesian de­ci­sion the­ory should di­rectly point to why it mat­ters.

With re­spect to this sec­ond quest, CCT is in­ter­est­ing be­cause Dutch Book and money-pump ar­gu­ments point out ir­ra­tional­ity in agents by ex­ploit­ing the ir­ra­tional agent. CCT is more amenable to a model in which you point out ir­ra­tional­ity by helping the ir­ra­tional agent. I am work­ing on a more thor­ough ex­pan­sion of that view with some co-au­thors.

Other Foundations

(Skip this sec­tion if you just want to know about CCT, and not why I claim it is bet­ter than al­ter­na­tives.)

I give an overview of many pro­posed foun­da­tional ar­gu­ments for Bayesi­anism in the first post in this se­ries. I called out Dutch Book and money-pump ar­gu­ments as the most promis­ing, in terms of mo­ti­vat­ing de­ci­sion the­ory only from “win­ning”. The sec­ond post in the se­ries at­tempted to mo­ti­vate all of de­ci­sion the­ory from only those two ar­gu­ments (ex­tend­ing work of Stu­art Arm­strong along those lines), and suc­ceeded. How­ever, the re­sult­ing ar­gu­ment was in it­self not very satis­fy­ing. If you look at the struc­ture of the ar­gu­ment, it jus­tifies con­straints on de­ci­sions via prob­lems which would oc­cur in hy­po­thet­i­cal games in­volv­ing money. Many philoso­phers have ar­gued that the Dutch Book ar­gu­ment is in fact a way of illus­trat­ing in­con­sis­tency in be­lief, rather than truly an ar­gu­ment that you must be con­sis­tent or else. I think this is right. I now think this is a se­ri­ous flaw be­hind both Dutch Book and money-pump ar­gu­ments. There is no pure con­se­quen­tial­ist rea­son to con­strain de­ci­sions based on con­sis­tency re­la­tion­ships with thought ex­per­i­ments.

The po­si­tion I’m defend­ing in the cur­rent post has much in com­mon with the pa­per Ac­tu­al­ist Ra­tion­al­ity by C. Man­ski. My dis­agree­ment with him lies in his dis­mis­sal of CCT as yet an­other bad ar­gu­ment. In my view, CCT seems to ad­dress his con­cerns al­most pre­cisely!

Caveat --

Dutch Book ar­gu­ments are fairly prac­ti­cal. Bet­ting with peo­ple, or ask­ing them to con­sider hy­po­thet­i­cal bets, is a use­ful tool. It may even be what con­vinces some­one to use prob­a­bil­ities to rep­re­sent de­grees of be­lief. How­ever, the ar­gu­ment falls apart if you ex­am­ine it too closely, or at least re­quires ex­tra as­sump­tions which you have to ar­gue in a differ­ent way. Sim­ply put, be­lief is not liter­ally the same thing as will­ing­ness to bet. Con­se­quen­tial­ist de­ci­sion the­o­ries are in the busi­ness of re­lat­ing be­liefs to ac­tions, not re­lat­ing be­liefs to bet­ting be­hav­ior.

Similarly, money-pump ar­gu­ments can some­times be ex­tremely prac­ti­cal. The re­source you’re pumped of doesn’t need to be money—it can sim­ply be the cost of think­ing longer. If you spin for­ever be­tween differ­ent op­tions be­cause you pre­fer straw­berry ice cream to choco­late and choco­late to vanilla and vanilla to straw­berry, you will not get any ice cream. How­ever, the set-up to money pump as­sumes that you will not no­tice this hap­pen­ing; what­ever the ex­tra cost of in­de­ci­sion is, it is placed out­side of the con­sid­er­a­tions which can in­fluence your de­ci­sion.

So, Dutch Book “defines” be­lief as will­ing­ness-to-bet, and money-pump “defines” prefer­ence as will­ing­ness-to-pay; in do­ing so, both ar­gu­ments put the jus­tifi­ca­tion of de­ci­sion the­ory into hy­po­thet­i­cal ex­ploita­tion sce­nar­ios which are not quite the same as the ac­tual de­ci­sions we face. If these were the best jus­tifi­ca­tions for con­se­quen­tial­ism we could muster, I would be some­what dis­satis­fied, but would likely leave it alone. For­tu­nately, a bet­ter al­ter­na­tive ex­ists: com­plete class the­o­rems.

Four Com­plete Class Theorems

For a thor­ough in­tro­duc­tion to com­plete class the­o­rems, I recom­mend Peter Hoff’s course notes. I’m go­ing to walk through four com­plete class the­o­rems deal­ing with what I think are par­tic­u­larly in­ter­est­ing cases. Here’s a map:

In words: first we’ll look at the stan­dard setup, which as­sumes like­li­hood func­tions. Then we will re­move the as­sump­tion of like­li­hood func­tions, since we want to ar­gue for prob­a­bil­ity the­ory from scratch. Then, we will switch from talk­ing about de­ci­sion the­ory to so­cial choice the­ory, and use CCT to de­rive a var­i­ant of Harsanyi’s util­i­tar­ian the­o­rem, AKA Harsanyi’s so­cial ag­gre­ga­tion the­o­rem, which tells us about co­op­er­a­tion be­tween agents with com­mon be­liefs (but differ­ent util­ity func­tions). Fi­nally, we’ll add like­li­hoods back in. This gets us a ver­sion of Critch’s multi-ob­jec­tive learn­ing frame­work, which tells us about co­op­er­a­tion be­tween agents with differ­ent be­liefs and differ­ent util­ity func­tions.

I think of Harsanyi’s util­i­tar­i­anism the­o­rem as the best jus­tifi­ca­tion for util­i­tar­i­anism, in much the same way that I think of CCT as the best jus­tifi­ca­tion for Bayesian de­ci­sion the­ory. It is not an ar­gu­ment that your per­sonal val­ues are nec­es­sar­ily util­i­tar­ian-al­tru­ism. How­ever, it is a strong ar­gu­ment for util­i­tar­ian al­tru­ism as the most co­her­ent way to care about oth­ers; and fur­ther­more, to the ex­tent that groups can make ra­tio­nal de­ci­sions, I think it is an ex­tremely strong ar­gu­ment that the group de­ci­sion should be util­i­tar­ian. AlexMen­nen dis­cusses the the­o­rem and im­pli­ca­tions for CEV here.

I some­what jok­ingly think of Critch’s vari­a­tion as “Critch’s Futarchy the­o­rem”—in the same way that Harsanyi shows that util­i­tar­i­anism is the unique way to make ra­tio­nal col­lec­tive de­ci­sions when ev­ery­one agrees about the facts on the ground, Critch shows that ra­tio­nal col­lec­tive de­ci­sions when there is dis­agree­ment must in­volve a bet­ting mar­ket. How­ever, Critch’s con­clu­sion is not quite Futarchy. It is more ex­treme: in Critch’s frame­work, agents bet their vot­ing stake rather than money! The more bets you win, the more con­trol you have over the sys­tem; the more bets you lose, the less your prefer­ences will be taken into ac­count. This is, per­haps, rather harsh in com­par­i­son to gov­er­nance sys­tems we would want to im­ple­ment. How­ever, ra­tio­nal agents of the clas­si­cal Bayesian va­ri­ety are happy to make this trade.

Without fur­ther adieu, let’s dive into the the­o­rems.

Ba­sic CCT

We set up de­ci­sion prob­lems like this:

  • is the set of pos­si­ble states of the ex­ter­nal world.
  • is the set of pos­si­ble ob­ser­va­tions.
  • is the set of ac­tions which the agent can take.
  • is a like­li­hood func­tion, giv­ing the prob­a­bil­ity of an ob­ser­va­tion un­der a par­tic­u­lar world-state .
  • is a set of de­ci­sion rules. For , out­puts an ac­tion. Stochas­tic de­ci­sion rules are al­lowed, though, in which case we should re­ally think of it as out­putting an ac­tion prob­a­bil­ity.
  • , the loss func­tion, takes a world and an ac­tion and re­turns a real-val­ued “loss”. en­codes prefer­ences: the lower the loss, the bet­ter. One way of think­ing about this is that the agent knows how its ac­tions play out in each pos­si­ble world; the agent is only un­cer­tain about con­se­quences be­cause it doesn’t know which pos­si­ble world is the case.

In this post, I’m only go­ing to deal with cases where and are finite. This is not a minor the­o­ret­i­cal con­ve­nience—things get sig­nifi­cantly more com­pli­cated with un­bounded sets, and the jus­tifi­ca­tion for Bayesi­anism in par­tic­u­lar is weaker. So, it’s po­ten­tially quite in­ter­est­ing. How­ever, there’s only so much I want to deal with in one post.

Some more defi­ni­tions:

The risk of a policy in a par­tic­u­lar true world-state: =.

A de­ci­sion rule is a pareto im­prove­ment over an­other rule if and only if for all , and strictly > for at least one. This is typ­i­cally called dom­i­nance in treat­ments of CCT, but it’s ex­actly par­allel to the idea of pareto-im­prove­ment from eco­nomics and game the­ory: ev­ery­one is at least as well off, and at least one per­son is bet­ter off. An im­prove­ment which harms no one. The only differ­ence here is that it’s with re­spect to pos­si­ble states, rather than peo­ple.

A de­ci­sion rule is ad­mis­si­ble if and only if there is no pareto im­prove­ment over it. The idea is that there should be no rea­son not to take pareto im­prove­ments, since you’re only do­ing bet­ter no mat­ter what state the world turns out to be in. (We could also call this pareto-op­ti­mal.)

A class of de­ci­sion rules is a com­plete class if and only if for any rule not in , , there ex­ists a rule in which is a pareto im­prove­ment. Note, not ev­ery rule in a com­plete class will be ad­mis­si­ble it­self. In par­tic­u­lar, the set of all de­ci­sion rules is a com­plete class. So, the com­plete class is a de­vice for prov­ing a weaker re­sult than ad­mis­si­bil­ity. This will ac­tu­ally be a bit silly for the finite case, be­cause we can char­ac­ter­ize the set of ad­mis­si­ble de­ci­sion rules. How­ever, it is the name­sake of com­plete class the­o­rems in gen­eral; so, I figured that it would be con­fus­ing not to in­clude it here.

Given a prob­a­bil­ity dis­tri­bu­tion on world-states, the Bayes risk is the ex­pected risk over wor­lds, IE: .

A prob­a­bil­ity dis­tri­bu­tion is non-dog­matic when for all .

A de­ci­sion rule is bayes-op­ti­mal with re­spect to a dis­tri­bu­tion if it min­i­mizes Bayes risk with re­spect to . (This is usu­ally called a Bayes rule with re­spect to , but that seems fairly con­fus­ing, since it sounds like “Bayes’ rule” aka Bayes’ the­o­rem.)

THEOREM: When and are finite, de­ci­sion rules which are bayes-op­ti­mal with re­spect to a non-dog­matic are ad­mis­si­ble.

PROOF: On the one hand, if is Bayes-op­ti­mal with re­spect to non-dog­matic , it min­i­mizes the ex­pec­ta­tion . Since for each world, any pareto-im­prove­ment (which must be strictly bet­ter in some world, and not worse in any) must de­crease this ex­pec­ta­tion. So, must be min­i­miz­ing the ex­pec­ta­tion if it is Bayes-op­ti­mal.

THEOREM: (ba­sic CCT) When and are finite, a de­ci­sion rule is ad­mis­si­ble if and only if it is Bayes-op­ti­mal with re­spect to some prior .

PROOF: If is ad­mis­si­ble, we wish to show that it is Bayes-op­ti­mal with re­spect to some .

A de­ci­sion rule has a risk in each world; think of this as a vec­tor in . The set of achiev­able risk vec­tors in (given by all ) is con­vex, since we can make mixed strate­gies be­tween any two de­ci­sion rules. It is also closed, since and are finite. Con­sider a risk vec­tor as a point in this space (not nec­es­sar­ily achiev­able by any ). Define the lower quad­rant to be the set of points which would be pareto im­prove­ments if they were achiev­able by a de­ci­sion rule. Note that for an ad­mis­si­ble de­ci­sion rule with risk vec­tor , and are dis­joint. By the hy­per­plane sep­a­ra­tion the­o­rem, there is a sep­a­rat­ing hy­per­plane be­tween and . We can define by tak­ing a vec­tor nor­mal to the hy­per­plane and nor­mal­iz­ing it to sub to one. This is a prior for which is Bayes-op­ti­mal, es­tab­lish­ing the de­sired re­sult.

If this is con­fus­ing, I again sug­gest Peter Hoff’s course notes. How­ever, here is a sim­plified illus­tra­tion of the idea for two wor­lds, four pure ac­tions, and no ob­ser­va­tions:

(I used be­cause I am more com­fortable with think­ing of “good” as “up”, IE, think­ing in terms of util­ity rather than loss.)

The black “cor­ners” com­ing from and show the be­gin­ning of the Q() set for those two points. (You can imag­ine the other two, for and .) Noth­ing is pareto-dom­i­nated ex­cept for , which is dom­i­nated by ev­ery­thing. In eco­nomics ter­minol­ogy, the first three ac­tions are on the pareto fron­tier. In par­tic­u­lar, is not pareto-dom­i­nated. Put­ting some num­bers to it, could be worth (2,2), that is, worth two in each world. could be worth (1,10), and could be worth (10,1). There is no prior over the two wor­lds in which a Bayesian would want to take ac­tion . So, how do we rule it out through our ad­mis­si­bil­ity re­quire­ment? We add mixed strate­gies:

Now, there’s a new pareto fron­tier: the line stretch­ing be­tween and , con­sist­ing of strate­gies which have some prob­a­bil­ity of tak­ing those two ac­tions. Every­thing else is pareto-dom­i­nated. An agent who starts out con­sid­er­ing can see that mix­ing be­tween and is just a bet­ter idea, no mat­ter what world they’re in. This is the essence of the CCT ar­gu­ment.

Once we move to the pareto fron­tier of the set of mixed strate­gies, we can draw the sep­a­rat­ing hy­per­planes men­tioned in the proof:

(There may be a unique line, or sev­eral sep­a­rat­ing lines.) The sep­a­rat­ing hy­per­plane al­lows us to de­rive a (non-dog­matic) prior which the cho­sen de­ci­sion rule is con­sis­tent with.

Re­mov­ing Like­li­hoods (and other un­for­tu­nate as­sump­tions)

As­sum­ing the ex­is­tence of a like­li­hood func­tion is rather strange, if our goal is to ar­gue that agents should use prob­a­bil­ity and ex­pected util­ity to make de­ci­sions. A pur­ported de­ci­sion-the­o­retic foun­da­tion should not as­sume that an agent has any prob­a­bil­is­tic be­liefs to start out.

For­tu­nately, this is an ex­tremely easy mod­ifi­ca­tion of the ar­gu­ment: re­strict­ing to ei­ther be zero or one is just a spe­cial case of the ex­ist­ing the­o­rem. This does not limit our ex­pres­sive power. Pre­vi­ously, a world in which the true tem­per­a­ture is zero de­grees would have some prob­a­bil­ity of emit­ting the ob­ser­va­tion “the tem­per­a­ture is one de­gree”, due to ob­ser­va­tion er­ror. Now, we con­sider the er­ror a part of the world: there is a world where the true tem­per­a­ture is zero and the mea­sure­ment is one, as well as one where the true tem­per­a­ture is zero and the mea­sure­ment is zero, and so on.

Another re­lated con­cern is the as­sump­tion that we have mixed strate­gies, which are de­scribed via prob­a­bil­ities. Un­for­tu­nately, this is much more cen­tral to the ar­gu­ment, so we have to do a lot more work to re-state things in a way which doesn’t as­sume prob­a­bil­ities di­rectly. Bear with me—it’ll be a few para­graphs be­fore we’ve done enough work to elimi­nate the as­sump­tion that mixed strate­gies are de­scribed by prob­a­bil­ities.

It will be eas­ier to first get rid of the as­sump­tion that we have car­di­nal-val­ued loss . In­stead, as­sume that we have an or­di­nal prefer­ence for each world, . We then ap­ply the VNM the­o­rem within each , to get a car­di­nal-val­ued util­ity within each world. The CCT ar­gu­ment can then pro­ceed as usual.

Ap­ply­ing VNM is a lit­tle un­satis­fy­ing, since we need to as­sume the VNM ax­ioms about our prefer­ences. Hap­pily, it is easy to weaken the VNM ax­ioms, in­stead let­ting the as­sump­tions from the CCT set­ting do more work. A de­tailed write-up of the fol­low­ing is be­ing worked on, but to briefly sketch:

First, we can get rid of the in­de­pen­dence ax­iom. A mixed strat­egy is re­ally a strat­egy which in­volves ob­serv­ing coin-flips. We can put the coin-flips in­side the world (break­ing each into more sub-wor­lds in which coin-flips come out differ­ently). When we do this, the in­de­pen­dence ax­iom is a con­se­quence of ad­mis­si­bil­ity; any vi­o­la­tion of in­de­pen­dence can be un­done by a pareto im­prove­ment.

Se­cond, hav­ing made coin-flips ex­plicit, we can get rid of the ax­iom of con­ti­nu­ity. We ap­ply the VNM-like the­o­rem from the pa­per Ad­di­tive rep­re­sen­ta­tion of sep­a­rable prefer­ences over in­finite prod­ucts, by Mar­cus Pi­vato. This gives us car­di­nal-val­ued util­ity func­tions, but with­out the con­ti­nu­ity ax­iom, our util­ity may some­times be rep­re­sented by in­fini­ties. (Speci­fi­cally, we can con­sider sur­real-num­bered util­ity as the most gen­eral case.) You can as­sume this never hap­pens if it both­ers you.

More im­por­tantly, at this point we don’t need to as­sume that mixed strate­gies are rep­re­sented via pre-ex­ist­ing prob­a­bil­ities any­more. In­stead, they’re rep­re­sented by the coins.

I’m fairly happy with this re­sult, and apol­o­gize for the brief treat­ment. How­ever, let’s move on for now to the com­par­i­son to so­cial choice the­ory I promised.

Utilitarianism

I said that are “pos­si­ble world states” and that there is an “agent” who is “un­cer­tain about which world-state is the case”—how­ever, no­tice that I didn’t re­ally use any of that in the the­o­rem. What mat­ters is that for each , there is a prefer­ence re­la­tion on ac­tions. CCT is ac­tu­ally about com­pro­mis­ing be­tween differ­ent prefer­ence re­la­tions.

If we drop the ob­ser­va­tions, we can in­ter­pret the as peo­ple, and the as po­ten­tial col­lec­tive ac­tions. The are po­ten­tial so­cial choices, which are ad­mis­si­ble when they are pareto-effi­cient with re­spect to in­di­vi­d­ual’s prefer­ences.

Mak­ing the hy­per­plane ar­gu­ment as be­fore, we get a which places pos­i­tive weight on each in­di­vi­d­ual. This is in­ter­preted as each in­di­vi­d­ual’s weight in the coal­i­tion. The col­lec­tive de­ci­sion must be the re­sult of a (pos­i­tive) lin­ear com­bi­na­tion of each in­di­vi­d­ual’s car­di­nal util­ities—and those car­di­nal util­ities can in turn be con­structed via an ap­pli­ca­tion of VNM to in­di­vi­d­ual or­di­nal prefer­ences. This re­sult is very similar to Harsanyi’s util­i­tar­i­anism the­o­rem.

This is not only a nice ar­gu­ment for util­i­tar­i­anism, it is also an amus­ing math­e­mat­i­cal pun, since it puts util­i­tar­ian “so­cial util­ity” and de­ci­sion-the­o­retic “ex­pected util­ity” into the same math­e­mat­i­cal frame­work. Just be­cause both can be de­rived via pareto-op­ti­mal­ity ar­gu­ments doesn’t mean they’re nec­es­sar­ily the same thing, though.

Harsanyi’s the­o­rem is not the most-cited jus­tifi­ca­tion for util­i­tar­i­anism. One rea­son for this may be that it is “overly prag­matic”: util­i­tar­i­anism is about val­ues; Harsanyi’s the­o­rem is about co­her­ent gov­er­nance. Harsanyi’s the­o­rem re­lies on imag­in­ing a col­lec­tive de­ci­sion which has to com­pro­mise be­tween ev­ery­one’s val­ues, and speci­fies what it must be like. Utili­tar­i­ans don’t imag­ine such a global de­ci­sion can re­ally be made, but rather, are try­ing to spec­ify their own al­tru­is­tic val­ues. Nonethe­less, a similar ar­gu­ment ap­plies: al­tru­is­tic val­ues are enough of a “global de­ci­sion” that, hy­po­thet­i­cally, you’d want to run the Harsanyi ar­gu­ment if you had de­scrip­tions of ev­ery­one’s util­ity func­tions and if you ac­cepted pareto im­prove­ments. So there’s an ar­gu­ment to be made that that’s still what you want to ap­prox­i­mate.

Another rea­son, men­tioned by Jes­si­cata in the com­ments, is that util­i­tar­i­ans typ­i­cally value egal­i­tar­i­anism. Harsanyi’s the­o­rem only says that you must put some weight on each in­di­vi­d­ual, not that you have to be fair. I don’t think this is much of a prob­lem—just as CCT ar­gues for “some” prior, but re­al­is­tic agents have fur­ther con­sid­er­a­tions which make them skew to­wards max­i­mally spread out pri­ors, CCT in so­cial choice the­ory can tell us that we need some weights, and there can be ex­tra con­sid­er­a­tions which push us to­ward egal­i­tar­ian weights. Harsanyi’s the­o­rem is still a strong ar­gu­ment for a big chunk of the util­i­tar­ian po­si­tion.

Futarchy

Now, as promised, Critch’s ‘futarchy’ the­o­rem.

If we add ob­ser­va­tions back in to the multi-agent in­ter­pre­ta­tion, as­so­ci­ates each agent with a prob­a­bil­ity dis­tri­bu­tion on ob­ser­va­tions. This can be in­ter­preted as each agent’s be­liefs. In the pa­per Toward Ne­go­tiable Re­in­force­ment Learn­ing, Critch ex­am­ined pareto-op­ti­mal se­quen­tial de­ci­sion rules in this set­ting. Not only is there a func­tion which gives a weight for each agent in the coal­i­tion, but this is up­dated via Bayes’ Rule as ob­ser­va­tions come in. The in­ter­pre­ta­tion of this is that the agents in the coal­i­tion want to bet on their differ­ing be­liefs, so that agents who make more cor­rect bets gain more in­fluence over the de­ci­sions of the coal­i­tion.

This differs from Robin Han­son’s futarchy, whose motto “vote on val­ues, but bet be­liefs” sug­gests that ev­ery­one gets an equal vote—you lose money when you bet, which loses you in­fluence on im­ple­men­ta­tion of pub­lic policy, but you still get an equal share of value. How­ever, Critch’s anal­y­sis shows that Robin’s ver­sion can be strictly im­proved upon, re­sult­ing in Critch’s ver­sion. (Also, Critch is not propos­ing his solu­tion as a sys­tem of gov­er­nance, only as a no­tion of multi-ob­jec­tive learn­ing.) Nonethe­less, the spirit still seems similar to Futarchy, in that the con­trol of the sys­tem is dis­tributed based on bets.

If Critch’s sys­tem seems harsh, it is be­cause we wouldn’t re­ally want to bet away all our share of the col­lec­tive value, nor do we want to pun­ish those who would bet away all their value too severely. This sug­gests that we (a) just wouldn’t bet ev­ery­thing away, and so wouldn’t end up too badly off; and (b) would want to still take care of those who bet their own value away, so that the con­se­quences for those peo­ple would not ac­tu­ally be so harsh. Nonethe­less, we can also try to take the prob­lem more se­ri­ously and think about al­ter­na­tive for­mu­la­tions which seem less strik­ingly harsh.

Conclusion

One po­ten­tial re­search pro­gram which may arise from this is: take the anal­ogy be­tween so­cial choice the­ory and de­ci­sion the­ory very se­ri­ously. Look closely at more com­pli­cated mod­els of so­cial choice the­ory, in­clud­ing vot­ing the­ory and per­haps mechanism de­sign. Un­der­stand the struc­ture of ra­tio­nal col­lec­tive choice in de­tail. Then, try to port the les­sons from this back to the in­di­vi­d­ual-agent case, to cre­ate de­ci­sion the­o­ries more so­phis­ti­cated than sim­ple Bayes. Mir­ror­ing this on the four-quad­rant di­a­gram from early on:

And, if you squint at this di­a­gram, you can see the let­ters “CCT”.

(Clos­ing vi­sual pun by Cas­par Öster­held.)