A (small) critique of total utilitarianism

In to­tal util­i­tar­i­anism, it is a morally neu­tral act to kill some­one (in a painless and un­ex­pected man­ner) and cre­at­ing/​giv­ing birth to an­other be­ing of com­pa­rable hap­piness (or prefer­ence satis­fac­tion or welfare). In fact if one can kill a billion peo­ple to cre­ate a billion and one, one is morally com­pel­led to do so. And this is true for real peo­ple, not just thought ex­per­i­ment peo­ple—liv­ing peo­ple with dreams, as­pira­tions, grudges and an­noy­ing or en­dear­ing quirks. To avoid caus­ing ex­tra pain to those left be­hind, it is bet­ter that you kill off whole fam­i­lies and com­mu­ni­ties, so that no one is left to mourn the dead. In fact the most morally com­pel­ling act would be to kill off the whole of the hu­man species, and re­place it with a slightly larger pop­u­la­tion.

We have many real world analogues to this thought ex­per­i­ment. For in­stance, it seems that there is only a small differ­ence be­tween the hap­piness of richer na­tions and poorer na­tions, while the first con­sume many more re­sources than the sec­ond. Hence to in­crease util­ity we should sim­ply kill off all the rich, and let the poor mul­ti­ply to take their place (con­tinu­ally bump­ing off any of the poor that gets too rich). Of course, the rich world also pro­duces most of the farm­ing sur­plus and the tech­nol­ogy in­no­va­tion, which al­low us to sup­port a larger pop­u­la­tion. So we should aim to kill ev­ery­one in the rich world apart from farm­ers and sci­en­tists—and enough sup­port staff to keep these pro­fes­sions run­ning (Carl Shul­man cor­rectly points out that we may re­quire most of the rest of the econ­omy as “sup­port staff”. Still, it’s very likely that we could kill off a sig­nifi­cant seg­ment of the pop­u­la­tion—those with the high­est con­sump­tion rel­a­tive to their im­pact of farm­ing and sci­ence—and still “im­prove” the situ­a­tion).

Even if turns out to be prob­le­matic to im­ple­ment in prac­tice, a true to­tal util­i­tar­ian should be think­ing: “I re­ally, re­ally wish there was a way to do tar­geted kil­ling of many peo­ple in the USA, Europe and Ja­pan, large parts of Asia and Latin Amer­ica and some parts of Africa—it makes me sick to the stom­ach to think that I can’t do that!” Or maybe: “I re­ally re­ally wish I could make ev­ery­one much poorer with­out af­fect­ing the size of the econ­omy—I wake up at night with night­mare be­cause these peo­ple re­main above the poverty line!”

I won’t be­labour the point. I find those ac­tions per­son­ally re­pel­lent, and I be­lieve that nearly ev­ery­one finds them some­what re­pel­lent or at least did so at some point in their past. This doesn’t mean that it’s the wrong thing to do—af­ter all, the ac­cepted an­swer to the tor­ture vs dust speck dilemma feels in­tu­itively wrong, at least the first time. It does mean, how­ever, that there must be very strong coun­ter­vailing ar­gu­ments to bal­ance out this ini­tial re­pul­sion (maybe even a math­e­mat­i­cal the­o­rem). For with­out that… how to jus­tify all this kil­ling?

Hence for the rest of this post, I’ll be ar­gu­ing that to­tal util­i­tar­i­anism is built on a foun­da­tion of dust, and thus pro­vides no rea­son to go against your ini­tial in­tu­itive judge­ment in these prob­lems. The points will be:

  1. Bayesi­anism and the fact that you should fol­low a util­ity func­tion in no way com­pel you to­wards to­tal util­i­tar­i­anism. The similar­ity in names does not mean the con­cepts are on similarly rigor­ous foun­da­tions.

  2. To­tal util­i­tar­i­anism is nei­ther a sim­ple, nor an el­e­gant the­ory. In fact, it is un­der-defined and ar­bi­trary.

  3. The most com­pel­ling ar­gu­ment for to­tal util­i­tar­i­anism (ba­si­cally the one that es­tab­lishes the re­pug­nant con­clu­sion), is a very long chain of im­perfect rea­son­ing, so there is no rea­son for the con­clu­sion to be solid.

  4. Con­sid­er­ing the prefer­ences of non-ex­is­tent be­ings does not es­tab­lish to­tal util­i­tar­i­anism.

  5. When con­sid­er­ing com­pet­ing moral the­o­ries, to­tal util­i­tar­i­anism does not “win by de­fault” thanks to its large val­ues as the pop­u­la­tion in­creases.

  6. Pop­u­la­tion ethics is hard, just as nor­mal ethics is.

A util­ity func­tion does not com­pel to­tal (or av­er­age) utilitarianism

There are strong rea­sons to sus­pect that the best de­ci­sion pro­cess is one that max­imises ex­pected util­ity for a par­tic­u­lar util­ity func­tion. Any pro­cess that does not do so, leaves it­self open to be money pumped or taken ad­van­tage of. This point has been re­it­er­ated again and again on Less Wrong, and rightly so.

Your util­ity func­tion must be over states of the uni­verse—but that’s the only re­stric­tion. The the­o­rem says noth­ing fur­ther about the con­tent of your util­ity func­tion. If you pre­fer a world with a trillion ec­static su­per-hu­mans to one with a sep­til­lion sub­sis­tence farm­ers—or vice versa—then as long you max­imise your ex­pected util­ity, the money pumps can’t touch you, and the stan­dard Bayesian ar­gu­ments don’t in­fluence you to change your mind. Your val­ues are fully rigor­ous.

For in­stance, in the tor­ture vs dust speck sce­nario, av­er­age util­i­tar­i­anism also com­pels you to take tor­ture, as do a host of other pos­si­ble util­ity func­tions. A lot of ar­gu­ments around this sub­ject, that may im­plic­itly feel to be in favour of to­tal util­i­tar­i­anism, turn out to be noth­ing of the sort. For in­stance, avoid­ing scope in­sen­si­tivity does not com­pel you to to­tal util­i­tar­i­anism, and you can perfectly al­low birth-death asym­me­tries or similar in­tu­itions, while re­main­ing an ex­pected util­ity max­imiser.

To­tal util­i­tar­i­anism is not sim­ple nor el­e­gant, but is arbitrary

To­tal util­i­tar­i­anism is defined as max­imis­ing the sum of ev­ery­one’s in­di­vi­d­ual util­ity func­tion. That’s a sim­ple defi­ni­tion. But what are these in­di­vi­d­ual util­ity func­tions? Do peo­ple act like ex­pected util­ity max­imisers? In a word… no. In an­other word… NO. In yet an­other word… NO!

So what are these util­ities? Are they the util­ity that the in­di­vi­d­u­als “should have”? Ac­cord­ing to what and who’s crite­ria? Is it “welfare”? How is that defined? Is it hap­piness? Again, how is that defined? Is it prefer­ences? On what scale? And what if the in­di­vi­d­ual dis­agrees with the util­ity they are sup­posed to have? What if their re­vealed prefer­ences are differ­ent again?

There are (var­i­ous differ­ent) ways to start re­solv­ing these prob­lems, and philoso­phers have spent a lot of ink and time do­ing so. The point re­mains that to­tal util­i­tar­i­anism can­not claim to be a sim­ple the­ory, if the ob­jects that it sums over are so poorly and con­tro­ver­sially defined.

And the sum it­self is a huge prob­lem. There is no nat­u­ral scale on which to com­pare util­ity func­tions. Divide one util­ity func­tion by a billion, mul­ti­ply the other by eπ, and they are still perfectly valid util­ity func­tions. In a study group at the FHI, we’ve been look­ing at var­i­ous ways of com­bin­ing util­ity func­tions—equiv­a­lently, of do­ing in­ter­per­sonal util­ity com­par­i­sons (IUC). Turns out it’s very hard, there seems no nat­u­ral way of do­ing this, and a lot has also been writ­ten about this, con­clud­ing lit­tle. Un­less your the­ory comes with a par­tic­u­lar IUC method, the only way of sum­ming these util­ities is to do an es­sen­tially ar­bi­trary choice for each in­di­vi­d­ual be­fore sum­ming. Thus stan­dard to­tal util­i­tar­i­anism is an ar­bi­trary sum of ill defined, non-nat­u­ral ob­jects.

Why then is it so pop­u­lar? Well, one rea­son is that there are mod­els that make use of some­thing like to­tal util­i­tar­i­anism to great effect. Clas­si­cal eco­nomic the­ory, for in­stance, mod­els ev­ery­one as perfectly ra­tio­nal ex­pected util­ity max­imisers. It gives good pre­dic­tions—but it re­mains a model, with a do­main of val­idity. You wouldn’t con­clude from that eco­nomic model that, say, men­tal ill­nesses don’t ex­ist. Similarly, mod­el­ling each life as hav­ing the same value and max­imis­ing ex­pected lives saved is sen­si­ble and in­tu­itive in many sce­nar­ios—but not nec­es­sar­ily all.

Maybe if we had a bit more in­for­ma­tion about the af­fected pop­u­la­tions, we could use a more so­phis­ti­cated model, such as one in­cor­po­rat­ing qual­ity ad­justed life years (QALY). Or maybe we could let other fac­tors af­fect our think­ing—what if we had to choose be­tween sav­ing a pop­u­la­tion of 1000 ver­sus a pop­u­la­tion of 1001, of same av­er­age QALYs, but where the first set con­tained the en­tire Awá tribe/​cul­ture of 300 peo­ple, and the sec­ond is made up of rep­re­sen­ta­tives from much larger eth­nic groups, much more cul­turally re­place­able? Should we let that in­fluence our de­ci­sion? Well maybe we should, maybe we shouldn’t, but it would be wrong to say “well, I would re­ally like to save the Awá, but the model I set­tled on ear­lier won’t al­low me to, so I best fol­low the model”. The mod­els are there pre­cisely to model our moral in­tu­itions (the clue is in the name), not freeze them.

The re­pug­nant con­clu­sion is at the end of a flimsy chain

There is a seem­ingly sound ar­gu­ment for the re­pug­nant con­clu­sion, which goes some way to­wards mak­ing to­tal util­i­tar­i­anism plau­si­ble. It goes like this:

  1. Start with a pop­u­la­tion of very happy/​util­itied/​welfared/​prefer­ence satis­fied peo­ple.

  2. Add other peo­ple whose lives are worth liv­ing, but whose av­er­age “util­ity” is less than that of the ini­tial pop­u­la­tion.

  3. Redis­tribute “util­ity” in an egal­i­tar­ian way across the whole pop­u­la­tion, in­creas­ing the av­er­age a lit­tle as you do so (but mak­ing sure the top rank have their util­ity low­ered).

  4. Re­peat as of­ten as re­quired.

  5. End up with a huge pop­u­la­tion whose lives are barely worth liv­ing.

If all these steps in­crease the qual­ity of the out­come (and it seems in­tu­itively that they do), then the end state much be bet­ter than the start­ing state, agree­ing with to­tal util­i­tar­i­anism. So, what could go wrong with this rea­son­ing? Well, as seen be­fore, the term “util­ity” is very much un­defined, as is its scale—hence egal­i­tar­ian is ex­tremely un­defined. So this ar­gu­ment is not math­e­mat­i­cally pre­cise, its rigour is illu­sion­ary. And when you re­cast the ar­gu­ment in qual­i­ta­tive terms, as you must, it be­come much weaker.

Go­ing through the iter­a­tion, there will come a point when the hu­man world is go­ing to lose its last anime, its last opera, its last copy of the Lord of the Rings, its last math­e­mat­ics, its last on­line dis­cus­sion board, its last foot­ball game—any­thing that might cause more-than-ap­pro­pri­ate en­joy­ment. At that stage, would you be en­tirely sure that the loss was worth­while, in ex­change of a weakly defined “more equal” so­ciety? More to the point, would you be sure that when iter­at­ing this pro­cess billions of times, ev­ery re­dis­tri­bu­tion will be an im­prove­ment? This is a con­junc­tive state­ment, so you have to be nearly en­tirely cer­tain of ev­ery link in the chain, if you want to be­lieve the out­come. And, to re­it­er­ate, these links can­not be re­duced to sim­ple math­e­mat­i­cal state­ments—you have to be cer­tain that each step is qual­i­ta­tively bet­ter than the pre­vi­ous one.

And you also have to be cer­tain that your the­ory does not al­low path de­pen­dency. One can take the perfectly valid po­si­tion that “If there were an ex­ist­ing poorer pop­u­la­tion, then the right thing to do would be to re­dis­tribute wealth, and thus lose the last copy of Ak­ira. How­ever, cur­rently there is no ex­ist­ing poor pop­u­la­tion, hence I would op­pose it com­ing into be­ing, pre­cisely be­cause it would re­sult in the lose of Ak­ira.” You can re­ject this type of rea­son­ing, and a va­ri­ety of oth­ers that block the re­pug­nant con­clu­sion at some stage of the chain (the Stan­ford En­cy­clopae­dia of Philos­o­phy has a good en­try on the Repug­nant Con­clu­sion and the ar­gu­ments sur­round­ing it). But most rea­sons for do­ing so already pre-sup­pose to­tal util­i­tar­i­anism. In that case, you can­not use the above as an ar­gu­ment for your the­ory.

Hy­po­thet­i­cal be­ings have hy­po­thet­i­cal (and com­pli­cated) things say to you

There is an­other ma­jor strand of ar­gu­ment for to­tal util­i­tar­i­anism, which claims that we owe it to non-ex­is­tent be­ings to satisfy their prefer­ences, that they would pre­fer to ex­ist rather than re­main non-ex­is­tent, and hence we should bring them into ex­is­tence. How does this ar­gu­ment fare?

First of all, it should be em­pha­sised that one is free to ac­cept or re­ject that ar­gu­ment with­out any fear of in­con­sis­tency. If one main­tains that never-ex­is­tent be­ings have no rele­vant prefer­ences, then one will never stum­ble over a prob­lem. They don’t ex­ist, they can’t make de­ci­sions, they can’t con­tra­dict any­thing. In or­der to raise them to the point where their de­ci­sions are rele­vant, one has to raise them to ex­is­tence, in re­al­ity or in simu­la­tion. By the time they can an­swer “would you like to ex­ist?”, they already do, so you are talk­ing about whether or not to kill them, not whether or not to let them ex­ist.

But sec­ondly, it seems that the “non-ex­is­tent be­ings” ar­gu­ment is of­ten ad­vanced for the sole pur­pose of ar­gu­ing for to­tal util­i­tar­i­anism, rather than as a defen­si­ble po­si­tion in its own right. Rarely are its im­pli­ca­tion analysed. What would a proper the­ory of non-ex­is­tent be­ings look like?

Well, for a start the whole hap­piness/​util­ity/​prefer­ence prob­lem comes back with ex­tra sting. It’s hard enough to make a util­ity func­tion out of real world peo­ple, but how to do so with hy­po­thet­i­cal peo­ple? Is it an es­sen­tially ar­bi­trary pro­cess (de­pen­dent en­tirely on “which types of peo­ple we think of first”), or is it done prop­erly, teas­ing out the “choices” and “life ex­pe­riences” of the hy­po­thet­i­cals? In that last case, if we do it in too much de­tail, we could ar­gue that we’ve already cre­ated the be­ing in simu­la­tion, so it comes back to the death is­sue.

But imag­ine that we’ve some­how ex­tracted a util­ity func­tion from the prefer­ences of non-ex­is­tent be­ings. Ap­par­ently, they would pre­fer to ex­ist rather than not ex­ist. But is this true? There are many peo­ple in the world who would pre­fer not to com­mit suicide, but would not mind much if ex­ter­nal events ended their lives—they cling to life as a habit. Pre­sum­ably non-ex­is­tent ver­sions of them “would not mind” re­main­ing non-ex­is­tent.

Even for those that would pre­fer to ex­ist, we can ask ques­tions about the in­ten­sity of that de­sire, and how it com­pares with their other de­sires. For in­stance, among these hy­po­thet­i­cal be­ings, some would be moth­ers of hy­po­thet­i­cal in­fants, lead­ers of hy­po­thet­i­cal re­li­gions, in­mates of hy­po­thet­i­cal pris­ons, and would only pre­fer to ex­ist if they could bring/​couldn’t bring the rest of their hy­po­thet­i­cal world with them. But this is ridicu­lous—we can’t bring the hy­po­thet­i­cal world with them, they would grow up in ours—so are we only re­ally talk­ing about the prefer­ences of hy­po­thet­i­cal ba­bies, or hy­po­thet­i­cal (and non-con­scious) foe­tuses?

If we do look at adults, brack­et­ing the is­sue above, then we get some that would pre­fer that they not ex­ist, as long as cer­tain oth­ers do—or con­versely that they not ex­ist, as long as oth­ers also not ex­ist. How should we take that into ac­count? As­sum­ing the uni­verse in­finite, any hy­po­thet­i­cal be­ing would ex­ist some­where. Is mere ex­is­tence enough, or do we have to have a large mea­sure or den­sity of ex­is­tence? Do we need them to ex­ist close to us? Are their own prefer­ences rele­vant—ie we only have a duty to bring into the world, those be­ings that would de­sire to ex­ist in mul­ti­ple copies ev­ery­where? Or do we feel these have already “enough ex­is­tence” and se­lect the un­der-counted be­ings? What if very few hy­po­thet­i­cal be­ings are to­tal util­i­tar­i­ans—is that rele­vant?

On a more per­sonal note, ev­ery time we make a de­ci­sion, we elimi­nate a par­tic­u­lar be­ing. We can not longer be the per­son who took the other job offer, or read the other book at that time and place. As these differ­ences ac­cu­mu­late, we di­verge quite a bit from what we could have been. When we do so, do we feel that we’re kil­ling off these ex­tra hy­po­thet­i­cal be­ings? Why not? Should we be com­pel­led to lead dou­ble lives, as­sum­ing two (or more) com­pletely sep­a­rate iden­tities, to in­crease the num­ber of be­ings in the world? If not, why not?

Th­ese are some of the ques­tions that a the­ory of non-ex­is­tent be­ings would have to grap­ple with, be­fore it can be­come an “ob­vi­ous” ar­gu­ment for to­tal util­i­tar­i­anism.

Mo­ral un­cer­tainty: to­tal util­i­tar­i­anism doesn’t win by default

An ar­gu­ment that I have met oc­ca­sion­ally is that while other eth­i­cal the­o­ries such as av­er­age util­i­tar­i­anism, birth-death asym­me­try, path de­pen­dence, prefer­ences of non-loss of cul­ture, etc… may have some val­idity, to­tal util­i­tar­i­anism wins as the pop­u­la­tion in­creases be­cause the oth­ers don’t scale in the same way. By the time we reach the trillion trillion trillion mark, to­tal util­i­tar­i­anism will com­pletely dom­i­nate, even if we gave it lit­tle weight at the be­gin­ning.

But this is the wrong way to com­pare com­pet­ing moral the­o­ries. Just as differ­ent peo­ple’s util­ities don’t have a com­mon scale, differ­ent moral util­ities don’t have a com­mon scale. For in­stance, would you say that square-to­tal util­i­tar­i­anism is cer­tainly wrong? This the­ory is sim­ply to­tal util­i­tar­i­anism fur­ther mul­ti­plied by the pop­u­la­tion; it would cor­re­spond roughly to the num­ber of con­nec­tions be­tween peo­ple. Or what about ex­po­nen­tial-square-to­tal util­i­tar­i­anism? This would cor­re­spond roughly to the set of pos­si­ble con­nec­tions be­tween peo­ple. As long as we think that ex­po­nen­tial-square-to­tal util­i­tar­i­anism is not cer­tainly com­pletely wrong, then the same ar­gu­ment as above would show it quickly dom­i­nat­ing as pop­u­la­tion in­creases.

Or what about 3^^^3 av­er­age util­i­tar­i­anism—which is sim­ply av­er­age util­i­tar­i­anism, mul­ti­plied by 3^^^3? Ob­vi­ously that ex­am­ple is silly—we know that rescal­ing shouldn’t change any­thing about the the­ory. But similarly, di­vid­ing to­tal util­i­tar­i­anism by 3^^^3 shouldn’t change any­thing, so to­tal util­i­tar­i­anism’s scal­ing ad­van­tage is illu­sory.

As men­tioned be­fore, com­par­ing differ­ent util­ity func­tions is a hard and sub­tle pro­cess. One method that seems to have sur­pris­ingly nice prop­er­ties (to such an ex­tent that I recom­mend always us­ing as a first try) is to nor­mal­ise the low­est pos­si­ble at­tain­able util­ity to zero, the high­est at­tain­able util­ity to one, mul­ti­ply by the weight you give to the the­ory, and then add the nor­mal­ised util­ities to­gether.

For in­stance, as­sume you equally val­ued av­er­age util­i­tar­i­anism and to­tal util­i­tar­i­anism, giv­ing them both weights of one (and you had solved all the defi­ni­tional prob­lems above). Among the choices you were fac­ing, the worst out­come for both the­o­ries is an empty world. The best out­come for av­er­age util­i­tar­i­anism would be ten peo­ple with an av­er­age “util­ity” of 100. The best out­come for to­tal util­i­tar­i­anism would be a quadrillion peo­ple with an av­er­age “util­ity” of 1. Then how would ei­ther of those com­pare to ten trillion peo­ple with an av­er­age util­ity of 60? Well, the nor­mal­ised util­ity of this for the av­er­age util­i­tar­ian is 0.6, while for the to­tal util­i­tar­ian it’s also 60/​100=0.6, and 0.6+0.6=1.2. This is bet­ter that the util­ity for the small world (1+10-9) or the large world (0.01+1), so it beats ei­ther of the ex­tremal choices.

Ex­tend­ing this method, we can bring in such the­o­ries as ex­po­nen­tial-square-to­tal util­i­tar­i­anism (prob­a­bly with small weights!), with­out need­ing to fear that it will swamp all other moral the­o­ries. And with this nor­mal­i­sa­tion (or similar ones), even small weights to moral the­o­ries such as “cul­ture has some in­trin­sic value” will of­ten pre­vent to­tal util­i­tar­i­anism from walk­ing away with all of the mar­bles.

(Pop­u­la­tion) ethics is still hard

What is the con­clu­sion? At Less Wrong, we’re used to re­al­is­ing that ethics is hard, that value is frag­ile, that there is no sin­gle easy moral the­ory to safely pro­gram the AI with. But it seemed for a while that pop­u­la­tion ethics might be differ­ent—that there may be nat­u­ral and easy ways to de­ter­mine what to do with large groups, even though we couldn’t de­cide what to do with in­di­vi­d­u­als. I’ve ar­gued strongly here that it’s not the case—that pop­u­la­tion ethics re­main hard, that we have to figure out what the­ory we want to have with­out ac­cess to easy short­cuts.

But in an­other way it’s liber­at­ing. To those who are mainly to­tal util­i­tar­i­ans but in­ter­nally doubt that a world with in­finitely many barely happy peo­ple sur­rounded by noth­ing but “muzak and pota­toes” is re­ally among the best of the best—well, you don’t have to con­vince your­self of that. You may choose to be­lieve it, or you may choose not to. No voice in the sky or in the math will force you ei­ther way. You can start putting to­gether a moral the­ory that in­cor­po­rates all your moral in­tu­itions—those that drove you to to­tal util­i­tar­i­anism, and those that don’t quite fit in that frame­work.