Total Utility is Illusionary

(Ab­stract: We have the no­tion that peo­ple can have a “to­tal util­ity” value, defined per­haps as the sum of all their changes in util­ity over time. This is usu­ally not a use­ful con­cept, be­cause util­ity func­tions can change. In many cases the less-con­fus­ing ap­proach is to look only at the util­ity from each in­di­vi­d­ual de­ci­sion, and not at­tempt to con­sider the to­tal over time. This leads to in­sights about util­i­tar­i­anism.)

Let’s con­sider the util­ity of a fel­low named Bob. Bob likes to track his to­tal util­ity; he writes it down in a log­book ev­ery night.

Bob is a stamp col­lec­tor; he gets +1 utilon ev­ery time he adds a stamp to his col­lec­tion, and he gets −1 utilon ev­ery time he re­moves a stamp from his col­lec­tion. Bob’s util­ity was zero when his col­lec­tion was empty, so we can say that Bob’s to­tal util­ity is the num­ber of stamps in his col­lec­tion.

One day a movie the­ater opens, and Bob learns that he likes go­ing to movies. Bob counts +10 utilons ev­ery time he sees a movie. Now we can say that Bob’s to­tal util­ity is the num­ber of stamps in his col­lec­tion, plus ten times the num­ber of movies he has seen.

(A note on ter­minol­ogy: I’m say­ing that Bob’s util­ity func­tion is the thing that emits +1 or −1 or +10, and his to­tal util­ity is the sum of all those emits over time. I’m not sure if this is stan­dard ter­minol­ogy.)

This should strike us as a lit­tle bit strange: Bob now has a term in his to­tal util­ity which is mostly based on his­tory, and mostly in­de­pen­dent of the pre­sent state of the world. Tech­ni­cally, we might hand­wave and say that Bob places value on his mem­o­ries of watch­ing those movies. But Bob knows that’s not ac­tu­ally true: it’s the act of watch­ing the movies that he en­joys, and he rarely thinks about them once they’re over.

If a hyp­no­tist con­vinced Bob that he had watched ten billion movies, Bob would write down in his log­book that he had a hun­dred billion utilons. (Plus the num­ber of stamps in his stamp col­lec­tion.)

Let’s talk some more about that stamp col­lec­tion. Bob wakes up on June 14 and de­cides that he doesn’t like stamps any more. Now, Bob gets −1 utilon ev­ery time he adds a stamp to his col­lec­tion, and +1 utilon ev­ery time he re­moves one. What can we say about his to­tal util­ity? We might say that Bob’s to­tal util­ity is the num­ber of stamps in his col­lec­tion at the start of June 14, plus ten times the num­ber of movies he’s watched, plus the num­ber of stamps he re­moved from his col­lec­tion af­ter June 14. Or we might say that all Bob’s util­ity from his stamp col­lec­tion prior to June 14 was false util­ity, and we should strike it from the record books. Which an­swer is bet­ter?

...Really, nei­ther an­swer is bet­ter, be­cause the “to­tal util­ity” num­ber we’re dis­cussing just isn’t very use­ful. Bob has a very clear util­ity func­tion which emits num­bers like +1 and +10 and −1; he doesn’t gain any­thing by keep­ing track of the to­tal sep­a­rately. His to­tal util­ity doesn’t seem to track how happy he ac­tu­ally feels, ei­ther. It’s not clear what Bob gains from think­ing about this to­tal util­ity num­ber.

I think some of the con­fu­sion might be com­ing from Less Wrong’s fo­cus on AI de­sign.

When you’re writ­ing a util­ity func­tion for an AI, one thing you might try is to spec­ify your util­ity func­tion by spec­i­fy­ing the to­tal util­ity first: you might say “your to­tal util­ity is the num­ber of balls you have placed in this bucket” and then let the AI work out the im­ple­men­ta­tion de­tails of how happy each in­di­vi­d­ual ac­tion makes it.

How­ever, if you’re look­ing at util­ity func­tions for ac­tual peo­ple, you might en­counter some­thing weird like “I get +10 util­ity ev­ery time I watch a movie”, or “I woke up to­day and my util­ity func­tion changed”, and then if you try to com­pute the to­tal util­ity for that per­son, you can get con­fused.

Let’s now talk about util­i­tar­i­anism. For sim­plic­ity, let’s as­sume we’re talk­ing about a util­i­tar­ian gov­ern­ment which is mak­ing de­ci­sions on be­half of its con­stituency. (In other words, we’re not talk­ing about util­i­tar­i­anism as a moral the­ory.)

We have the no­tion of to­tal util­i­tar­i­anism, in which the gov­ern­ment tries to max­i­mize the sum of the util­ity val­ues of each of its con­stituents. This leads to “re­pug­nant con­clu­sion” is­sues in which the gov­ern­ment gen­er­ates new con­stituents at a high rate un­til all of them are mis­er­able.

We also have the no­tion of av­er­age util­i­tar­i­anism, in which the gov­ern­ment tries to max­i­mize the av­er­age of the util­ity val­ues of each of its con­stituents. This leads to is­sues—I’m not sure if there’s a snappy name—where the gov­ern­ment tries to kill off the least happy con­stituents so as to bring the av­er­age up.

The prob­lem with both of these no­tions is that they’re tak­ing the no­tion of “to­tal util­ity of all con­stituents” as an in­put, and then they’re chang­ing the num­ber of con­stituents, which changes the un­der­ly­ing util­ity func­tion.

I think the right way to do util­i­tar­i­anism is to ig­nore the “to­tal util­ity” thing; that’s not a real num­ber any­way. In­stead, ev­ery time you ar­rive at a de­ci­sion point, eval­u­ate what ac­tion to take by check­ing the util­ity of your con­stituents from each ac­tion. I pro­pose that we call this “delta util­i­tar­i­anism”, be­cause it isn’t look­ing at the to­tal or the av­er­age, just at the delta in util­ity from each ac­tion.

This solves the “re­pug­nant con­clu­sion” is­sue be­cause, at the time when you’re con­sid­er­ing adding more peo­ple, it’s more clear that you’re con­sid­er­ing the util­ity of your con­stituents at that time, which does not in­clude the po­ten­tial new peo­ple.