Morality as Parfitian-filtered Decision Theory?

Non-poli­ti­cal fol­low-up to: Un­grate­ful Hitch­hik­ers (offsite)

Re­lated to: Prices or Bind­ings?, The True Pri­soner’s Dilemma

Sum­mary: Si­tu­a­tions like the Parfit’s Hitch­hiker prob­lem se­lect for a cer­tain kind of mind: speci­fi­cally, one that rec­og­nizes that an ac­tion can be op­ti­mal, in a self-in­ter­ested sense, even if it can no longer cause any fu­ture benefit. A mind that can iden­tify such ac­tions might put them in a differ­ent cat­e­gory which en­ables it to perform them, in defi­ance of the (fu­ture­ward) con­se­quen­tial­ist con­cerns that nor­mally need to mo­ti­vate it. Our evolu­tion­ary his­tory has put us through such “Parfi­tian filters”, and the cor­re­spond­ing ac­tions, viewed from the in­side, feel like “some­thing we should do”, even if we don’t do it, and even if we rec­og­nize the lack of a fu­ture benefit. Therein lies the ori­gin of our moral in­tu­itions, as well as the ba­sis for cre­at­ing the cat­e­gory “moral­ity” in the first place.

In­tro­duc­tion: What kind of mind sur­vives Parfit’s Dilemma?

Parfit’s Dilemma – my ver­sion – goes like this: You are lost in the desert and near death. A su­per­be­ing known as Omega finds you and con­sid­ers whether to take you back to civ­i­liza­tion and sta­bi­lize you. It is a perfect pre­dic­tor of what you will do, and only plans to res­cue you if it pre­dicts that you will, upon re­cov­er­ing, give it $0.01 from your bank ac­count. If it doesn’t pre­dict you’ll pay, you’re left in the desert to die. [1]

So what kind of mind wakes up from this? One that would give Omega the money. Most im­por­tantly, the mind is not con­vinced to with­hold pay­ment on the ba­sis that the benefit was re­ceived only in the past. Even if it rec­og­nizes that no fu­ture benefit will re­sult from this de­ci­sion—and only fu­ture costs will re­sult—it de­cides to make the pay­ment any­way.

If a mind is likely to en­counter such dilem­mas, it would be an ad­van­tage to have a de­ci­sion the­ory ca­pa­ble of mak­ing this kind of “un-con­se­quen­tial­ist” de­ci­sion. And if a de­ci­sion the­ory passes through time by be­ing loss­ily stored by a self-repli­cat­ing gene (and some de­com­press­ing ap­para­tus), then only those that shift to en­cod­ing this kind of men­tal­ity will be ca­pa­ble of prop­a­gat­ing them­selves through Parfit’s Hitch­hiker-like sce­nar­ios (call these sce­nar­ios “Parfi­tian filters”).

Sus­tain­able self-repli­ca­tion as a Parfi­tian filter

Though evolu­tion­ary psy­chol­ogy has its share of pit­falls, one ques­tion should have an un­con­tro­ver­sial solu­tion: “Why do par­ents care for their chil­dren, usu­ally at great cost to them­selves?” The an­swer is that their de­sires are largely set by evolu­tion­ary pro­cesses, in which a “blueprint” is slightly mod­ified over time, and the more effec­tive self-repli­cat­ing blueprint-pieces dom­i­nate the con­struc­tion of liv­ing things. Par­ents that did not have suffi­cient “built-in de­sire” to care for their chil­dren would be weeded out; what’s left is (genes that con­struct) minds that do have such a de­sire.

This pro­cess can be viewed as a Parfi­tian filter: re­gard­less of how much par­ents might fa­vor their own sur­vival and satis­fac­tion, they could not get to that point un­less they were “at­tached” to a de­ci­sion the­ory that out­puts ac­tions suffi­ciently more fa­vor­able to­ward one’s chil­dren than one’s self. Ad­den­dum (per pjeby’s com­ment): The par­allel to Parfit’s Hitch­hiker is this: Nat­u­ral se­lec­tion is the Omega, and the mind prop­a­gated through gen­er­a­tions by nat­u­ral se­lec­tion is the hitch­hiker. The mind only gets to the “de­cide to pay”/​”de­cide to care for chil­dren” if it had the right de­ci­sion the­ory be­fore the “res­cue”/​”copy to next gen­er­a­tion”.

Ex­plana­tory value of util­ity functions

Let us turn back to Parfit’s Dilemma, an ideal­ized ex­am­ple of a Parfi­tian filter, and con­sider the task of ex­plain­ing why some­one de­cided to pay Omega. For sim­plic­ity, we’ll limit our­selves to two the­o­ries:

The­ory 1a: The sur­vivor’s util­ity func­tion places pos­i­tive weight on benefits both to the sur­vivor and to Omega; in this case, the util­ity of “Omega re­ceiv­ing the $0.01” (as viewed by the sur­vivor’s func­tion) ex­ceeds the util­ity of keep­ing it.

The­ory 1b: The sur­vivor’s util­ity func­tion only places weight on benefits to him/​her­self; how­ever, the sur­vivor is limited to us­ing de­ci­sion the­o­ries ca­pa­ble of sur­viv­ing this Parfi­tian filter.

The the­o­ries are ob­ser­va­tion­ally equiv­a­lent, but 1a is worse be­cause it makes strictly more as­sump­tions: in par­tic­u­lar, the ques­tion­able one that the sur­vivor some­how val­ues Omega in some ter­mi­nal, rather than in­stru­men­tal sense. [2] The same anal­y­sis can be car­ried over to the ear­lier ques­tion about nat­u­ral se­lec­tion, albeit dis­turbingly. Con­sider these two analo­gous the­o­ries at­tempt­ing to ex­plain the be­hav­ior of par­ents:

The­ory 2a: Par­ents have a util­ity func­tion that places pos­i­tive weight on both them­selves and their chil­dren.

The­ory 2b: Par­ents have a util­ity func­tion that places pos­i­tive weight on only them­selves (!!!); how­ever, they are limited to im­ple­ment­ing de­ci­sion the­o­ries ca­pa­ble of sur­viv­ing nat­u­ral se­lec­tion.

The point here is not to pro­mote some cyn­i­cal, in­sult­ing view of par­ents; rather, I will show how this “acausal self-in­ter­est” so closely al­igns with the be­hav­ior we laud as moral.

SAMELs vs. CaMELs, Mo­ral­ity vs. Selfish­ness

So what makes an is­sue be­long in the “moral­ity” cat­e­gory in the first place? For ex­am­ple, the de­ci­sion of which ice cream fla­vor to choose is not re­garded as a moral dilemma. (Call this Dilemma A.) How do you turn it into a moral dilemma? One way is to make the de­ci­sion have im­pli­ca­tions for the well-be­ing of oth­ers: “Should you eat your fa­vorite ice cream fla­vor, in­stead of your next-fa­vorite, if do­ing so short­ens the life of an­other per­son?” (Call this Dilemma B.)

De­ci­sion-the­o­ret­i­cally, what is the differ­ence be­tween A and B? Fol­low­ing Gary Drescher’s treat­ment in Chap­ter 7 of Good and Real, I see an­other salient differ­ence: You can reach the op­ti­mal de­ci­sion in A by look­ing only at causal means-end links (CaMELs), while Dilemma B re­quires that you con­sider the sub­junc­tive acausal means-end links (SAMELs). Less jar­gonishly, in Dilemma B, an ideal agent will rec­og­nize that their de­ci­sion to pick their fa­vorite ice cream at the ex­pense of an­other per­son sug­gests that oth­ers in the same po­si­tion will do (and have done) like­wise, for the same rea­son. In con­trast, an agent in Dilemma A (as stated) will do no worse as a re­sult of ig­nor­ing all such en­tail­ments.

More for­mally, a SAMEL is a re­la­tion­ship be­tween your choice and the satis­fac­tion of a goal, in which your choice does not (fu­ture­wardly) cause the goal’s achieve­ment or failure, while in a CaMEL, it does. Drescher ar­gues that ac­tions that im­plic­itly rec­og­nize SAMELs tend to be called “eth­i­cal”, while those that only rec­og­nize CaMELs tend to be called “self­ish”. I will show how these dis­tinc­tions (be­tween causal and acausal, eth­i­cal and un­eth­i­cal) shed light on moral dilem­mas, and on how we re­spond to them, by look­ing at some fa­mil­iar ar­gu­ments.

Joshua Greene, Re­vis­ited: When ra­tio­nal­iz­ing wins

A while back, LW read­ers dis­cussed Greene’s dis­ser­ta­tion on moral­ity. In it, he re­views ex­per­i­ments in which peo­ple are given moral dilem­mas and asked to jus­tify their po­si­tion. The twist: nor­mally peo­ple jus­tify their po­si­tion by refer­ence to some con­se­quence, but that con­se­quence is care­fully re­moved from be­ing a pos­si­bil­ity in the dilemma’s set-up. The re­sult? The sub­jects con­tinued to ar­gue for their po­si­tion, in­vok­ing such stop­signs as, “I don’t know, I can’t ex­plain it, [sic] I just know it’s wrong” (p. 151, cit­ing Haidt).

Greene re­gards this as mis­guided rea­son­ing, and in­ter­prets it to mean that peo­ple are ir­ra­tionally mak­ing choices, ex­ces­sively rely­ing on poor in­tu­itions. He in­fers that we need to fun­da­men­tally change how we think and talk about moral is­sues so as to elimi­nate these ques­tion­able bar­ri­ers in our rea­son­ing.

In light of Parfi­tian filters and SAMELs, I think a differ­ent in­fer­ence is available to us. First, re­call that there are cases where the best choices don’t cause a fu­ture benefit. In those cases, an agent will not be able to log­i­cally point to such a benefit as jus­tifi­ca­tion, even de­spite the choice’s op­ti­mal­ity. Fur­ther­more, if an agent’s de­ci­sion the­ory was formed through evolu­tion, their propen­sity to act on SAMELs (se­lected for due to its op­ti­mal­ity) arose long be­fore they were ca­pa­ble of care­ful self-re­flec­tive anal­y­sis of their choices. This, too, can ac­count for why most peo­ple a) opt for some­thing that doesn’t cause a fu­ture benefit, b) stick to that choice with or with­out such a benefit, and c) place it in a spe­cial cat­e­gory (“moral­ity”) when jus­tify­ing their ac­tion.

This does not mean we should give up on ra­tio­nally ground­ing our de­ci­sion the­ory, “be­cause ra­tio­nal­iz­ers win too!” Nor does it mean that ev­ery­one who re­treats to a “moral prin­ci­ples” defense is re­ally act­ing op­ti­mally. Rather, it means it is far too strict to re­quire that our de­ci­sions all cause a fu­ture benefit; we need to count acausal “con­se­quences” (SAMELs) on par with causal ones (CaMELs) – and moral in­tu­itions are a mechanism that can make us do this.

As Drescher notes, the op­ti­mal­ity of such acausal benefits can be felt, in­tu­itively, when mak­ing a de­ci­sion, even if they are in­suffi­cient to over­ride other de­sires, and even if we don’t rec­og­nize it in those ex­act terms (pp. 318-9):

Both the one-box in­tu­ition in New­comb’s Prob­lem (an in­tu­ition you can feel … even if you ul­ti­mately de­cide to take both boxes), and in­cli­na­tions to­ward al­tru­is­tic … be­hav­ior (in­cli­na­tions you like­wise can feel even if you end up be­hav­ing oth­er­wise), in­volve what I have ar­gued are acausal means-end re­la­tions. Although we do not … ex­plic­itly re­gard the links as means-end re­la­tions, as a prac­ti­cal mat­ter we do tend to treat them ex­actly as only means-end re­la­tions should be treated: our recog­ni­tion of the re­la­tion be­tween the ac­tion and the goal in­fluences us to take the ac­tion (even if con­trary in­fluences some­times pre­vail).

I spec­u­late that it is not co­in­ci­den­tal that in prac­tice, we treat these means-end re­la­tions as what they re­ally are. Rather, I sus­pect that the prac­ti­cal recog­ni­tion of means-end re­la­tions is fun­da­men­tal to our cog­ni­tive ma­chin­ery: it treats means-end re­la­tions (causal and acausal) as such be­cause do­ing so is cor­rect – that is, be­cause nat­u­ral se­lec­tion fa­vored ma­chin­ery that cor­rectly rec­og­nizes and acts on means-end re­la­tions with­out in­sist­ing that they be causal….

If we do not ex­plic­itly con­strue those moral in­tu­itions as recog­ni­tions of sub­junc­tive means-end links, we tend in­stead to per­ceive the in­tu­itions as recog­ni­tions of some oth­er­wise-un­grounded in­her­ent de­served­ness by oth­ers of be­ing treated well (or, in the case of re­tri­bu­tion, of be­ing treated badly).

To this we can add the Parfit’s Hitch­hiker prob­lem: how do you feel, in­ter­nally, about not pay­ing Omega? One could just as eas­ily crit­i­cize your de­sire to pay Omega as “ra­tio­nal­iza­tion”, as you can­not iden­tify a fu­ture benefit caused by your ac­tion. But the prob­lem, if any, lies in failing to rec­og­nize acausal benefits, not in your de­sire to pay.

The Pri­soner’s Dilemma, Re­vis­ited: Self-sac­ri­fi­cial car­ing is (some­times) self-optimizing

In this light, con­sider the Pri­soner’s Dilemma. Ba­si­cally, you and your part­ner-in-crime are de­cid­ing whether to rat each other out; the sum of the benefit to you both is high­est if you stay silent, but one can do bet­ter at the cost of the other by con­fess­ing. (La­bel this sce­nario that is used to teach it as the “Literal Pri­soner’s Dilemma Si­tu­a­tion”, or LPDS.)

Eliezer Yud­kowsky pre­vi­ously claimed in The True Pri­soner’s Dilemma that men­tion­ing the LPDS in­tro­duces a ma­jor con­fu­sion (and I agreed): real peo­ple in that situ­a­tion do not, in­tu­itively, see the pay­off ma­trix as it’s pre­sented. To most of us our satis­fac­tion with the out­come is not solely a func­tion of how much jail time we avoid: we also care about the other per­son, and don’t want to be a back­stab­ber. So, the ar­gu­ment goes, we need a re­ally con­trived situ­a­tion to get a pay­off ma­trix like that.

I sug­gest an al­ter­nate in­ter­pre­ta­tion of this dis­con­nect: the pay­off ma­trix is cor­rect, but the hu­mans fac­ing the dilemma have been Parfi­tian-filtered to the point where their de­ci­sion the­ory con­tains dis­po­si­tions that as­sist them in win­ning on these prob­lems, even given that pay­off ma­trix. To see why, con­sider an­other set of the­o­ries to choose from, like the two above:

The­ory 3a: Hu­mans in a literal Pri­soner’s Dilemma (LPDS) have a pos­i­tive weight in their util­ity func­tion both for them­selves, and their ac­com­plices, and so would be hurt to see the other one suffer jail time.

The­ory 3b: Hu­mans in a literal Pri­soner’s Dilemma (LPDS) have a pos­i­tive weight in their util­ity func­tion only for them­selves, but are limited to us­ing a de­ci­sion the­ory that sur­vived past so­cial/​biolog­i­cal Parfi­tian filters.

As with the point about par­ents, the les­son is not that you don’t care about your friends; rather, it’s that your ac­tions based on car­ing are the same as that of a self-in­ter­ested be­ing with a good de­ci­sion the­ory. What you rec­og­nize as “just wrong” could be the feel­ing of a differ­ent “rea­son­ing mod­ule” act­ing.


By view­ing moral in­tu­itions as mechanism that al­lows prop­a­ga­tion through Parfi­tian filters, we can bet­ter un­der­stand:

1) what moral in­tu­itions are (the set of in­tu­itions that were se­lected for be­cause they saw op­ti­mal­ity in the ab­sence of a causal link);

2) why they arose (be­cause agents with them pass through the Parfi­tian filters that weed out oth­ers, evolu­tion be­ing one of them); and

3) why we view this as a rele­vant cat­e­gory bound­ary in the first place (be­cause they are all similar in that they ele­vate the per­ceived benefit of an ac­tion that lacks a self-serv­ing, causal benefit).


[1] My var­i­ant differs in that there is no com­mu­ni­ca­tion be­tween you and Omega other than knowl­edge of your con­di­tional be­hav­iors, and the price is ab­surdly low to make sure the rele­vant in­tu­itions in your mind are firing.

[2] Note that 1b’s as­sump­tion of con­straints on the agent’s de­ci­sion the­ory does not pe­nal­ize it, as this must be as­sumed in both cases, and ad­di­tional im­pli­ca­tions of ex­ist­ing as­sump­tions do not count as ad­di­tional as­sump­tions for pur­poses of gaug­ing prob­a­bil­ities.