Quick thoughts on empathic metaethics

Years ago, I wrote an un­finished se­quence of posts called “No-Non­sense Me­taethics.” My last post, Plu­ral­is­tic Mo­ral Re­duc­tion­ism, said I would next ex­plore “em­pathic metaethics,” but I never got around to writ­ing those posts. Re­cently, I wrote a high-level sum­mary of some ini­tial thoughts on “em­pathic metaethics” in sec­tion 6.1.2 of a re­port pre­pared for my em­ployer, the Open Philan­thropy Pro­ject. With my em­ployer’s per­mis­sion, I’ve adapted that sec­tion for pub­li­ca­tion here, so that it can serve as the long-over­due con­clud­ing post in my se­quence on metaethics.

In my pre­vi­ous post, I dis­t­in­guished “aus­tere metaethics” and “em­pathic metaethics,” where aus­tere metaethics con­fronts moral ques­tions roughly like this:

Tell me what you mean by ‘right’, and I will tell you what is the right thing to do. If by ‘right’ you mean X, then Y is the right thing to do. If by ‘right’ you mean P, then Z is the right thing to do. But if you can’t tell me what you mean by ‘right’, then you have failed to ask a co­her­ent ques­tion, and no one can an­swer an in­co­her­ent ques­tion.

Mean­while, em­pathic metaethics says in­stead:

You may not know what you mean by ‘right.’ But let’s not stop there. Here, let me come alongside you and help de­code the cog­ni­tive al­gorithms that gen­er­ated your ques­tion in the first place, and then we’ll be able to an­swer your ques­tion. Then we can tell you what the right thing to do is.

Below, I provide a high-level sum­mary of some of my ini­tial thoughts on what one ap­proach to “em­pathic metaethics” could look like.

Given my metaeth­i­cal ap­proach, when I make a “moral judg­ment” about some­thing (e.g. about which kinds of be­ings are moral pa­tients), I don’t con­ceive of my­self as per­ceiv­ing an ob­jec­tive moral truth, or com­ing to know an ob­jec­tive moral truth via a se­ries of ar­gu­ments. Nor do I con­ceive of my­self as merely ex­press­ing my moral feel­ings as they stand to­day. Rather, I con­ceive of my­self as mak­ing a con­di­tional fore­cast about what my val­ues would be if I un­der­went a cer­tain “ideal­iza­tion” or “ex­trap­o­la­tion” pro­ce­dure (com­ing to know more true facts, hav­ing more time to con­sider moral ar­gu­ments, etc.).[1]

Thus, in a (hy­po­thet­i­cal) “ex­treme effort” at­tempt to en­gage in em­pathic metaethics (for think­ing about my own moral judg­ments), I would do some­thing like the fol­low­ing:

  1. I would try to make the sce­nario I’m aiming to fore­cast as con­crete as pos­si­ble, so that my brain is able to treat it as a gen­uine fore­cast­ing challenge, akin to par­ti­ci­pat­ing in a pre­dic­tion mar­ket or fore­cast­ing tour­na­ment, rather than as a fan­tasy about which my brain feels “al­lowed” to make up what­ever story feels nice, or sig­nals my val­ues to oth­ers, or achieves some­thing else that isn’t fore­cast­ing ac­cu­racy.[2] In my case, I con­cretize the ex­trap­o­la­tion pro­ce­dure as one in­volv­ing a large pop­u­la­tion of copies of me who learn many true facts, con­sider many moral ar­gu­ments, and un­dergo var­i­ous other ex­pe­riences, and then col­lec­tively ad­vise me about what I should value and why.[3]

  2. How­ever, I would also try to make fore­casts I can ac­tu­ally check for ac­cu­racy, e.g. about what my moral judg­ment about var­i­ous cases will be 2 months in the fu­ture.

  3. When mak­ing these fore­casts, I would try to draw on the best re­search I’ve seen con­cern­ing how to make ac­cu­rate es­ti­mates and fore­casts. For ex­am­ple I would try to “think like a fox, not like a hedge­hog,” and I’ve already done sev­eral hours of prob­a­bil­ity cal­ibra­tion train­ing, and some amount of fore­cast­ing train­ing.[4]

  4. Clearly, my cur­rent moral in­tu­itions would serve as one im­por­tant source of ev­i­dence about what my ex­trap­o­lated val­ues might be. How­ever, re­cent find­ings in moral psy­chol­ogy and re­lated fields lead me to as­sign more ev­i­den­tial weight to some moral in­tu­itions than to oth­ers. More gen­er­ally, I in­ter­pret my cur­rent moral in­tu­itions as data gen­er­ated partly by my moral prin­ci­ples, and partly by var­i­ous “er­ror pro­cesses” (e.g. a hard-wired dis­gust re­ac­tion to spi­ders, which I don’t en­dorse upon re­flec­tion). Do­ing so al­lows me to make use of some stan­dard les­sons from statis­ti­cal curve-fit­ting when think­ing about how much ev­i­den­tial weight to as­sign to par­tic­u­lar moral in­tu­itions.[5]

  5. As part of fore­cast­ing what my ex­trap­o­lated val­ues might be, I would try to con­sider differ­ent pro­cesses and con­texts that could gen­er­ate al­ter­nate moral in­tu­itions in moral rea­son­ers both similar and dis­similar to my cur­rent self, and I would try to con­sider how I feel about the the “le­gi­t­i­macy” of those mechanisms as pro­duc­ers of moral in­tu­itions. For ex­am­ple I might ask my­self ques­tions such as “How might I feel about that prac­tice if I was born into a world in which it was already com­mon­place?” and “How might I feel about that case if my built-in (and largely un­con­scious) pro­cesses for as­so­ci­a­tive learn­ing and imi­ta­tive learn­ing had been ex­posed to differ­ent life his­to­ries than my own?” and “How might I feel about that case if I had been born in a differ­ent cen­tury, or a differ­ent coun­try, or with a greater propen­sity for clini­cal de­pres­sion?” and “How might a moral rea­soner on an­other planet feel about that case if it be­longed to a more strongly r-se­lected species (com­pared to hu­mans) but had roughly hu­man-like gen­eral rea­son­ing abil­ity?”[6]

  6. Ob­serv­able pat­terns in how peo­ple’s val­ues change (seem­ingly) in re­sponse to com­po­nents of my pro­posed ex­trap­o­la­tion pro­ce­dure (learn­ing more facts, con­sid­er­ing moral ar­gu­ments, etc.) would serve as an­other source of ev­i­dence about what my ex­trap­o­lated val­ues might be. For ex­am­ple, the cor­re­la­tion be­tween ag­gre­gate hu­man knowl­edge and our “ex­pand­ing cir­cle of moral con­cern” (Singer 2011) might (very weakly) sug­gest that, if I con­tinued to learn more true facts, my cir­cle of moral con­cern would con­tinue to ex­pand. Un­for­tu­nately, such cor­re­la­tions are badly con­founded, and might not provide much ev­i­dence.[7]

  7. Per­sonal facts about how my own val­ues have evolved as I’ve learned more, con­sid­ered moral ar­gu­ments, and so on, would serve as yet an­other source of ev­i­dence about what my ex­trap­o­lated val­ues might be. Of course, these re­la­tions are likely con­founded as well, and need to be in­ter­preted with care.[8]


1. This gen­eral ap­proach some­times goes by names such as “ideal ad­vi­sor the­ory” or, ar­guably, “re­flec­tive equil­ibrium.” Di­verse sources ex­pli­cat­ing var­i­ous ex­trap­o­la­tion pro­ce­dures (or frag­ments of ex­trap­o­la­tion pro­ce­dures) in­clude: Rosati (1995); Daniels (2016); Camp­bell (2013); chap­ter 9 of Miller (2013); Muehlhauser & Willi­am­son (2013); Trout (2014); Yud­kowsky’s “Ex­trap­o­lated vo­li­tion (nor­ma­tive moral the­ory)” (2016); Baker (2016); Stanovich (2004), pp. 224-275; Stanovich (2013).

2. For more on fore­cast­ing ac­cu­racy, see this blog post. My use of re­search on the psy­cholog­i­cal pre­dic­tors of fore­cast­ing ac­cu­racy for the pur­poses of do­ing moral philos­o­phy is one ex­am­ple of my sup­port for the use of “ame­lio­ra­tive psy­chol­ogy” in philo­soph­i­cal prac­tice — see e.g. Bishop & Trout (2004, 2008).

3. Speci­fi­cally, the sce­nario I try to imag­ine (and make con­di­tional fore­casts about) looks some­thing like this:

  1. In the dis­tant fu­ture, I am non-de­struc­tively “up­loaded.” In other words, my brain and some sup­port­ing cells are scanned (non-de­struc­tively) at a fine enough spa­tial and chem­i­cal re­s­olu­tion that, when this scan is com­bined with ac­cu­rate mod­els of how differ­ent cell types carry out their in­for­ma­tion-pro­cess­ing func­tions, one can cre­ate an ex­e­cutable com­puter model of my brain that matches my biolog­i­cal brain’s in­put-out­put be­hav­ior al­most ex­actly. This whole brain em­u­la­tion (“em”) is then con­nected to a vir­tual world: com­puted in­puts are fed to the em’s (now vir­tual) sig­nal trans­duc­tion neu­rons for sight, sound, etc., and com­puted out­puts from the em’s vir­tual arm move­ments, speech, etc. are re­ceived by the vir­tual world, which com­putes ap­pro­pri­ate changes to the vir­tual world in re­sponse. (I don’t think any­thing re­motely like this will ever hap­pen, but as far as I know it is a phys­i­cally pos­si­ble world that can be de­scribed in some de­tail; for one at­tempt, see Han­son 2016.) Given func­tion­al­ism, this “em” has the same mem­o­ries, per­son­al­ity, and con­scious ex­pe­rience that I have, though it ex­pe­riences quite a shock when it awak­ens to a vir­tual world that might look and feel some­what differ­ent from the “real” world.

  2. This ini­tial em is copied thou­sands of times. Some of the copies in­ter­act in­side the same vir­tual world, other copies are placed in­side iso­lated vir­tual wor­lds.

  3. Then, these ems spend a very long time (a) col­lect­ing and gen­er­at­ing ar­gu­ments and ev­i­dence about moral­ity and re­lated top­ics, (b) un­der­go­ing var­i­ous ex­pe­riences, in vary­ing or­ders, and re­flect­ing on those ex­pe­riences, (c) di­alogu­ing with ems sourced from other biolog­i­cal hu­mans who have differ­ent val­ues than I do, and per­haps with so­phis­ti­cated chat-bots meant to simu­late the plau­si­ble rea­son­ing of other types of peo­ple (from the past, or from other wor­lds) who were not available to be up­loaded, and so on. They are able to do these things for a very long time be­cause they and their vir­tual wor­lds are run at speeds thou­sands of times faster than my biolog­i­cal brain runs, al­low­ing sub­jec­tive eons to pass in mere months of “ob­jec­tive” time.

  4. Fi­nally, at some time, the ems di­alogue with each other about which val­ues seem “best,” they en­gage in moral trade (Ord 2015), and they try to ex­plain to me what val­ues they think I should have and why. In the end, I am not forced to ac­cept any of the val­ues they then hold (col­lec­tively or in­di­vi­d­u­ally), but I am able to come to much bet­ter-in­formed moral judg­ments than I could have with­out their in­put.

For more con­text on this sort of val­ues ex­trap­o­la­tion pro­ce­dure, see Muehlhauser & Willi­am­son (2013).

4. For more on fore­cast­ing “best prac­tices,” see this blog post.

5. Fol­low­ing Han­son (2002) and ch. 2 of Beck­stead (2013), I con­sider my moral in­tu­itions in the con­text of Bayesian curve-fit­ting. To ex­plain, I’ll quote Beck­stead (2013) at some length:

Curve fit­ting is a prob­lem fre­quently dis­cussed in the philos­o­phy of sci­ence. In the stan­dard pre­sen­ta­tion, a sci­en­tist is given some data points, usu­ally with an in­de­pen­dent vari­able and a de­pen­dent vari­able, and is asked to pre­dict the val­ues of the de­pen­dent vari­able given other val­ues of the in­de­pen­dent vari­able. Typ­i­cally, the data points are ob­ser­va­tions, such as “mea­sured height” on a scale or “re­ported in­come” on a sur­vey, rather than true val­ues, such as height or in­come. Thus, in mak­ing pre­dic­tions about ad­di­tional data points, the sci­en­tist has to ac­count for the pos­si­bil­ity of er­ror in the ob­ser­va­tions. By an er­ror pro­cess I mean any­thing that makes the ob­served val­ues of the data points differ from their true val­ues. Er­ror pro­cesses could arise from a faulty scale, failures of mem­ory on the part of sur­vey par­ti­ci­pants, bias on the part of the ex­per­i­menter, or any num­ber of other sources. While some treat­ments of this prob­lem fo­cus on pre­dict­ing ob­ser­va­tions (such as mea­sured height), I’m go­ing to fo­cus on pre­dict­ing the true val­ues (such as true height).
…For any con­sis­tent data set, it is pos­si­ble to con­struct a curve that fits the data ex­actly… If the sci­en­tist chooses one of these polyno­mial curves for pre­dic­tive pur­poses, the re­sult will usu­ally be overfit­ting, and the sci­en­tist will make worse pre­dic­tions than he would have if he had cho­sen a curve that did not fit the data as well, but had other virtues, such as a straight line. On the other hand, always go­ing with the sim­plest curve and giv­ing no weight to the data leads to un­der­fit­ting
I in­tend to carry over our think­ing about curve fit­ting in sci­ence to re­flec­tive equil­ibrium in moral philos­o­phy, so I should note im­me­di­ately that curve fit­ting is not limited to the case of two vari­ables. When we must un­der­stand re­la­tion­ships be­tween mul­ti­ple vari­ables, we can turn to mul­ti­ple-di­men­sional spaces and fit planes (or hy­per­planes) to our data points. Differ­ent axes might cor­re­spond to differ­ent con­sid­er­a­tions which seem rele­vant (such as to­tal well-be­ing, equal­ity, num­ber of peo­ple, fair­ness, etc.), and an­other axis could cor­re­spond to the value of the al­ter­na­tive, which we can as­sume is a func­tion of the rele­vant con­sid­er­a­tions. Direct Bayesian up­dat­ing on such data points would be im­prac­ti­cal, but the philo­soph­i­cal is­sues will not be af­fected by these difficul­ties.
…On a Bayesian ap­proach to this prob­lem, the sci­en­tist would con­sider a num­ber of differ­ent hy­pothe­ses about the re­la­tion­ship be­tween the two vari­ables, in­clud­ing both hy­pothe­ses about the phe­nom­ena (the re­la­tion­ship be­tween X and Y) and hy­pothe­ses about the er­ror pro­cess (the re­la­tion­ship be­tween ob­served val­ues of Y and true val­ues of Y) that pro­duces the ob­ser­va­tions…
…Les­sons from the Bayesian ap­proach to curve fit­ting ap­ply to moral philos­o­phy. Our moral in­tu­itions are the data, and there are er­ror pro­cesses that make our moral in­tu­itions de­vi­ate from the truth. The com­plete moral the­o­ries un­der con­sid­er­a­tion are the hy­pothe­ses about the phe­nom­ena. (Here, I use “the­ory” broadly to in­clude any com­plete set of pos­si­bil­ities about the moral truth. My use of the word “the­ory” does not as­sume that the truth about moral­ity is sim­ple, sys­tem­atic, and neat rather than com­plex, cir­cum­stan­tial, and messy.) If we ex­pect the er­ror pro­cesses to be wide­spread and sig­nifi­cant, we must rely on our pri­ors more. If we ex­pect the er­ror pro­cesses to be, in ad­di­tion, bi­ased and cor­re­lated, then we will have to rely sig­nifi­cantly on our pri­ors even when we have a lot of in­tu­itive data.

Beck­stead then sum­ma­rizes the frame­work with a table (p. 32), ed­ited to fit into LessWrong’s for­mat­ting:

  • Hy­pothe­ses about phenomena

    • (Science) Differ­ent tra­jec­to­ries of a ball that has been dropped

    • (Mo­ral Philos­o­phy) Mo­ral the­o­ries (spe­cific ver­sions of util­i­tar­i­anism, Kan­ti­anism, con­trac­tu­al­ism, plu­ral­is­tic de­on­tol­ogy, etc.)

  • Hy­pothe­ses about er­ror processes

    • (Science) Our po­si­tion mea­sure­ments are ac­cu­rate on av­er­age, and are within 1 inch 95% of the time (with nor­mally dis­tributed er­ror)

    • (Mo­ral Philos­o­phy) Differ­ent hy­pothe­ses about the causes of er­ror in his­tor­i­cal cases; cog­ni­tive and moral bi­ases; differ­ent hy­pothe­ses about the bi­ases that cause in­con­sis­tent judg­ments in im­por­tant philo­soph­i­cal cases

  • Observations

    • (Science) Recorded po­si­tion of a ball at differ­ent times recorded with a cer­tain clock

    • (Mo­ral Philos­o­phy) In­tu­itions about par­tic­u­lar cases or gen­eral prin­ci­ples, and any other rele­vant observations

  • Back­ground theory

    • (Science) The ball never bounces higher than the height it started at. The ball always moves along a con­tin­u­ous tra­jec­tory.

    • (Mo­ral Philos­o­phy) Meta-eth­i­cal or nor­ma­tive back­ground the­ory (or the­o­ries)

6. For more on this, see my con­ver­sa­tion with Carl Shul­man, O’Neill (2015), the liter­a­ture on the evolu­tion of moral val­ues (e.g. de Waal et al. 2014; Sin­nott-Arm­strong & Miller 2007; Joyce 2005), the liter­a­ture on moral psy­chol­ogy more gen­er­ally (e.g. Gra­ham et al. 2013; Doris 2010; Liao 2016; Christen et al. 2014; Sun­stein 2005), the liter­a­ture on how moral val­ues vary be­tween cul­tures and eras (e.g. see Flana­gan 2016; In­gle­hart & Welzel 2010; Pinker 2011; Mor­ris 2015; Fried­man 2005; Prinz 2007, pp. 187-195), and the liter­a­ture on moral thought ex­per­i­ments (e.g. Tit­tle 2004, ch. 7). See also Wil­son (2016)’s com­ments on in­ter­nal and ex­ter­nal val­idity in eth­i­cal thought ex­per­i­ments, and Bakker (2017) on “alien philos­o­phy.”

I do not read much fic­tion, but I sus­pect that some types of fic­tion — e.g. his­tor­i­cal fic­tion, fan­tasy, and sci­ence fic­tion — can help read­ers to tem­porar­ily trans­port them­selves into fully-re­al­ized al­ter­nate re­al­ities, in which read­ers can test how their moral in­tu­itions differ when they are tem­porar­ily “lost” in an al­ter­nate world.

7. There are many sources which dis­cuss how peo­ple’s val­ues seem to change along with (and per­haps in re­sponse to) com­po­nents of my pro­posed ex­trap­o­la­tion pro­ce­dure, such as learn­ing more facts, rea­son­ing through more moral ar­gu­ments, and di­alogu­ing with oth­ers who have differ­ent val­ues. See e.g. In­gle­hart & Welzel (2010), Pinker (2011), Sher­mer (2015), and Buchanan & Pow­ell (2016). See also the liter­a­tures on “en­light­ened prefer­ences” (Althaus 2003, chs. 4-6) and on “de­liber­a­tive pol­ling.”

8. For ex­am­ple, as I’ve learned more, con­sid­ered more moral ar­gu­ments, and di­alogued more with peo­ple who don’t share my val­ues, my moral val­ues have be­come more “sec­u­lar-ra­tio­nal” and “self-ex­pres­sive” (In­gle­hart & Welzel 2010), more ge­o­graph­i­cally global, more ex­ten­sive (e.g. through­out more of the an­i­mal king­dom), less per­son-af­fect­ing, and sub­ject to greater moral un­cer­tainty (Bykvist 2017).

No comments.