Not for the Sake of Pleasure Alone

Re­lated: Not for the Sake of Hap­piness (Alone), Value is Frag­ile, Fake Fake Utility Func­tions, You can­not be mis­taken about (not) want­ing to wire­head, Utilons vs. He­dons, Are wire­heads happy?

When some­one tells me that all hu­man ac­tion is mo­ti­vated by the de­sire for plea­sure, or that we can solve the Friendly AI prob­lem by pro­gram­ming a ma­chine su­per­in­tel­li­gence to max­i­mize plea­sure, I use a two-step ar­gu­ment to per­suade them that things are more com­pli­cated than that.

First, I pre­sent them with a vari­a­tion on Noz­ick’s ex­pe­rience ma­chine,1 some­thing like this:

Sup­pose that an ad­vanced team of neu­ro­scien­tists and com­puter sci­en­tists could hook your brain up to a ma­chine that gave you max­i­mal, be­yond-or­gas­mic plea­sure for the rest of an ab­nor­mally long life. Then they will blast you and the plea­sure ma­chine into deep space at near light-speed so that you could never be in­terfered with. Would you let them do this for you?

Most peo­ple say they wouldn’t choose the plea­sure ma­chine. They be­gin to re­al­ize that even though they usu­ally ex­pe­rience plea­sure when they get what they de­sired, they want more than just plea­sure. They also want to visit Costa Rica and have good sex and help their loved ones suc­ceed.

But we can be mis­taken when in­fer­ring our de­sires from such in­tu­itions, so I fol­low this up with some neu­ro­science.

Want­ing and liking

It turns out that the neu­ral path­ways for ‘want­ing’ and ‘lik­ing’ are sep­a­rate, but over­lap quite a bit. This ex­plains why we usu­ally ex­pe­rience plea­sure when we get what we want, and thus are tempted to think that all we de­sire is plea­sure. It also ex­plains why we some­times don’t ex­pe­rience plea­sure when we get what we want, and why we wouldn’t plug in to the plea­sure ma­chine.

How do we know this? We now have ob­jec­tive mea­sures of want­ing and lik­ing (de­sire and plea­sure), and these pro­cesses do not always oc­cur to­gether.

liking expressionsOne ob­jec­tive mea­sure of lik­ing is ‘lik­ing ex­pres­sions.’ Hu­man in­fants, pri­mates, and rats ex­hibit ho­molo­gous fa­cial re­ac­tions to pleas­ant and un­pleas­ant tastes.2 For ex­am­ple, both rats and hu­man in­fants dis­play rhyth­mic lip-lick­ing move­ments when pre­sented with sug­ary wa­ter, and both rats and hu­man in­fants dis­play a gap­ing re­ac­tion and mouth-wipes when pre­sented with bit­ter wa­ter.3

More­over, these an­i­mal lik­ing ex­pres­sions change in ways analo­gous to changes in hu­man sub­jec­tive plea­sure. Food is more plea­surable to us when we are hun­gry, and sweet tastes elicit more lik­ing ex­pres­sions in rats when they are hun­gry than when they are full.4 Similarly, both rats and hu­mans re­spond to in­tense doses of salt (more con­cen­trated than in sea­wa­ter) with mouth gapes and other aver­sive re­ac­tions, and hu­mans re­port sub­jec­tive dis­plea­sure. But if hu­mans or rats are de­pleted of salt, both hu­mans and rats re­act in­stead with lik­ing ex­pres­sions (lip-lick­ing), and hu­mans re­port sub­jec­tive plea­sure.5

Luck­ily, these lik­ing and dis­lik­ing ex­pres­sions share a com­mon evolu­tion­ary his­tory, and use the same brain struc­tures in rats, pri­mates, and hu­mans. Thus, fMRI scans have un­cov­ered to some de­gree the neu­ral cor­re­lates of plea­sure, giv­ing us an­other ob­jec­tive mea­sure of plea­sure.6

As for want­ing, re­search has re­vealed that dopamine is nec­es­sary for want­ing but not for lik­ing, and that dopamine largely causes want­ing.7

Now we are ready to ex­plain how we know that we do not de­sire plea­sure alone.

First, one can ex­pe­rience plea­sure even if dopamine-gen­er­at­ing struc­tures have been de­stroyed or de­pleted.8 Cho­co­late milk still tastes just as plea­surable de­spite the se­vere re­duc­tion of dopamine neu­rons in pa­tients suffer­ing from Park­in­son’s dis­ease,9 and the plea­sure of am­phetamine and co­caine per­sists through­out the use of dopamine-block­ing drugs or dietary-in­duced dopamine de­ple­tion — even while these same treat­ments do sup­press the want­ing of am­phetamine and co­caine.10

Se­cond, ele­va­tion of dopamine causes an in­crease in want­ing, but does not cause an in­crease in lik­ing (when the goal is ob­tained). For ex­am­ple, mice with raised dopamine lev­els work harder and re­sist dis­trac­tions more (com­pared to mice with nor­mal dopamine lev­els) to ob­tain sweet food re­wards, but they don’t ex­hibit stronger lik­ing re­ac­tions when they ob­tain the re­wards.11 In hu­mans, drug-in­duced dopamine in­creases cor­re­late well with sub­jec­tive rat­ings of ‘want­ing’ to take more of the drug, but not with rat­ings of ‘lik­ing’ that drug.12 In these cases, it be­comes clear that we want some things be­sides the plea­sure that usu­ally re­sults when we get what we want.

In­deed, it ap­pears that mam­mals can come to want some­thing that they have never be­fore ex­pe­rienced plea­sure when get­ting. In one study,13 re­searchers ob­served the neu­ral cor­re­lates of want­ing while feed­ing rats in­tense doses of salt dur­ing their very first time in a state of salt-de­ple­tion. That is, the rats had never be­fore ex­pe­rienced in­tense doses of salt as plea­surable (be­cause they had never been salt-de­pleted be­fore), and yet they wanted salt the very first time they en­coun­tered it in a salt-de­pleted state.

Com­min­gled signals

But why are lik­ing and want­ing so com­min­gled that we might con­fuse the two, or think that the only thing we de­sire is plea­sure? It may be be­cause the two differ­ent sig­nals are liter­ally com­min­gled on the same neu­rons. Re­sarchers ex­plain:

Mul­ti­plexed sig­nals com­min­gle in a man­ner akin to how wire and op­ti­cal com­mu­ni­ca­tion sys­tems carry tele­phone or com­puter data sig­nals from mul­ti­ple tele­phone con­ver­sa­tions, email com­mu­ni­ca­tions, and in­ter­net web traf­fic over a sin­gle wire. Just as the differ­ent sig­nals can be re­solved at their des­ti­na­tion by re­ceivers that de­code ap­pro­pri­ately, we be­lieve that mul­ti­ple re­ward sig­nals [lik­ing, want­ing, and learn­ing] can be packed into the ac­tivity of sin­gle ven­tral pal­li­dal neu­rons in much the same way, for po­ten­tial un­pack­ing down­stream.

......we have ob­served a sin­gle neu­ron to en­code all three sig­nals… at var­i­ous mo­ments or in differ­ent ways (Smith et al., 2007; Tin­dell et al., 2005).14

Conclusion

In the last decade, neu­ro­science has con­firmed what in­tu­ition could only sug­gest: that we de­sire more than plea­sure. We act not for the sake of plea­sure alone. We can­not solve the Friendly AI prob­lem just by pro­gram­ming an AI to max­i­mize plea­sure.

Notes

1 Noz­ick (1974), pp. 44-45.

2 Steiner (1973); Steiner et al (2001).

3 Grill & Ber­ridge (1985); Grill & Nor­gren (1978).

4 Ber­ridge (2000).

5 Ber­ridge et al. (1984); Schulkin (1991); Tin­dell et al. (2006).

6 Ber­ridge (2009).

7 Ber­ridge (2007); Robin­son & Ber­ridge (2003).

8 Ber­ridge & Robin­son (1998); Ber­ridge et al. (1989); Pecina et al. (1997).

9 Sienkiewicz-Jarosz et al. (2005).

10 Brauer et al. (2001); Brauer & de Wit (1997); Ley­ton (2009); Ley­ton et al. (2005).

11 Cag­niard et al. (2006); Pecina et al. (2003); Tin­dell et al. (2005); Wyvell & Ber­ridge (2000).

12 Evans et al. (2006); Ley­ton et al. (2002).

13 Tin­dell et al. (2009).

13 Al­dridge & Ber­ridge (2009). See Smith et al. (2011) for more re­cent de­tails on com­min­gling.

References

Al­dridge & Ber­ridge (2009). Neu­ral cod­ing of plea­sure: ‘rose-tinted glasses’ of the ven­tral pal­li­dum. In Kringelbach & Ber­ridge (eds.), Plea­sures of the brain (pp. 62-73). Oxford Univer­sity Press.

Ber­ridge (2000). Mea­sur­ing he­do­nic im­pact in an­i­mals and in­fants: Microstruc­ture of af­fec­tive taste re­ac­tivity pat­terns. Neu­ro­science and Biobe­hav­ioral Re­views, 24: 173-198.

Ber­ridge (2007). The de­bate over dopamine’s role in re­ward: the case for in­cen­tive salience. Psy­chophar­ma­col­ogy, 191: 391-431.

Ber­ridge (2009). ‘Lik­ing’ and ‘want­ing’ food re­wards: Brain sub­strates and roles in eat­ing di­s­or­ders. Phys­iol­ogy & Be­hav­ior, 97: 537-550.

Ber­ridge, Flynn, Schulkin, & Grill (1984). Sodium de­ple­tion en­hances salt palata­bil­ity in rats. Be­hav­ioral Neu­ro­science, 98: 652-660.

Ber­ridge, Ve­nier, & Robin­son (1989). Taste re­ac­tivity anal­y­sis of 6-hy­drox­y­dopamine-in­duced apha­gia: Im­pli­ca­tions for arousal and an­he­do­nia hy­pothe­ses of dopamine func­tion. Be­hav­ioral Neu­ro­science, 103: 36-45.

Ber­ridge & Robin­son (1998). What is the role of dopamine in re­ward: He­donic im­pact, re­ward learn­ing, or in­cen­tive salience? Brain Re­search Re­views, 28: 309-369.

Brauer, Cram­blett, Pax­ton, & Rose (2001). Haloperi­dol re­duces smok­ing of both nico­tine-con­tain­ing and deni­co­tinized cigarettes. Psy­chophar­ma­col­ogy, 159: 31-37.

Brauer & de Wit (1997). High dose pi­moz­ide does not block am­phetamine-in­duced eu­pho­ria in nor­mal vol­un­teers. Phar­ma­col­ogy Bio­chem­istry & Be­hav­ior, 56: 265-272.

Cag­niard, Beeler, Britt, McGe­hee, Marinelli, & Zhuang (2006). Dopamine scales perfor­mance in the ab­sence of new learn­ing. Neu­ron, 51: 541-547.

Evans, Pavese, Lawrence, Tai, Ap­pel, Doder, Brooks, Lees, & Pic­cini (2006). Com­pul­sive drug use linked to sen­si­tized ven­tral stri­atal dopamine trans­mis­sion. An­nals of Neu­rol­ogy, 59: 852-858.

Grill & Ber­ridge (1985). Taste re­ac­tivity as a mea­sure of the neu­ral con­trol of palata­bil­ity. In Ep­stein & Sprague (eds.), Progress in Psy­chobiol­ogy and Phys­iolog­i­cal Psy­chol­ogy, Vol 2 (pp. 1-6). Aca­demic Press.

Grill & Nor­gren (1978). The taste re­ac­tivity test II: Mimetic re­sponses to gus­ta­tory stim­uli in chronic tha­la­mic and chronic de­cere­brate rats. Brain Re­search, 143: 263-279.

Ley­ton, Boileau, Benkelfat, Dik­sic, Baker, & Dagher (2002). Am­phetamine-in­duced in­creases in ex­tra­cel­lu­lar dopamine, drug want­ing, and nov­elty seek­ing: a PET/​[11C]raclo­pride study in healthy men. Neu­ropsy­chophar­ma­col­ogy, 27: 1027-1035.

Ley­ton, Casey, De­laney, Ko­li­vakis, & Benkelfat (2005). Co­caine crav­ing, eu­pho­ria, and self-ad­minis­tra­tion: a pre­limi­nary study of the effect of cat­e­cholamine pre­cur­sor de­ple­tion. Be­hav­ioral Neu­ro­science, 119: 1619-1627.

Ley­ton (2009). The neu­ro­biol­ogy of de­sire: Dopamine and the reg­u­la­tion of mood and mo­ti­va­tional states in hu­mans. In Kringelbach & Ber­ridge (eds.), Plea­sures of the brain (pp. 222-243). Oxford Univer­sity Press.

Noz­ick (1974). Anar­chy, State, and Utopia. Ba­sic Books.

Pecina, Ber­ridge, & Parker (1997). Pi­moz­ide does not shift palat­i­bil­ity: Sepa­ra­tion of an­he­do­nia from sen­so­ri­mo­tor sup­pres­sion by taste re­ac­tivity.Phar­ma­col­ogy Bio­chem­istry and Be­hav­ior, 58: 801-811.

Pecina, Cag­niard, Ber­ridge, Al­dridge, & Zhuang (2003). Hyper­dopamin­er­gic mu­tant mice have higher ‘want­ing’ but not ‘lik­ing’ for sweet re­wards. The Jour­nal of Neu­ro­science, 23: 9395-9402.

Robin­son & Ber­ridge (2003). Ad­dic­tion. An­nual Re­view of Psy­chol­ogy, 54: 25-53.

Schulkin (1991). Sodium Hunger: the Search for a Salty Taste. Cam­bridge Univer­sity Press.

Sienkiewicz-Jarosz, Scin­ska, Ku­ran, Ry­glewicz, Ro­gowski, Wro­bel, Korkosz, Kukwa, Kos­towski, & Bienkowski (2005). Taste re­sponses in pa­tients with Park­in­son’s dis­ease. Jour­nal of Neu­rol­ogy, Neu­ro­surgery, & Psy­chi­a­try, 76: 40-46.

Smith, Ber­ridge, & Al­dridge (2007). Ven­tral pal­li­dal neu­rons dis­t­in­guish ‘lik­ing’ and ‘want­ing’ ele­va­tions caused by opi­oids ver­sus dopamine in nu­cleus acum­bens. Pro­gram No. 310.5, 2007 Neu­ro­science Meet­ing Plan­ner. San Diego, CA: So­ciety for Neu­ro­science.

Smith, Ber­ridge, & Al­dridge (2011). Disen­tan­gling plea­sure from in­cen­tive salience and learn­ing sig­nals in brain re­ward cir­cuitry. Pro­ceed­ings of the Na­tional Academy of Sciences PNAS Plus, 108: 1-10.

Steiner (1973). The gusto­fa­cial re­sponse: Ob­ser­va­tion on nor­mal and ane­cephalic new­born in­fants. Sym­po­sium on Oral Sen­sa­tion and Per­cep­tion, 4: 254-278.

Steiner, Glaser, Haw­illo, & Ber­ridge (2001). Com­par­a­tive ex­pres­sion of he­do­nic im­pact: af­fec­tive re­ac­tions to taste by hu­man in­fants and other pri­mates.Neu­ro­science and Biobe­hav­ioral Re­views, 25: 53-74.

Tin­dell, Ber­ridge, Zhang, Pecina, & Al­dridge (2005). Ven­tral pal­li­dal neu­rons code in­cen­tive mo­ti­va­tion: Am­plifi­ca­tion by mesolim­bic sen­si­ti­za­tion and am­phetamine. Euro­pean Jour­nal of Neu­ro­science, 22: 2617-2634.

Tin­dell, Smith, Pecina, Ber­ridge, & Al­dridge (2006). Ven­tral pal­li­dum firing codes he­do­nic re­ward: When a bad taste turns good. Jour­nal of Neu­ro­phys­iol­ogy, 96: 2399-2409.

Tin­dell, Smith, Ber­ridge, & Al­dridge (2009). Dy­namic com­pu­ta­tion of in­cen­tive salience: ‘want­ing’ what was never ‘liked’. The Jour­nal of Neu­ro­science, 29: 12220-12228.

Wyvell & Ber­ridge (2000). In­tra-ac­cum­bens am­phetamine in­creases the con­di­tioned in­cen­tive salience of su­crose re­ward: En­hance­ment of re­ward ‘want­ing’ with­out en­hanced ‘lik­ing’ or re­sponse re­in­force­ment. Jour­nal of Neu­ro­science, 20: 8122-8130.