The Neuroscience of Desire

Who knows what I want to do? Who knows what any­one wants to do? How can you be sure about some­thing like that? Isn’t it all a ques­tion of brain chem­istry, sig­nals go­ing back and forth, elec­tri­cal en­ergy in the cor­tex? How do you know whether some­thing is re­ally what you want to do or just some kind of nerve im­pulse in the brain? Some minor lit­tle ac­tivity takes place some­where in this unim­por­tant place in one of the brain hemi­spheres and sud­denly I want to go to Mon­tana or I don’t want to go to Mon­tana.

- Don DeLillo, White Noise

Win­ning at life means achiev­ing your goals that is, satis­fy­ing your de­sires. As such, it will help to un­der­stand how our de­sires work. (I was tempted to ti­tle this ar­ti­cle The Hid­den Com­plex­ity of Wishes: Science Edi­tion!)

Pre­vi­ously, I in­tro­duced read­ers to the neu­ro­science of emo­tion (af­fec­tive neu­ro­science), and ex­plained that the re­ward sys­tem in the brain has three ma­jor com­po­nents: lik­ing, want­ing, and learn­ing. That post dis­cussed ‘lik­ing’ or plea­sure. To­day we dis­cuss ‘want­ing’ or de­sire.

The birth of neuroeconomics

Much work has been done on the af­fec­tive neu­ro­science of de­sire,1 but I am less in­ter­ested with de­sire as an emo­tion than I am with de­sire as a cause of de­ci­sions un­der un­cer­tainty. This lat­ter as­pect of de­sire is mostly stud­ied by neu­roe­co­nomics,2 not af­fec­tive neu­ro­science.

From about 1880-1960, neo­clas­si­cal eco­nomics pro­posed sim­ple, ax­io­matic mod­els of hu­man choice-mak­ing fo­cused on the idea that agents make ra­tio­nal de­ci­sions aimed at max­i­miz­ing ex­pected util­ity. In the 1950s and 60s, how­ever, economists dis­cov­ered some para­doxes of hu­man be­hav­ior that vi­o­lated the ax­ioms of these mod­els.3 In the 70s and 80s, psy­chol­ogy launched an even broader at­tack on these mod­els. For ex­am­ple, while economists as­sumed that choices among ob­jects should not de­pend on how they are de­scribed (‘de­scrip­tive in­var­i­ance’), psy­chol­o­gists dis­cov­ered pow­er­ful fram­ing effects.4

In re­sponse, the field of be­hav­ioral eco­nomics be­gan to offer mod­els of hu­man choice-mak­ing that fit the ex­per­i­men­tal data bet­ter than sim­ple mod­els of neo­clas­si­cal eco­nomics did.5 Be­hav­ioral economists of­ten pro­posed mod­els that could be thought of as in­for­ma­tion-pro­cess­ing al­gorithms, so neu­ro­scien­tists be­gan look­ing for ev­i­dence of these al­gorithms in the hu­man brain, and neu­roe­co­nomics was born.

(Warn­ing: the rest of this post as­sumes some fa­mil­iar­ity with microe­co­nomics.)

Valu­a­tion and choice in the brain

De­spite their differ­ences, mod­els of de­ci­sion-mak­ing from neo­clas­si­cal eco­nomics,6 be­hav­ioral eco­nomics,7 and even com­puter sci­ence8 share a com­mon con­clu­sion:

De­ci­sion mak­ers in­te­grate the var­i­ous di­men­sions of an op­tion into a sin­gle mea­sure of its idiosyn­cratic sub­jec­tive value and then choose the op­tion that is most valuable. Com­par­i­sons be­tween differ­ent kinds of op­tions rely on this ab­stract mea­sure of sub­jec­tive value, a kind of ‘com­mon cur­rency’ for choice. That hu­mans can in­fact com­pare ap­ples to or­anges when they buy fruit is ev­i­dence for this ab­stract com­mon scale.9

Though economists tend to claim only that agents act ‘as if’ they use the ax­ioms of eco­nomic the­ory to make de­ci­sions,10 there is now sur­pris­ing ev­i­dence that sub­jec­tive value and eco­nomic choice are en­coded by par­tic­u­lar neu­rons in the brain.11

More than a dozen stud­ies show that the sub­jec­tive util­ity of differ­ent goods or ac­tions are en­coded on a com­mon scale by the ven­tro­me­dial pre­frontal cor­tex and the stri­a­tum in pri­mates (in­clud­ing hu­mans),12 as is tem­po­ral dis­count­ing.13 More­over, the brain tracks fore­casted and ex­pe­rienced value, prob­a­bly for the pur­pose of learn­ing.14 Re­searchers have also shown how mod­u­la­tion of a com­mon value sig­nal could ac­count for loss aver­sion and am­bi­guity aver­sion,15 two psy­cholog­i­cal dis­cov­er­ies that had threat­ened stan­dard eco­nomic mod­els of de­ci­sion-mak­ing. Fi­nally, sub­jec­tive value is learned via iter­a­tive up­dat­ing (af­ter ex­pe­rience) in dopamin­er­gic neu­rons.16

Once a com­mon-cur­rency val­u­a­tion of goods and ac­tions has been performed, how is a choice made be­tween them? Ev­i­dence im­pli­cates (at least) the lat­eral pre­frontal and pari­etal cor­tex in a pro­cess that in­cludes neu­rons en­cod­ing prob­a­bil­is­tic rea­son­ing.17 In­ter­est­ingly, while val­u­a­tion struc­tures en­code ab­solute (and thus tran­si­tive) sub­jec­tive value, choice-mak­ing struc­tures “rescale these ab­solute val­ues so as to max­i­mize the differ­ences be­tween the available op­tions be­fore choice is at­tempted,”18 per­haps via a nor­mal­iza­tion mechanism like the one dis­cov­ered in the vi­sual cor­tex.19

Beyond these ba­sic con­clu­sions, many open ques­tions and con­tro­ver­sies re­main.20 The hottest de­bate to­day con­cerns whether differ­ent val­u­a­tion sys­tems en­code in­con­sis­tent val­ues for the same ac­tions (lead­ing to differ­ent con­clu­sions on which ac­tion to take),21 or whether differ­ent val­u­a­tion sys­tems con­tribute to the same fi­nal val­u­a­tion pro­cess (lead­ing to a sin­gle, un­am­bigu­ous con­clu­sion on which ac­tion to take).22 I think this race is too close to call, though I lean to­ward the lat­ter model due to the per­sua­sive case made for it by Glim­cher (2010).

De­spite these open ques­tions, 15 years of neu­roe­co­nomics re­search sug­gests an im­pres­sive re­duc­tion from eco­nomics to psy­chol­ogy to neu­ro­science may be pos­si­ble, re­sult­ing in some­thing like this23:

Self-help

With this ba­sic frame­work in place, what can the neu­ro­science of de­sire tell us about how to win at life?

  1. Want­ing is differ­ent than lik­ing, and we don’t only want hap­piness or plea­sure.24 Thus, the perfect he­do­nist might not be fully satis­fied. Pay at­ten­tion to all your de­sires, not just your de­sires for plea­sure.

  2. In par­tic­u­lar, you should sub­ject your­self to novel and challeng­ing ac­tivi­ties reg­u­larly through­out your life. Do­ing so keeps your dopamine (mo­ti­va­tion) sys­tem flow­ing, be­cause novel and challeng­ing cir­cum­stances drive you to act and find solu­tions, which in turn leads to greater satis­fac­tion than do ‘lazy’ plea­sures like sleep­ing and eat­ing.25

  3. In par­tic­u­lar, do­ing novel and challeng­ing ac­tivi­ties with your sig­nifi­cant other will help you ex­pe­rience satis­fac­tion to­gether, and im­prove bond­ing and in­ti­macy.26

  4. Your brain gen­er­ates re­ward sig­nals when ex­pe­rienced value sur­passes fore­casted value.14 So: lower your ex­pec­ta­tions and your brain will be pleas­antly sur­prised when things go well. Things go­ing perfectly ac­cord­ing to plan is not the norm, so don’t treat it as if it is.

  5. Many of the neu­rons in­volved in val­u­a­tion and choice have stochas­tic fea­tures, mean­ing that when the sub­jec­tive util­ity of two or more op­tions are similar (rep­re­sented in the brain by neu­rons with similar firing rates), we some­times choose to do some­thing other than the ac­tion that has the most sub­jec­tive util­ity.27 In other words, we some­times fail to do what we most want to do, even if stan­dard bi­ases and faults (akra­sia, etc.) are con­sid­ered to be part of the val­u­a­tion equa­tion. So don’t beat your­self up if you have a hard time choos­ing be­tween op­tions of roughly equal sub­jec­tive util­ity, or if you feel you’ve cho­sen an op­tion that does not have the great­est sub­ject util­ity.

The neu­ro­science of de­sire is pro­gress­ing rapidly, and I have no doubt that we will know much more about it in an­other five years. In the mean­time, it has already pro­duced use­ful re­sults.

And the neu­ro­science of plea­sure and de­sire is not only rele­vant to self-help, of course. In later posts, I will ex­am­ine the im­pli­ca­tions of re­cent brain re­search for meta-ethics and for Friendly AI.

Notes

1 Ber­ridge (2007); Ley­ton (2009).

2 Good overviews of neu­roe­co­nomics in­clude: Glim­cher (2010, 2009); Glim­cher et al. (2008); Kable & Glim­cher (2009); Glim­cher & Rus­ti­chini (2004); Camerer et al (2005); San­fey et al (2006); Politser (2008); Mon­tague (2007). Berns (2005) is an overview from a self-help per­spec­tive.

3 Most fa­mously, the Allais Para­dox (Allais, 1953) and the Ells­berg para­dox (Ells­berg, 1961). Eliezer wrote three posts on the Allais para­dox.

4 Tver­sky & Kah­ne­man (1981).

5 The most fa­mous ex­am­ple is Prospect The­ory (Kah­ne­man & Tver­sky, 1979).

6 von Neu­mann & Mor­gen­stern (1944).

7 Kah­ne­man & Tver­sky (1979).

8 Sut­ton & Barto (1998).

9 Kable & Glim­cher (2009).

10 Fried­man (1953); Gul & Pe­sendorfer (2008).

11 Kable & Glim­cher (2009) is a good overview, as are sec­tions 2 and 3 of Glim­cher (2010).

12 Kable & Glim­cher (2009); Padoa-Schioppa & As­sad (2006, 2008); Taka­hashi et al. (2009); Lau & Glim­cher (2008); Same­jima et al. (2005); Plass­mann et al. (2007); Hare et al. (2008); Hare et al. (2009).

13 Kable & Glim­cher (2007); Louie & Glim­cher (2010).

14 Rut­ledge et al. (2010); Del­gado (2007); Knut­son & Cooper (2005); O’Do­herty (2004).

15 Fox & Poldrack (2008); Tom et al. (2007); Levy et al. (2007); Levy et al. (2010).

16 Niv & Mon­tague (2009); Schultz et al. (1997); Tobler et al. (2003, 2005); Waelti et al. (2001); Bayer & Glim­cher (2005); Fio­rillo et al. (2003, 2008); Kobayashi & Schultz (2008); Roesch et al. (2007); D’Ar­denne et al. (2008); Zaghloul et al. (2009); Pes­siglione e tal. (2006).

17 For tech­ni­cal rea­sons, most of this work has been done on the sac­cadic-con­trol sys­tem: Glim­cher & Sparks (1992); Basso & Wurtz (1998); Dor­ris & Munoz (1998); Platt & Glim­cher (1999); Yang & Shadlen (2007); Dor­ris & Glim­cher (2004); Su­grue et al. (2004); Shadlen & New­some (2001); Church­land et al. (2008); Ki­ani et al. (2008); Wang (2008); Kable & Glim­cher (2007); Yu & Dayan (2005). But Glim­cher (2010) pro­vides some rea­sons to think these re­sults will gen­er­al­ize.

18 Kable & Glim­cher (2009).

19 Heeger (1992).

20 See Kable & Glim­cher (2009), and the fi­nal chap­ter of Glim­cher (2010). Neu­roe­conomists are also be­gin­ning to model how game-the­o­retic calcu­la­tions oc­cur in the brain: Fehr & Camerer (2007); Lee (2008); Mon­tague & Lohrenz (2007); Singer & Fehr (2005).

21 Balleine et al. (2008); Bos­saerts et al. (2009); Daw et al. (2005); Dayan and Balleine (2002); Ran­gel et al. (2008).

22 Glim­cher (2009); Levy et al. (2010).

23 Figure 16.1 from Glim­cher (2010).

24 Smith et al. (2009).

25 Berns (2005) pro­vides a pop­u­lar-level overview of the ev­i­dence, here. Some of the rele­vant re­search pa­pers in­clude: Berns et al. (2001); Ben­jamin et al. (1996); Kem­per­mann et al. (1997).

26 Aron et al. (2000, 2003).

27 See chap­ters 9 and 10 of Glim­cher (2010).

References

Allais (1953). Le com­porte­ment de l’homme ra­tionnel de­vant le risque: cri­tique des pos­tu­lats et ax­iomes de l’école Améri­caine. Econo­met­rica, 21: 503-546.

Aron, Nor­man, Aron, McKenna, & Hey­man (2000). Cou­ples shared par­ti­ci­pa­tion in novel and arous­ing ac­tivi­ties and ex­pe­rienced re­la­tion­ship qual­ity. Jour­nal of Per­son­al­ity and So­cial Psy­chol­ogy, 78: 273-283.

Aron, Nor­man, Aron, & Le­wandowski (2003). Shared par­ti­ci­pa­tion in self- ex­pand­ing ac­tivi­ties: Pos­i­tive effects on ex­pe­rienced mar­i­tal qual­ity. In Nol­ler & Feeney (eds.), Mar­i­tal in­ter­ac­tion (pp. 177-196). Cam­bridge Univer­sity Press.

Balleine, Daw, & O’Do­herty (2009). Mul­ti­ple forms of value learn­ing and the func­tion of dopamine. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 367-387). Aca­demic Press.

Basso & Wurtz (1998). Mo­du­la­tion of neu­ronal ac­tivity in su­pe­rior col­licu­lus by changes in tar­get prob­a­bil­ity. Jour­nal of Neu­ro­science, 18: 7519–7534.

Bayer & Glim­cher (2005). Mid­brain dopamine neu­rons en­codea quan­ti­ta­tive re­ward pre­dic­tion er­ror sig­nal. Neu­ron, 47: 129–141.

Ben­jamin, Li, Pat­ter­son, Green­berg, Mur­phy, & Hamer (1996). Pop­u­la­tion and fa­mil­ial as­so­ci­a­tion be­tween the D4 dopamine re­cep­tor gene and mea­sures of nov­elty seek­ing. Na­ture Ge­net­ics, 12: 81-84.

Berns (2005). Satis­fac­tion: the sci­ence of find­ing true fulfill­ment. Henry Holt and Co.

Berns, McClure, Pagnoni, & Mon­tague (2001). Pre­dictabil­ity mod­u­lates hu­man brain re­sponse to re­ward. Jour­nal of Neu­ro­science, 21: 2793-2798.

Ber­ridge (2007). The de­bate over dopamine’s role in re­ward: the case for in­cen­tive salience. Psy­chophar­ma­col­ogy, 191: 391-431.

Bos­saerts, , Preuschoff, & Hsu (2009). The neu­ro­biolog­i­cal foun­da­tions of val­u­a­tion in hu­man de­ci­sion-mak­ing un­der un­cer­tainty. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 353–365). Aca­demic Press.

Camerer, Loewen­stein, & Pr­elec (2005). Neu­roe­co­nomics: how neu­ro­science can in­form eco­nomics. Jour­nal of Eco­nomic Liter­a­ture, 43: 9–64.

Church­land, Ki­ani, & Shadlen (2008). De­ci­sion-mak­ing with mul­ti­ple al­ter­na­tives. Na­ture Neu­ro­science, 11: 693–702.

D’Ar­denne, McClure, Nys­trom, & Co­hen (2008). BOLD re­sponses reflect­ing dopamin­er­gic sig­nals in the hu­man Ven­tral Teg­men­tal Area. Science, 319: 1264–1267.

Daw, Niv, & Dayan (2005). Uncer­tainty-based com­pe­ti­tion be­tween pre­frontal and dor­so­lat­eral stri­atal sys­tems for be­hav­ioral con­trol. Na­ture Neu­ro­science, 8: 1704–1711.

Dayan and Balleine (2002). Re­ward, mo­ti­va­tion, and re­in­force­ment learn­ing. Neu­ron, 36: 285–298.

Del­gado (2007). Re­ward-re­lated re­sponses in the hu­man stri­a­tum. An­nals of the New York Academy of Sciences, 1104: 70–88.

Dor­ris & Munoz (1998). Sac­cadic prob­a­bil­ity in­fluences mo­tor­prepa­ra­tion sig­nals and time to sac­cadic ini­ti­a­tion. Jour­nal of Neu­ro­science, 18: 7015–7026.

Dor­ris & Glim­cher (2004). Ac­tivity in pos­te­rior pari­etal cor­tex is cor­re­lated with the rel­a­tive sub­jec­tive de­sir­a­bil­ity of ac­tion. Neu­ron 44: 365–378.

Ells­berg (1961). Risk, Am­bi­guity, and the Sav­age Ax­ioms. Quar­terly Jour­nal of Eco­nomics, 75(4): 643–669.

Fehr & Camerer (2007). So­cial neu­roe­co­nomics: The neu­ral cir­cuitry of so­cial prefer­ences. Trends in Cog­ni­tive Science, 11: 419–427.

Fio­rillo, Tobler, & Schultz (2003). Discrete cod­ing of re­ward prob­a­bil­ity and un­cer­tainty by dopamine neu­rons. Science, 299: 1898–1902.

Fio­rillo, New­some, & Schultz (2008). The tem­po­ral pre­ci­sion of re­ward pre­dic­tion in dopamine neu­rons. Na­ture Neu­ro­science, 11: 966–973.

Fox & Poldrack (2008). Prospect the­ory and the brain. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 145-173). Aca­demic Press.

Fried­man (1953). The method­ol­ogy of pos­i­tive eco­nomics. In Fried­man, Es­says in Pos­i­tive Eco­nomics. Chicago Press.

Glim­cher (2009). Neu­ro­science, Psy­chol­ogy, and Eco­nomic Be­hav­ior: The Emerg­ing Field of Neu­roe­co­nomics. In: Tom­masi, Peter­son, & Nadel (eds.), Cog­ni­tive Biol­ogy: Evolu­tion­ary and Devel­ope­men­tal Per­spec­tives on Mind, Brain, and Be­hav­ior (pp. 261-287). MIT Press.

Glim­cher (2009). Choice: Towards a Stan­dard Back-pocket Model. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 503-521). Aca­demic Press.

Glim­cher (2010). Foun­da­tions of Neu­roe­co­nomic Anala­y­sis. Oxford Univer­sity Press.

Glim­cher & Sparks (1992). Move­ment se­lec­tion in ad­vance of ac­tion in the su­pe­rior col­licu­lus. Na­ture, 355: 542–545.

Glim­cher & Rus­ti­chini (2004). Neu­roe­co­nomics: the con­silience of brain and de­ci­sion. Science, 306: 447–452.

Glim­cher, Camerer, Fehr, & Poldrack (2008). In­tro­duc­tion: A Brief His­tory of Neu­roe­co­nomics. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 1-12). Aca­demic Press.

Gul & Pe­sendorfer (2008). The case for mind­less eco­nomics. In Ca­plan & Schot­ter (eds.), The Foun­da­tions of Pos­i­tive and Nor­ma­tive Eco­nomics (pp. 3–41). Oxford Univer­sity Press.

Hare, O’Do­herty, Camerer, Schultz, & Ran­gel (2008). Dis­so­ci­at­ing the role of the or­bitofrontal cor­tex and the stri­a­tum in the com­pu­ta­tion of goal val­ues and pre­dic­tion er­rors. Jour­nal of Neu­ro­science, 28: 5623–5630.

Hare, Camerer, & Ran­gel (2009). Self-con­trol in de­ci­sion­mak­ing in­volves mod­u­la­tion of the vmPFC val­u­a­tion sys­tem. Science, 324: 646–648.

Heeger (1992). Nor­mal­iza­tion of cell re­sponses in cat stri­ate cor­tex. Vi­sual Neu­ro­science, 9: 181–197.

Kable & Glim­cher (2007). The neu­ral cor­re­lates of sub­jec­tive value dur­ing in­tertem­po­ral choice. Na­ture Neu­ro­science, 10: 1625–1633.

Kable & Glim­cher (2009). The Neu­ro­biol­ogy of De­ci­sion: Con­sen­sus and Con­tro­versy. Neu­ron, 63: 733-745.

Kah­ne­man & Tver­sky (1979). Prospect The­ory: An Anal­y­sis of De­ci­sion un­der Risk. Econo­met­rica, XLVII: 263-291.

Kem­per­mann, Kuhn, & Gage (1997). More hip­pocam­pal neu­rons in adult mice liv­ing in an en­riched en­vi­ron­ment. Na­ture, 386: 493-495.

Ki­ani, Hanks, & Shadlen (2008). Bounded in­te­gra­tion in pari­etal cor­tex un­der­lies de­ci­sions even when view­ing du­ra­tion is dic­tated by the en­vi­ron­ment. Jour­nal of Neu­ro­science, 28: 3017–3029.

Knut­son & Cooper (2005). Func­tional mag­netic res­o­nance imag­ing of re­ward pre­dic­tion. Cur­rent Opinions in Neu­rol­ogy, 18: 411–417.

Kobayashi & Schultz (2008). Influence of re­ward de­lays on re­sponses of dopamine neu­rons. Jour­nal of Neu­ro­science, 28: 7837–7846.

Lau & Glim­cher (2008). Value rep­re­sen­ta­tions in the pri­mate stri­a­tum dur­ing match­ing be­hav­ior. Neu­ron, 58: 451–463.

Lee (2008). Game the­ory and neu­ral ba­sis of so­cial de­ci­sion mak­ing. Na­ture Neu­ro­science, 11: 404–409.

Levy, Rus­ti­chini & Glim­cher (2007). A sin­gle sys­tem rep­re­sents sub­jec­tive value un­der both risky and am­bigu­ous de­ci­sion-mak­ing in hu­mans. In 37th An­nual So­ciety for Neu­ro­science Meet­ing, San Diego, Cal­ifor­nia.

Levy, Snell, Nel­son, Rus­ti­chini, & Glim­cher (2010). Neu­ral Rep­re­sen­ta­tion of Sub­jec­tive Value Un­der Risk and Am­bi­guity. Jour­nal of Neu­ro­phys­iol­ogy, 103: 1036-1047.

Ley­ton (2009). The neu­ro­biol­ogy of de­sire: Dopamine and the reg­u­la­tion of mood and mo­ti­va­tional states in hu­mans. In Kringelbach & Ber­ridge (eds.), Plea­sures of the brain (pp. 222-243). Oxford Univer­sity Press.

Louie & Glim­cher (2010). Separat­ing value from choice: de­lay dis­count­ing ac­tivity in the lat­eral in­tra­pari­etal area. Jour­nal of Neu­ro­science, 30(16): 5498-5507.

Mon­tague (2007). Your brain is (al­most) perfect: How we make de­ci­sions. Plume.

Mon­tague & Lohrenz (2007). To de­tect and cor­rect: Norm vi­o­la­tions and their en­force­ment. Neu­ron, 56: 14–18.

Niv & Mon­tague (2008). The­o­ret­i­cal and em­piri­cal stud­ies oflearn­ing. In Glim­cher, Camerer, Fehr, & Poldrack (eds.), Neu­roe­co­nomics: De­ci­sion Mak­ing and the Brain (pp. 331-351). Aca­demic Press.

O’Do­herty (2004). Re­ward rep­re­sen­ta­tions and re­ward-re­lated learn­ing in the hu­man brain: in­sights from neu­roimag­ing. Cur­rent Opinions in Neu­ro­biol­ogy, 14: 769–776.

Padoa-Schioppa & As­sad (2006). Neu­rons in the or­bitofrontal­cor­tex en­code eco­nomic value. Na­ture, 441: 223–226.

Padoa-Schioppa & As­sad (2008). The rep­re­sen­ta­tion of eco­nomic value in the or­bitofrontal cor­tex is in­var­i­ant for changes of menu. Na­ture Neu­ro­science 11: 95–102.

Pes­siglione, Sey­mour, Flandin, Dolan, & Frith (2006). Dopamine-de­pen­dent pre­dic­tion er­rors un­der­pin re­ward-seek­ing be­havi­our in hu­mans. Na­ture, 442: 1042–1045.

Plass­mann, O’Do­herty, & Ran­gel (2007). Or­bitofrontal cor­tex en­codes will­ing­ness to pay in ev­ery­day eco­nomic trans­ac­tions. Jour­nal of Neu­ro­science, 27: 9984–9988.

Platt & Glim­cher (1999). Neu­ral cor­re­lates of de­ci­sion vari­ables in pari­etal cor­tex. Na­ture, 400: 233–238.

Politser (2008). Neu­roe­co­nomics: a guide to the new sci­ence of mak­ing choices. Oxford Univer­sity Press.

Ran­gel, Camerer, & Mon­tague (2008). A frame­work for study­ing the neu­ro­biol­ogy of value-based de­ci­sion mak­ing. Na­ture Re­views Neu­ro­science, 9: 545–556

Roesch, Calu, & Schoen­baum (2007). Dopamine neu­rons en­code the bet­ter op­tion in rats de­cid­ing be­tween differ­ently de­layed or sized re­wards. Na­ture Neu­ro­science, 10: 1615–1624.

Rut­ledge, Dean, Caplin, & Glim­cher (2010). Test­ing the re­ward pre­dic­tion er­ror hy­poth­e­sis with an ax­io­matic model. Jour­nal of Neu­ro­science, 30(40): 13525-13536.

Same­jima, Ueda, Doya & Ki­mura (2005). Rep­re­sen­ta­tion ofac­tion-specific re­ward val­ues in the stri­a­tum. Science, 310: 1337–1340.

San­fey, Loewen­stein, McClure, & Co­hen (2006). Neu­roe­co­nomics: cross-cur­rents in re­search on de­ci­sion-mak­ing. Trends in Cog­ni­tive Science, 10: 108–116.

Schultz, Dayan, & Mon­tague (1997). A neu­ral sub­strate of pre­dic­tion and re­ward. Science, 275: 1593–1599.

Shadlen & New­some (2001). Neu­ral ba­sis of a per­cep­tual de­ci­sion in the pari­etal cor­tex (area LIP) of the rhe­sus mon­key. Jour­nal of Neu­ro­phys­iol­ogy, 86: 1916–1936.

Singer & Fehr (2005). The neu­roe­co­nomics of mind read­ing and em­pa­thy. Amer­i­can Eco­nomic Re­view, 95: 340–345.

Smith, Mahler, Pecina, & Ber­ridge (2009). He­donic hotspots: gen­er­at­ing sen­sory plea­sure in the brain. In Kringelbach & Ber­ridge (eds.), Plea­sures of the brain (pp. 27-49). Oxford Univer­sity Press.

Su­grue, Cor­rado, & New­some (2004). Match­ing be­hav­ior and the rep­re­sen­ta­tion of value in the pari­etal cor­tex. Science, 304: 1782–1787.

Sut­ton & Barto (1998). Re­in­force­ment Learn­ing: An In­tro­duc­tion. MIT Press.

Taka­hashi, Roesch, Stal­naker, Haney, Calu, Tay­lor, Burke, & Schoen­baum (2009). The or­bitofrontal cor­tex and­ven­tral tegmen­tal area are nec­es­sary for learn­ing from un­ex­pected out­comes. Neu­ron, 62: 269–280.

Tobler, Dick­in­son, & Schultz (2003). Cod­ing of pre­dicted re­ward omis­sion by dopamine neu­rons in a con­di­tioned in­hi­bi­tion paradigm. Jour­nal of Neu­ro­science, 23: 10402–10410.

Tobler, Fio­rillo, & Schultz (2005). Adap­tive cod­ing of re­ward­value by dopamine neu­rons. Science, 307: 1642–1645.

Tom, Fox, Tre­pel & Poldrack (2007). The neu­ral ba­sis of loss aver­sion in de­ci­sion-mak­ing un­der risk. Science, 315: 515–518.

Tver­sky & Kah­ne­man (1981). The fram­ing of de­ci­sions and the psy­chol­ogy of choice. Science, 211(4481): 453–458.

von Neu­mann & Mor­gen­stern (1944). The­ory of Games and Eco­nomic Be­hav­ior. Prince­ton Univer­sity Press.

Waelti, Dick­in­son, & Schultz (2001). Dopamine re­sponses com­ply with ba­sic as­sump­tions of for­mal learn­ing the­ory. Na­ture, 412: 43–48.

Wang (2008). De­ci­sion mak­ing in re­cur­rent neu­ronal cir­cuits. Neu­ron, 60: 215–234.

Yang & Shadlen (2007). Prob­a­bil­is­tic rea­son­ing by neu­rons. Na­ture, 447: 1075–1080.

Yu & Dayan (2005). Uncer­tainty, neu­ro­mod­u­la­tion and at­ten­tion. Neu­ron, 46: 681–692.

Zaghloul, Blanco, Wei­de­mann, McGill, Jaggi, Bal­tuch, & Ka­hana (2009). Hu­man sub­stan­tia ni­gra neu­rons en­code un­ex­pected finan­cial re­wards. Science, 323: 1496–1499.