Review of Sharot & Dolan, ‘Neuroscience of Preference and Choice’ (2011)

[lukeprog]

The new edited volume Neuroscience of Preference and Choice includes chapters from a good chunk of the leading researchers in neuroeconomics. If you read only two books on neuroeconomics, they should be Glimcher (2010) and Neuroscience of Preference and Choice.

First, let me review the main conclusions from my Crash Course in the Neuroscience of Human Motivation:

1. Utilities [aka ‘decision values’] are real numbers ranging from 0 to 1,000 that take action potentials per second as their natural units.

2. Mean utilities are mean firing rates of specific populations of neurons in the final common path of human choice circuits.

3. Mean utilities predict choice stochastically, similar to random utility models from economics.

4. Utilities are encoded cardinally in firing rates relative to neuronal baseline firing rates. (This is opposed to post-Pareto, ordinal notions of utility.)

5. The choice circuit takes as its input a firing rate that encodes relative (normalized) stochastic expected utility.

6. As the choice set size grows, so does the error rate.

7. Final choice is implemented by an argmax function or a reservation price mechanism.

Much of Neuroscience of Preference and Choice repeats and reinforces these conclusions. In this review, I’ll focus on the major results that are discussed in Neuroscience of Preference and Choice but not in my ‘Crash Course’. The editors of Neuroscience of Preference and Choice see two major results from the work reviewed in the book:

1. Contrary to traditional decision-making theories, which assume choices are based on relatively steady preferences, preferences are in fact highly volatile and susceptible to the context in which the alternatives are presented. Moreover, our preferences are modified by the mere act of choosing and altered by changing choice sets.

2. Regardless of whether we are selecting a musical tune, a perfume, or a new car – the brain uses similar computational principles to compute the value of our options, which are tracked by common neural systems.

I will highlight three additional important results:

1. There are three competing valuation systems, and they respond to different stimuli. The first is “model-based” control, associated with goal-directed behavior. The second is “model-free” control, associated with reinforcement learning. The third is “Pavlovian control,” which imposes hard-wired motivation on the choice mechanism. We’re not sure by which exact algorithm the inputs from these three competing valuation systems determine final choice, but several possible algorithms are being tested.

2. Brain systems involved in valuation and choice acquire and use information about the environment and about utility in different ways. Values may not be consistent across systems, and choices emerging from some systems may not be consistent with their own underlying values.

3. In goal-directed behavior, we simulate achieving the goal, during which a dopamine signal tracks the value we anticipate getting from the achievement of that goal. Unfortunately, a variety of biases limit our capacity for accurate affective forecasting.

Many of the chapters do not report surprising results from the last 10 years of neuroeconomics; instead, they explain the neural mechanisms behind things we already knew about, for example common biases in decision-making, the social and emotional factors that contribute to value appraisal, the ways that context can affect preference, the ways that action can affect preference, and more.

Caplin’s chapter presents not a “result” but the interesting suggestion that choice sets can be modeled as percepts. This would be an interesting result, but we’ll have to wait for the tests he proposes to be performed.

The final chapter reviews the implications of the field’s results for public policy.

[MileyCyrus]

Utilities [aka ‘decision values’] are real numbers ranging from 0 to 1,000 that take action potentials per second as their natural units.

Does this mean there is a limit on how much we can want/​prefer something?

[lukeprog]

Yes, though keep in mind that these utilities are renormalized, so the important metric is how much more one option is valued over the others during a particular choice, not the absolute value of how many action potentials per second a particular option is “valued” with. The value for the exact same thing (say, an apple) could be represented by a different number of action potentials per second depending on the other options in the choice set, among other factors (e.g. context effects).

This does, it seems to me, give some neurobiological credence to a particular solution to Pascal’s Mugging: bounding your utility function.

[AlexSchell]

What is it for something to give neurobiological credence to the “bounded utility” answer to Pascal’s Mugging?

[buybuydandavis]

Do they have this down to particular physical neural pathways, so that the receptor types in the path area clear?

You mention dopamine neurons in goal directed valuation. Do all 3 competing valuation systems use dopamine? Does the aggregating circuit use this as well? It would be very interesting if you could change the balance between the 3 neurochemically.

[gwern]

First, let me review the main conclusions from my ‘Crash Course’:

Did you mean not to link it?

[lukeprog]

Fixed, thanks.

[amcknight]

There are three competing valuation systems, and they respond to different stimuli

Are you saying that these 3 systems are not performing valuation between the same items? Or is there a common set of items being injected into each system and the systems are performing different algorithms on the same item set? Something else? BTW, thanks for these neuroscience summaries!