The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value
Introduction
Many psychological and economic theories of decision making are couched within the framework of value maximization (e.g., Kahneman and Tversky, 1979, Samuelson, 1937). According to such a framework, an organism facing a choice first determines the scalar subjective value (SV) of each available alternative, and then selects the alternative with the greatest SV. SV serves as a common currency, allowing complex and qualitatively different alternatives to be compared on a common scale. Stable assessments of SV can support consistent decision behavior over time and across contexts.
Although there are noted and well-studied anomalies (see, e.g., Tversky and Kahneman, 1974), value maximization has been an enormously productive framework for understanding learning and decision making. As such, a major objective in decision neuroscience has been to uncover neural signals associated with the computation and representation of SV. A growing number of studies have used fMRI to measure blood oxygenation level dependent (BOLD) signal from the brain while human participants perform evaluation and decision-making tasks, comparing BOLD response amplitudes across outcomes or prospects that differ in SV. Qualitative reviews of this work (e.g., Delgado, 2007, Grabenhorst and Rolls, 2011, Kable and Glimcher, 2009, Knutson and Cooper, 2005, Levy and Glimcher, 2012, Montague and Berns, 2002, O'Doherty, 2004, Peters and Büchel, 2010) have concluded that: (1) BOLD signal in a small number of regions, including ventral striatum (VS), ventromedial prefrontal cortex (VMPFC) and (in some cases) posterior cingulate cortex (PCC) scales with the SV of the available alternatives during choice; (2) the same regions also respond when a reward is received, implicating a common set of regions in the evaluation of both prospects and outcomes; and (3) these regions respond to outcomes in multiple domains (e.g., primary rewards such as food, secondary rewards such as money, social rewards, etc.), as would be required for any domain-general common currency signal.
Though many studies support this consensus, there remains a risk that qualitative reviews have either overestimated the support for prevailing hypotheses or overlooked unanticipated but reliable findings. Such selectivity certainly can occur in individual studies, which may focus discussion on regions identified a priori while ignoring equally strong results elsewhere. In the present work we use meta-analytic techniques to quantify the state of the evidence for the above three claims. Quantitative meta-analysis offers a complementary perspective to well-designed individual experiments and qualitative reviews, allowing a large and diverse array of existing findings to be aggregated, regardless of how each experiment was originally interpreted.
We perform a series of coordinate-based meta-analyses of published fMRI findings relating to the neural correlates of SV. Our examination focuses on four specific questions.
Question 1: In what regions does BOLD correlate positively or negatively with SV? Our first step tests for reliable BOLD correlates of SV. Previous reviews have tended to emphasize BOLD responses that scale positively with SV. At the single neuron level, however, there seem to be roughly equal numbers of neurons that increase or decrease firing as SV changes (Kennerley et al., 2009). There have been preliminary suggestions, from both fMRI and electrophysiological studies, that these positively and negatively directed responses may be anatomically dissociable (Monosov and Hikosaka, 2012, O'Doherty et al., 2001).
Question 2: Do neural responses follow a linear or nonlinear function of SV? A second question is whether neural correlates of SV can be distinguished from potentially related signals of arousal or salience (cf. Zink et al., 2004). Fig. 1 illustrates two possible ways in which a neural response might vary across different levels of SV. Pattern A shows a response rising linearly with SV, while Pattern B shows a U-shaped response that is maximal either for extreme positive or extreme negative values. The linear pattern is more properly a correlate of SV, while the U-shaped pattern might be better described as a signal of arousal or salience. A source of difficulty in disambiguating these patterns is that their predictions overlap in experiments that only involve rewards. Neural structures that appear reward-responsive in such experiments may also respond strongly to penalties (Carretié et al., 2009, Levita et al., 2009). Previous work suggests it may be possible to distinguish areas that show either or both of the activity profiles in Fig. 1 (Litt et al., 2011).
Question 3: Do the same regions encode SV when a decision is made and when an outcome is experienced? We next ask whether the neural correlates of SV during decision-making are similar or different from SV responses during the receipt of an outcome (cf. “decision utility” vs. “experienced utility”; Kahneman et al., 1997). Numerous experiments have demonstrated that neural correlates of SV can indeed be identified during decision making (Kable and Glimcher, 2007, Knutson et al., 2007, Plassmann et al., 2007, Tom et al., 2007), and the same general regions are active when rewards are experienced. However, there is also evidence that neural responses during choice and during outcome delivery may be anatomically dissociable (Knutson et al., 2001b, Smith et al., 2010).
Question 4: Is SV encoded differently for primary versus secondary incentives? An important question is to what extent the neural representation of SV is domain-specific versus domain-general. Many studies targeting this question have investigated whether secondary rewards (e.g., money) are evaluated using the same neural architecture as primary rewards (e.g., juice). A number of studies have concluded that different categories of rewards recruit at least partially overlapping neural substrates (Chib et al., 2009, Kim et al., 2011, Valentin and O'Doherty, 2009). Partial overlap has also been noted for primary and secondary aversive outcomes (Delgado et al., 2011). Other work has reached the conclusion that neural mechanisms involved in evaluating primary and secondary incentives are dissociable (Sescousse et al., 2010).
Previous meta-analyses have examined subsets of the above questions in more restricted databases. Both Knutson and Greer (2008) and Diekhof et al. (2012) focused on the passive anticipation of rewards, comparing SV responses associated with outcome anticipation and delivery. Peters and Büchel (2010) and Levy and Glimcher (2012) presented anatomically circumscribed meta-analyses in VMPFC and adjacent regions, showing overlap between activation coordinates associated with multiple forms of SV. The present investigation extends previous work by examining all four of the above questions simultaneously across the entire brain in a large corpus of published studies. Our sample size affords us the statistical power to conduct comprehensive and rigorous inferential tests at the meta-analytic level.
Across the present meta-analyses, we find evidence for reliable positive correlates of SV in VMPFC and anterior VS. We find that these can be distinguished from responses in anterior insula, other striatal regions, and dorsomedial prefrontal cortex (DMPFC), which seem more related to arousal or salience. We find overlapping responses in VMPFC and VS to decision and experienced SV, as well as overlapping responses to both primary and secondary incentives. These results bolster the conclusion that VMPFC and VS form the core of a “valuation system” believed to play a critical role in value-based learning and decision-making.
Section snippets
Coordinate-based meta-analysis
The importance of quantitatively synthesizing the findings of independent experiments is well recognized across scientific domains. Quantitative meta-analysis poses unique challenges in the context of neuroimaging. Neuroimaging results are conventionally reported in terms of the anatomical location of peak statistical effects, expressed using a stereotactic system such as Talairach (Talairach and Tournoux, 1988) or Montreal Neurological Institute (MNI) coordinate space.
This reporting convention
Whole-brain meta-analysis
We first examine overall associations between SV and BOLD. Here we organize experimental findings solely according to the direction of the reported effects (positive or negative), aggregating across other attributes. An effect was coded as positive if a greater BOLD response was observed for more rewarding (or less aversive) outcomes; it was coded as negative if a greater BOLD response was observed for less rewarding (or more aversive) outcomes. Our corpus contains 200 studies that report
Directionality and effect of valence
In the present paper we report a series of meta-analyses examining neural effects of SV as measured using fMRI. Our first step was to compare the locations of positive and negative effects of SV on BOLD. Positive effects consist of larger BOLD responses to more rewarding or less aversive outcomes; negative effects consist of larger BOLD responses to less rewarding or more aversive outcomes. We observed a greater density of positive than negative effects in the VMPFC, PCC, and striatum. We
Acknowledgments
This work was supported by the National Institutes of Health (grant number R01-DA029149 to J.W.K., and F32-DA030870 to J.T.M.); and by a fellowship from the University of Pennsylvania Positive Psychology Center to O.B.C.
References (80)
Food reward: brain substrates of wanting and liking
Neurosci. Biobehav. Rev.
(1996)- et al.
The striatum beyond reward: caudate responds intensely to unpleasant pictures
Neuroscience
(2009) - et al.
Functional heterogeneity within Broca's area during verbal working memory
Physiol. Behav.
(2002) - et al.
The role of the human ventral striatum and the medial orbitofrontal cortex in the representation of reward magnitude: An activation likelihood estimation meta-analysis of neuroimaging studies of passive reward expectancy and outcome processing
Neuropsychologia
(2012) - et al.
Activation likelihood estimation meta-analysis revisited
NeuroImage
(2012) - et al.
Beyond the single study: function/location metanalysis in cognitive neuroimaging
Curr. Opin. Neurobiol.
(1998) - et al.
Value, pleasure and choice in the ventral prefrontal cortex
Trends Cogn. Sci.
(2011) - et al.
The neurobiology of decision: consensus and controversy
Neuron
(2009) - et al.
Neural predictors of purchases
Neuron
(2007) - et al.
The neural basis of financial risk taking
Neuron
(2005)
The bivalent side of the nucleus accumbens
NeuroImage
Neural economics and the biological substrates of valuation
Neuron
Reward representations and reward-related learning in the human brain: insights from neuroimaging
Curr. Opin. Neurobiol.
Neural representations of subjective reward value
Behav. Brain Res.
Functional imaging of brain responses to pain: a review and meta-analysis (2000)
Clin. Neurophysiol.
Can cognitive processes be inferred from neuroimaging data?
Trends Cogn. Sci.
Neural differentiation of expected reward and risk in human subcortical structures
Neuron
Neural computations associated with goal-directed choice
Curr. Opin. Neurobiol.
Meta-analysis of neuroimaging data: a comparison of image-based and coordinate-based pooling of studies
NeuroImage
Being right is its own reward: Load and performance related ventral striatum activation to correct responses during a working memory task in youth
NeuroImage
Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference
NeuroImage
Advances in functional and structural MR image analysis and implementation as FSL
NeuroImage
Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decision-making task
NeuroImage
Meta-analysis of the functional neuroanatomy of single-word reading: method and validation
NeuroImage
Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain
NeuroImage
Putting a spin on the dorsal–ventral divide of the striatum
Trends Neurosci.
Cognitive neuroscience 2.0: building a cumulative science of human brain function
Trends Cogn. Sci.
Human striatal responses to monetary reward depend on saliency
Neuron
Neuromarketing: the hope and hype of neuroimaging in business
Nat. Rev. Neurosci.
Adolescents, adults and rewards: comparing motivational neurocircuitry recruitment using fMRI
PLoS One
Human medial orbitofrontal cortex is recruited during experience of imagined and real rewards
J. Neurophysiol.
Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain
IEEE Trans. Med. Imaging
Neuroeconomics: how neuroscience can inform economics
J. Econ. Lit.
Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex
J. Neurosci.
Neural correlates of value, risk, and risk aversion contributing to decision making under risk
J. Neurosci.
The neural basis of altruistic punishment
Science
Reward-related responses in the human striatum
Ann. N. Y. Acad. Sci.
Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers
Front. Neurosci.
Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: a random-effects approach based on empirical estimates of spatial uncertainty
Hum. Brain Mapp.
Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold
Magn. Reson. Med.
Cited by (0)
- 1
All authors contributed equally.