Elsevier

NeuroImage

Volume 76, 1 August 2013, Pages 412-427
NeuroImage

The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value

https://doi.org/10.1016/j.neuroimage.2013.02.063Get rights and content

Highlights

  • Theories of decision making posit a domain-general subjective value signal.

  • Numerous fMRI studies have tested for correlates of such a signal.

  • We quantitatively meta-analyze findings from 206 studies.

  • We identify a “valuation system” in ventromedial PFC and ventral striatum.

  • Other regions show a mixture of positive and negative effects across experiments.

Abstract

Numerous experiments have recently sought to identify neural signals associated with the subjective value (SV) of choice alternatives. Theoretically, SV assessment is an intermediate computational step during decision making, in which alternatives are placed on a common scale to facilitate value-maximizing choice. Here we present a quantitative, coordinate-based meta-analysis of 206 published fMRI studies investigating neural correlates of SV. Our results identify two general patterns of SV-correlated brain responses. In one set of regions, both positive and negative effects of SV on BOLD are reported at above-chance rates across the literature. Areas exhibiting this pattern include anterior insula, dorsomedial prefrontal cortex, dorsal and posterior striatum, and thalamus. The mixture of positive and negative effects potentially reflects an underlying U-shaped function, indicative of signal related to arousal or salience. In a second set of areas, including ventromedial prefrontal cortex and anterior ventral striatum, positive effects predominate. Positive effects in the latter regions are seen both when a decision is confronted and when an outcome is delivered, as well as for both monetary and primary rewards. These regions appear to constitute a “valuation system,” carrying a domain-general SV signal and potentially contributing to value-based decision making.

Introduction

Many psychological and economic theories of decision making are couched within the framework of value maximization (e.g., Kahneman and Tversky, 1979, Samuelson, 1937). According to such a framework, an organism facing a choice first determines the scalar subjective value (SV) of each available alternative, and then selects the alternative with the greatest SV. SV serves as a common currency, allowing complex and qualitatively different alternatives to be compared on a common scale. Stable assessments of SV can support consistent decision behavior over time and across contexts.

Although there are noted and well-studied anomalies (see, e.g., Tversky and Kahneman, 1974), value maximization has been an enormously productive framework for understanding learning and decision making. As such, a major objective in decision neuroscience has been to uncover neural signals associated with the computation and representation of SV. A growing number of studies have used fMRI to measure blood oxygenation level dependent (BOLD) signal from the brain while human participants perform evaluation and decision-making tasks, comparing BOLD response amplitudes across outcomes or prospects that differ in SV. Qualitative reviews of this work (e.g., Delgado, 2007, Grabenhorst and Rolls, 2011, Kable and Glimcher, 2009, Knutson and Cooper, 2005, Levy and Glimcher, 2012, Montague and Berns, 2002, O'Doherty, 2004, Peters and Büchel, 2010) have concluded that: (1) BOLD signal in a small number of regions, including ventral striatum (VS), ventromedial prefrontal cortex (VMPFC) and (in some cases) posterior cingulate cortex (PCC) scales with the SV of the available alternatives during choice; (2) the same regions also respond when a reward is received, implicating a common set of regions in the evaluation of both prospects and outcomes; and (3) these regions respond to outcomes in multiple domains (e.g., primary rewards such as food, secondary rewards such as money, social rewards, etc.), as would be required for any domain-general common currency signal.

Though many studies support this consensus, there remains a risk that qualitative reviews have either overestimated the support for prevailing hypotheses or overlooked unanticipated but reliable findings. Such selectivity certainly can occur in individual studies, which may focus discussion on regions identified a priori while ignoring equally strong results elsewhere. In the present work we use meta-analytic techniques to quantify the state of the evidence for the above three claims. Quantitative meta-analysis offers a complementary perspective to well-designed individual experiments and qualitative reviews, allowing a large and diverse array of existing findings to be aggregated, regardless of how each experiment was originally interpreted.

We perform a series of coordinate-based meta-analyses of published fMRI findings relating to the neural correlates of SV. Our examination focuses on four specific questions.

  • Question 1: In what regions does BOLD correlate positively or negatively with SV? Our first step tests for reliable BOLD correlates of SV. Previous reviews have tended to emphasize BOLD responses that scale positively with SV. At the single neuron level, however, there seem to be roughly equal numbers of neurons that increase or decrease firing as SV changes (Kennerley et al., 2009). There have been preliminary suggestions, from both fMRI and electrophysiological studies, that these positively and negatively directed responses may be anatomically dissociable (Monosov and Hikosaka, 2012, O'Doherty et al., 2001).

  • Question 2: Do neural responses follow a linear or nonlinear function of SV? A second question is whether neural correlates of SV can be distinguished from potentially related signals of arousal or salience (cf. Zink et al., 2004). Fig. 1 illustrates two possible ways in which a neural response might vary across different levels of SV. Pattern A shows a response rising linearly with SV, while Pattern B shows a U-shaped response that is maximal either for extreme positive or extreme negative values. The linear pattern is more properly a correlate of SV, while the U-shaped pattern might be better described as a signal of arousal or salience. A source of difficulty in disambiguating these patterns is that their predictions overlap in experiments that only involve rewards. Neural structures that appear reward-responsive in such experiments may also respond strongly to penalties (Carretié et al., 2009, Levita et al., 2009). Previous work suggests it may be possible to distinguish areas that show either or both of the activity profiles in Fig. 1 (Litt et al., 2011).

  • Question 3: Do the same regions encode SV when a decision is made and when an outcome is experienced? We next ask whether the neural correlates of SV during decision-making are similar or different from SV responses during the receipt of an outcome (cf. “decision utility” vs. “experienced utility”; Kahneman et al., 1997). Numerous experiments have demonstrated that neural correlates of SV can indeed be identified during decision making (Kable and Glimcher, 2007, Knutson et al., 2007, Plassmann et al., 2007, Tom et al., 2007), and the same general regions are active when rewards are experienced. However, there is also evidence that neural responses during choice and during outcome delivery may be anatomically dissociable (Knutson et al., 2001b, Smith et al., 2010).

  • Question 4: Is SV encoded differently for primary versus secondary incentives? An important question is to what extent the neural representation of SV is domain-specific versus domain-general. Many studies targeting this question have investigated whether secondary rewards (e.g., money) are evaluated using the same neural architecture as primary rewards (e.g., juice). A number of studies have concluded that different categories of rewards recruit at least partially overlapping neural substrates (Chib et al., 2009, Kim et al., 2011, Valentin and O'Doherty, 2009). Partial overlap has also been noted for primary and secondary aversive outcomes (Delgado et al., 2011). Other work has reached the conclusion that neural mechanisms involved in evaluating primary and secondary incentives are dissociable (Sescousse et al., 2010).

Previous meta-analyses have examined subsets of the above questions in more restricted databases. Both Knutson and Greer (2008) and Diekhof et al. (2012) focused on the passive anticipation of rewards, comparing SV responses associated with outcome anticipation and delivery. Peters and Büchel (2010) and Levy and Glimcher (2012) presented anatomically circumscribed meta-analyses in VMPFC and adjacent regions, showing overlap between activation coordinates associated with multiple forms of SV. The present investigation extends previous work by examining all four of the above questions simultaneously across the entire brain in a large corpus of published studies. Our sample size affords us the statistical power to conduct comprehensive and rigorous inferential tests at the meta-analytic level.

Across the present meta-analyses, we find evidence for reliable positive correlates of SV in VMPFC and anterior VS. We find that these can be distinguished from responses in anterior insula, other striatal regions, and dorsomedial prefrontal cortex (DMPFC), which seem more related to arousal or salience. We find overlapping responses in VMPFC and VS to decision and experienced SV, as well as overlapping responses to both primary and secondary incentives. These results bolster the conclusion that VMPFC and VS form the core of a “valuation system” believed to play a critical role in value-based learning and decision-making.

Section snippets

Coordinate-based meta-analysis

The importance of quantitatively synthesizing the findings of independent experiments is well recognized across scientific domains. Quantitative meta-analysis poses unique challenges in the context of neuroimaging. Neuroimaging results are conventionally reported in terms of the anatomical location of peak statistical effects, expressed using a stereotactic system such as Talairach (Talairach and Tournoux, 1988) or Montreal Neurological Institute (MNI) coordinate space.

This reporting convention

Whole-brain meta-analysis

We first examine overall associations between SV and BOLD. Here we organize experimental findings solely according to the direction of the reported effects (positive or negative), aggregating across other attributes. An effect was coded as positive if a greater BOLD response was observed for more rewarding (or less aversive) outcomes; it was coded as negative if a greater BOLD response was observed for less rewarding (or more aversive) outcomes. Our corpus contains 200 studies that report

Directionality and effect of valence

In the present paper we report a series of meta-analyses examining neural effects of SV as measured using fMRI. Our first step was to compare the locations of positive and negative effects of SV on BOLD. Positive effects consist of larger BOLD responses to more rewarding or less aversive outcomes; negative effects consist of larger BOLD responses to less rewarding or more aversive outcomes. We observed a greater density of positive than negative effects in the VMPFC, PCC, and striatum. We

Acknowledgments

This work was supported by the National Institutes of Health (grant number R01-DA029149 to J.W.K., and F32-DA030870 to J.T.M.); and by a fellowship from the University of Pennsylvania Positive Psychology Center to O.B.C.

References (80)

  • L. Levita et al.

    The bivalent side of the nucleus accumbens

    NeuroImage

    (2009)
  • P.R. Montague et al.

    Neural economics and the biological substrates of valuation

    Neuron

    (2002)
  • J.P. O'Doherty

    Reward representations and reward-related learning in the human brain: insights from neuroimaging

    Curr. Opin. Neurobiol.

    (2004)
  • J. Peters et al.

    Neural representations of subjective reward value

    Behav. Brain Res.

    (2010)
  • R. Peyron et al.

    Functional imaging of brain responses to pain: a review and meta-analysis (2000)

    Clin. Neurophysiol.

    (2000)
  • R.A. Poldrack

    Can cognitive processes be inferred from neuroimaging data?

    Trends Cogn. Sci.

    (2006)
  • K. Preuschoff et al.

    Neural differentiation of expected reward and risk in human subcortical structures

    Neuron

    (2006)
  • A. Rangel et al.

    Neural computations associated with goal-directed choice

    Curr. Opin. Neurobiol.

    (2010)
  • G. Salimi-Khorshidi et al.

    Meta-analysis of neuroimaging data: a comparison of image-based and coordinate-based pooling of studies

    NeuroImage

    (2009)
  • T.D. Satterthwaite et al.

    Being right is its own reward: Load and performance related ventral striatum activation to correct responses during a working memory task in youth

    NeuroImage

    (2012)
  • S.M. Smith et al.

    Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference

    NeuroImage

    (2009)
  • S.M. Smith et al.

    Advances in functional and structural MR image analysis and implementation as FSL

    NeuroImage

    (2004)
  • B.W. Smith et al.

    Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decision-making task

    NeuroImage

    (2009)
  • P.E. Turkeltaub et al.

    Meta-analysis of the functional neuroanatomy of single-word reading: method and validation

    NeuroImage

    (2002)
  • N. Tzourio-Mazoyer et al.

    Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain

    NeuroImage

    (2002)
  • P. Voorn et al.

    Putting a spin on the dorsal–ventral divide of the striatum

    Trends Neurosci.

    (2004)
  • T. Yarkoni et al.

    Cognitive neuroscience 2.0: building a cumulative science of human brain function

    Trends Cogn. Sci.

    (2010)
  • C.F. Zink et al.

    Human striatal responses to monetary reward depend on saliency

    Neuron

    (2004)
  • D. Ariely et al.

    Neuromarketing: the hope and hype of neuroimaging in business

    Nat. Rev. Neurosci.

    (2010)
  • J.M. Bjork et al.

    Adolescents, adults and rewards: comparing motivational neurocircuitry recruitment using fMRI

    PLoS One

    (2010)
  • S. Bray et al.

    Human medial orbitofrontal cortex is recruited during experience of imagined and real rewards

    J. Neurophysiol.

    (2010)
  • E.T. Bullmore et al.

    Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain

    IEEE Trans. Med. Imaging

    (1999)
  • C. Camerer et al.

    Neuroeconomics: how neuroscience can inform economics

    J. Econ. Lit.

    (2005)
  • V.S. Chib et al.

    Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex

    J. Neurosci.

    (2009)
  • G.I. Christopoulos et al.

    Neural correlates of value, risk, and risk aversion contributing to decision making under risk

    J. Neurosci.

    (2009)
  • D.J.-F. de Quervain et al.

    The neural basis of altruistic punishment

    Science

    (2004)
  • M.R. Delgado

    Reward-related responses in the human striatum

    Ann. N. Y. Acad. Sci.

    (2007)
  • M.R. Delgado et al.

    Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers

    Front. Neurosci.

    (2011)
  • S.B. Eickhoff et al.

    Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: a random-effects approach based on empirical estimates of spatial uncertainty

    Hum. Brain Mapp.

    (2009)
  • S.D. Forman et al.

    Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold

    Magn. Reson. Med.

    (1995)
  • Cited by (0)

    1

    All authors contributed equally.

    View full text