Abstract
Despite widespread interest in neural mechanisms of decision-making, most investigations focus on decisions between just two options. Here we adapt a biophysically plausible model of decision-making to predict how a key decision variable, the value difference signal—encoding how much better one choice is than another—changes with the value of a third, but unavailable, alternative. The model predicts a surprising failure of optimal decision-making: greater difficulty choosing between two options in the presence of a third very poor, as opposed to very good, alternative. Both investigation of human decision-making and functional magnetic resonance imaging–based measurements of value difference signals in ventromedial prefrontal cortex (vmPFC) bore out this prediction. The vmPFC signal decreased in the presence of low-value third alternatives, and vmPFC effect sizes predicted individual variation in suboptimal decision-making in the presence of multiple alternatives. The effect contrasts with that of divisive normalization in parietal cortex.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Vickery, T.J., Chun, M.M. & Lee, D. Ubiquity and Specificity of Reinforcement Signals throughout the Human Brain. Neuron 72, 166–177 (2011).
Noonan, M.P. et al. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl. Acad. Sci. USA 107, 20547–20552 (2010).
Hunt, L.T. et al. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476 (2012).
Wang, X.J. Neural dynamics and circuit mechanisms of decision-making. Curr. Opin. Neurobiol. 22, 1039–1046 (2012).
Boorman, E.D., Behrens, T.E., Woolrich, M.W. & Rushworth, M.F. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733–743 (2009).
Camille, N., Griffiths, C.A., Vo, K., Fellows, L.K. & Kable, J.W. Ventromedial frontal lobe damage disrupts value maximization in humans. J. Neurosci. 31, 7527–7532 (2011).
Rangel, A., Camerer, C. & Montague, P.R. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9, 545–556 (2008).
Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009).
Rushworth, M.F., Noonan, M.P., Boorman, E.D., Walton, M.E. & Behrens, T.E. Frontal cortex and reward-guided learning and decision-making. Neuron 70, 1054–1069 (2011).
Bouret, S. & Richmond, B.J. Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. J. Neurosci. 30, 8591–8601 (2010).
Monosov, I.E. & Hikosaka, O. Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex. J. Neurosci. 32, 10318–10330 (2012).
Lebreton, M., Jorge, S., Michel, V., Thirion, B. & Pessiglione, M. An automatic valuation system in the human brain: evidence from functional neuroimaging. Neuron 64, 431–439 (2009).
Kable, J.W. & Glimcher, P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625–1633 (2007).
Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).
Louie, K. & Glimcher, P.W. Separating value from choice: delay discounting activity in the lateral intraparietal area. J. Neurosci. 30, 5498–5507 (2010).
FitzGerald, T.H., Seymour, B. & Dolan, R.J. The role of human orbitofrontal cortex in value comparison for incommensurable objects. J. Neurosci. 29, 8388–8395 (2009).
Philiastides, M.G., Biele, G. & Heekeren, H.R. A mechanistic account of value computation in the human brain. Proc. Natl. Acad. Sci. USA 107, 9430–9435 (2010).
Kolling, N., Behrens, T.E., Mars, R.B. & Rushworth, M.F. Neural mechanisms of foraging. Science 336, 95–98 (2012).
De Martino, B., Fleming, S.M., Garrett, N. & Dolan, R.J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).
Hare, T.A., Schultz, W., Camerer, C.F., O'Doherty, J.P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl. Acad. Sci. USA 108, 18120–18125 (2011).
Summerfield, C. & Tsetsos, K. Building bridges between perceptual and economic decision-making: neural and computational mechanisms. Front. Neurosci. 6, 70 (2012).
Churchland, A.K., Kiani, R. & Shadlen, M.N. Decision-making with multiple alternatives. Nat. Neurosci. 11, 693–702 (2008).
Usher, M. & McClelland, J.L. The time course of perceptual choice: the leaky, competing accumulator model. Psychol. Rev. 108, 550–592 (2001).
Wang, X.J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).
Mars, R.B. et al. Diffusion-weighted imaging tractography-based parcellation of the human parietal cortex and comparison with human and macaque resting-state functional connectivity. J. Neurosci. 31, 4087–4100 (2011).
Gould, I.C., Nobre, A.C., Wyart, V. & Rushworth, M.F. Effects of decision variables and intraparietal stimulation on sensorimotor oscillatory activity in the human brain. J. Neurosci. 32, 13805–13818 (2012).
Louie, K., Grattan, L.E. & Glimcher, P.W. Reward value-based gain control: divisive normalization in parietal cortex. J. Neurosci. 31, 10627–10639 (2011).
Jocham, G., Hunt, L.T., Near, J. & Behrens, T.E. A mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex. Nat. Neurosci. 15, 960–961 (2012).
Wunderlich, K., Dayan, P. & Dolan, R.J. Mapping value based planning and extensively trained choice in the human brain. Nat. Neurosci. 15, 786–791 (2012).
Lim, S.L., O'Doherty, J.P. & Rangel, A. The decision value computations in the vmPFC and striatum use a relative value code that is guided by visual attention. J. Neurosci. 31, 13214–13223 (2011).
Rudebeck, P.H. & Murray, E.A. Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 31, 10569–10578 (2011).
Walton, M.E., Behrens, T.E., Buckley, M.J., Rudebeck, P.H. & Rushworth, M.F. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65, 927–939 (2010).
Noonan, M.P., Mars, R.B. & Rushworth, M.F. Distinct roles of three frontal cortical areas in reward-guided behavior. J. Neurosci. 31, 14399–14412 (2011).
Fellows, L.K. Deciding how to decide: ventromedial frontal lobe damage affects information acquisition in multi-attribute decision making. Brain 129, 944–952 (2006).
Lee, J. & Maunsell, J.H. A normalization model of attentional modulation of single unit responses. PLoS ONE 4, e4651 (2009).
Lee, J. & Maunsell, J.H. Attentional modulation of MT neurons with single or multiple stimuli in their receptive fields. J. Neurosci. 30, 3058–3066 (2010).
Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Choosing the greater of two goods: neural currencies for valuation and decision making. Nat. Rev. Neurosci. 6, 363–375 (2005).
Padoa-Schioppa, C. Range-adapting representation of economic value in the orbitofrontal cortex. J. Neurosci. 29, 14004–14014 (2009).
Palminteri, S., Boraud, T., Lafargue, G., Dubois, B. & Pessiglione, M. Brain hemispheres selectively track the expected value of contralateral options. J. Neurosci. 29, 13465–13472 (2009).
Louie, K., Khaw, M.W. & Glimcher, P.W. Normalization is a general neural mechanism for context-dependent decision making. Proc. Natl. Acad. Sci. USA 110, 6139–6144 (2013).
Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010).
Soltani, A. & Wang, X.J. A biophysically based neural model of matching law behavior: melioration by stochastic synapses. J. Neurosci. 26, 3731–3744 (2006).
Ashbridge, E., Walsh, V. & Cowey, A. Temporal aspects of visual search studied by transcranial magnetic stimulation. Neuropsychologia 35, 1121–1131 (1997).
Sereno, A.B. & Maunsell, J.H.R. Shape selectivity in primate lateral intraparietal cortex. Nature 395, 500–503 (1998).
Grol, M.J., de Lange, F.P., Verstraten, F.A., Passingham, R.E. & Toni, I. Cerebral changes during performance of overlearned arbitrary visuomotor associations. J. Neurosci. 26, 117–125 (2006).
Noonan, M.P., Kolling, N., Walton, M.E. & Rushworth, M.F. Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. Eur. J. Neurosci. 35, 997–1010 (2012).
Wallis, J.D. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 15, 13–19 (2012).
Kaping, D., Vinck, M., Hutchison, R.M., Everling, S. & Womelsdorf, T. Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation. PLoS Biol. 9, e1001224 (2011).
Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009).
Simon, H.A. Rational choice and the structure of the environment. Psychol. Rev. 63, 129–138 (1956).
Deichmann, R., Gottfried, J.A., Hutton, C. & Turner, R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage 19, 430–441 (2003).
Smith, S.M. et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23 (suppl. 1): S208–S219 (2004).
Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17, 825–841 (2002).
Jenkinson, M. Fast, automated, N-dimensional phase-unwrapping algorithm. Magn. Reson. Med. 49, 193–197 (2003).
Jenkinson, M. & Smith, S. A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5, 143–156 (2001).
Beckmann, C.F., Jenkinson, M. & Smith, S.M. General multilevel linear modeling for group analysis in FMRI. Neuroimage 20, 1052–1063 (2003).
Woolrich, M.W., Behrens, T.E., Beckmann, C.F., Jenkinson, M. & Smith, S.M. Multilevel linear modelling for FMRI group analysis using Bayesian inference. Neuroimage 21, 1732–1747 (2004).
Woolrich, M. Robust group analysis using outlier inference. Neuroimage 41, 286–301 (2008).
Worsley, K.J., Evans, A.C., Marrett, S. & Neelin, P. A three-dimensional statistical analysis for CBF activation studies in human brain. J. Cereb. Blood Flow Metab. 12, 900–918 (1992).
Friston, K.J. et al. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6, 218–229 (1997).
Acknowledgements
This research was funded by the Wellcome Trust and UK Medical Research Council.
Author information
Authors and Affiliations
Contributions
Experimental design: B.K.H.C., M.E.W. and M.F.S.R. Behavioral and MRI data collection and analysis: B.K.H.C. Biophysical modeling: N.K. and L.T.H. Manuscript preparation: B.K.H.C. and M.F.S.R.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Biophysical model predictions for HV-D effect on PHV – PLV in all trials
Biophysical model prediction of difference in activity in HV and LV populations, PHV-PLV, as a function of HV-LV and HV-D regressors. The effect size is the regression coefficient (β weight) of HV-D. This analysis was performed by including all trials of the task. A similar analysis that included only difficult trials (when HV-LV was small) is shown in fig.1c.
Supplementary Figure 2 Complementary analysis of network signal and inhibitory population activity
Biophysical model prediction of difference in activity in HV and LV populations, (a) PHV-PLV, as a function of HV-LV and D regressors and (b) of Pi activity as a function of HV-LV and D regressors in all trials. The effect size is the regression coefficient (β weight) of D.
Supplementary Figure 3 The effect of HV – D on human behavior was specific to distractor trials
Here we present a more extensive analysis of performance in the distractor and two choice tasks. When we analyzed data from both the two-option and distractor trials we could see that subjects made more mistakes (chose HV less) as HV-LV decreased and when HV-D was large (suggesting it was more difficult to choose HV when the distractor had a much lower value). In this analysis, we added an additional binary regressor, Congruity, that describes whether both the HV option's reward magnitude component and the HV option's reward probability component were larger than the LV option's reward magnitude and reward probability components. In other words, it tested whether both components of the HV option were congruent with one another in suggesting HV was the better option. Choices were more accurate when there was congruency. Similar effects were also present when (a) we focused on analyzing the data from just the distractor trials. HV-D had a marginally negative impact on accuracy but the clearly significantly positive (HV-LV)(HV-D) interaction term indicated the HV-D effect was stronger when HV-LV was smaller. (b) When we analyzed the data from just the two-option trials using notional distractor values obtained from the matched distractor trials the results showed that the subjects' choices were not affected by the value of the notional distractor.
Supplementary Figure 4 The effect of HV – D on human behavior was caused by the value of the distractor
We calculated an alternative GLM using D as a regressor instead of HV-D, in cases of distractor trials when HV-LV was small (difficult trials) and when HV-LV was large (easy trials). On difficult trials (blue), we observed a significant impact not just of HV-LV on choice accuracy but also an impact of D which confirmed that when distractors were low in value (corresponding to the situation where HV-D is large) then subjects made less accurate choices. However, as the initial regression models had suggested the effect disappeared when decisions were easy and we looked at accuracy on trials where the HV-LV difference was large (red). Once again a main effect of HV-LV was found but there was no effect of D.
Supplementary Figure 5 The effect of HV – D was not dependent on the value rank of the distractor
We considered whether the negative effect of HV-D could be confounded by the value rank of the distractor. It is possible that when D had the highest value, the subject had to engage more inhibition to avoid choosing D and made more errors. (a) To test for this, we split the HV-D regressor by whether the distractor was highest or lowest in value. The results of this general linear model were comparable to that reported in our main manuscript. We found a positive effect of HV-LV, a negative effect of HV-D when distractor was highest in value, a negative effect of HV-D when distractor was lowest in value and a positive effect of (HV-LV)(HV-D). No significant difference was found between the two HV-D regressors. (b) We also performed another analysis in a similar spirit that, instead of splitting HV-D into two regressors, used an additional regressor that described the rank order of the value of D. Similar results were found in this model: a positive effect of HV-LV, a negative effect of HV-D and a positive effect of (HV-LV)(HV-D).
Supplementary Figure 6 Pilot human behavioral experiments
Prior to the current experiment, we ran three pilot studies outside the scanner using similar task paradigms which mainly differ from the current paradigm by trials with different duration component phases. In Study 1 and 2, the initial phase lasted 1s whereas there was no initial phase in Study 3 (i.e., the distractor was presented immediately at trial onset). In all three studies, the interval phase was 2s, outcome phase 1.5s, and inter-trial interval 3s. (Blue) In Study 1, which involved twelve subjects performing 100 two-option trials and 100 distractor trials, decisions had to be made within 0.8 s after the stimuli were presented. In this design, we kept the shared variance between HV, LV, D, HV-LV and HV-D under 0.25. We found similar behavioral effects: the subjects were less accurate for lower HV-LV and higher HV-D. (Red) Study 2 involved seven subjects performing a similar task (deadline for decision was at 1s), except now with 200 trials for each of the two types of trials. Consistently, there was a significant positive HV-LV effect and negative HV-D effect. (Green) Finally, in Study 3 (three subjects) distractor identity was indicated at the beginning of the trial. Again, we found a positive effect of HV-LV and a negative effect of HV-D (regression coefficients were log transformed to account for high variance due to one subject having a larger effect size) in difficult trials.
Supplementary Figure 7 Human behavior in a three-option task
In order to test whether the “distractor effects” on human behavior and on the biophysical model were dependent on whether the third option could be chosen or not, we also performed a study with twelve subjects in which all three options were available choices. The structure of this task was similar to that in our fMRI study; after the initial presentation of the three options for 1 s (initial phase), subjects were allowed to choose one of all three options within 4 s (decision phase). As in the three-option biophysical model, here an option with the lowest value in a trial was considered as a distractor. We excluded trials when D was chosen because it would not be surprising to see D itself being chosen more often when its value increased, and we were more interested in investigating whether the accuracy of choosing HV relative to choosing LV would change when D's value decreased; this analysis is analogous to those in our main manuscript that examined accuracy of HV choice relative to LV choice when D could not be chosen. (a) Subjects were more accurate in choosing HV as HV-LV increased (from bottom to top in the figure; similar to the positive HV-LV effect in the main manuscript). More mistakes were made by choosing LV when HV-D was large (from left to right in the figure; similar to the negative HV-D effect in the “distractor task” in the main manuscript). (b,c) We confirmed these results by performing logistic regression analyses.
Supplementary Figure 8 Two kinds of suboptimal choice effects caused by distractor value in a three-option task
To confirm that a third option affected subjects' decision as a function of both its individual value and its relative value in relation to other options we included the following regressors in a GLM to predict HV choice: HV, LV, D (for examining individual value effects) and the HV×D, LV×D interaction terms (for examining relative value effects). We found a positive impact of HV and a negative impact of LV on choosing the best option, which was similar to the positive HV-LV effects in SI.7 and in the “distractor task” in our main manuscript. We also found a negative impact of D, confirming the first kind of third option effect (individual value effect): increasing D's value makes it harder to choose the best option due to more D choices. Finally, the HV×D and LV×D interaction terms had a negative effect and a positive effect on choosing the best option respectively. This shows that by decreasing D value, it was harder to choose the best option in situations when HV had a low value and when LV had a high value. In other words, it was harder to choose the best option for increasing HV-D when HV-LV was small, due to more LV choices as suggested by the previous analysis using value difference regressors, this confirms the second kind of third option effect (relative value effect). This second phenomenon was analogous to the positive (HV-LV)(HV-D) effect reported in our main manuscript (i.e. it was harder to choose the better option when HV-LV was small and distractor value was small).
Supplementary Figure 9 Biophysical model predictions for correct difficult trials
A similar impact of HV-D (a) on the firing difference between HV and LV populations, PHV and PLV, and (b) on inhibitory firing PI could also be seen when we analyzed only difficult trials on which the biophysical model “selected” the correct option.
Supplementary Figure 10 HV – LV signal was not modulated by values of HV or HV + LV
(a-d) When the HV-LV value difference signals were binned as a function of the value of HV, there was no significant change in effect size in either type of trial in vmPFC or MIP. (e-h) Similarly, when we binned the HV-LV value difference signals as a function of the overall option value (HV+LV), there were no significant changes in effect size in either type of trial in vmPFC or MIP.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–10 and Supplementary Modeling (PDF 2047 kb)
Rights and permissions
About this article
Cite this article
Chau, B., Kolling, N., Hunt, L. et al. A neural mechanism underlying failure of optimal choice with multiple alternatives. Nat Neurosci 17, 463–470 (2014). https://doi.org/10.1038/nn.3649
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3649
This article is cited by
-
Activation and disruption of a neural mechanism for novel choice in monkeys
Nature (2021)
-
Global optimization based on active preference learning with radial basis functions
Machine Learning (2021)
-
Divisive normalization does influence decisions with multiple alternatives
Nature Human Behaviour (2020)
-
Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex
Nature Human Behaviour (2020)
-
The impact of phantom decoys on the neural processing of valuation
Brain Structure and Function (2020)