Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex

Nature Neuroscience volume 14, pages 1590–1597 (2011)Cite this article

Subjects

Abstract

The orbitofrontal cortex has been hypothesized to carry information regarding the value of expected rewards. Such information is essential for associative learning, which relies on comparisons between expected and obtained reward for generating instructive error signals. These error signals are thought to be conveyed by dopamine neurons. To test whether orbitofrontal cortex contributes to these error signals, we recorded from dopamine neurons in orbitofrontal-lesioned rats performing a reward learning task. Lesions caused marked changes in dopaminergic error signaling. However, the effect of lesions was not consistent with a simple loss of information regarding expected value. Instead, without orbitofrontal input, dopaminergic error signals failed to reflect internal information about the impending response that distinguished externally similar states leading to differently valued future rewards. These results are consistent with current conceptualizations of orbitofrontal cortex as supporting model-based behavior and suggest an unexpected role for this information in dopaminergic error signaling.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Apparatus and behavioral results.
Figure 2: Identification, waveform features and firing rates of putative dopamine and non-dopamine neurons.
Figure 3: Changes in activity of reward-responsive dopamine neurons in response to unexpected reward delivery and omission.
Figure 4: Changes in activity of reward-responsive dopamine neurons during and after odor cue sampling on forced- and free-choice trials.
Figure 5: Changes in dopamine neuron activity in response to OFC stimulation.
Figure 6: Comparison of model simulations and experimental data.

Similar content being viewed by others

References

  1. Hollerman, J.R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304–309 (1998).

    Article  CAS  Google Scholar 

  2. Pan, W.-X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

    Article  CAS  Google Scholar 

  3. Bayer, H.M. & Glimcher, P. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  4. Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction. (MIT Press, 1998).

    Google Scholar 

  5. Pearce, J.M. & Hall, G. A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532–552 (1980).

    Article  CAS  Google Scholar 

  6. Rescorla, R.A. & Wagner, A.R. in Classical Conditioning II: Current Research and Theory (eds. Black, A.H. & Prokasy, W.F.) 64–99 (Appleton-Century-Crofts, 1972).

  7. Pickens, C.L. et al. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci. 23, 11078–11084 (2003).

    Article  CAS  Google Scholar 

  8. Izquierdo, A., Suda, R.K. & Murray, E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540–7548 (2004).

    Article  CAS  Google Scholar 

  9. O'Doherty, J.P., Deichmann, R., Critchley, H.D. & Dolan, R.J. Neural responses during anticipation of a primary taste reward. Neuron 33, 815–826 (2002).

    Article  CAS  Google Scholar 

  10. Gottfried, J.A., O′Doherty, J. & Dolan, R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104–1107 (2003).

    Article  CAS  Google Scholar 

  11. Padoa-Schioppa, C. & Assad, J.A. Neurons in orbitofrontal cortex encode economic value. Nature 441, 223–226 (2006).

    Article  CAS  Google Scholar 

  12. Roesch, M.R., Taylor, A.R. & Schoenbaum, G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509–520 (2006).

    Article  CAS  Google Scholar 

  13. Takahashi, Y.K. et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62, 269–280 (2009).

    Article  CAS  Google Scholar 

  14. Vázquez-Borsetti, P., Cortes, R. & Artigas, F. Pyramidal neurons in rat prefrontal cortex projecting to ventral tegmental area and dorsal raphe nucleus express 5–HT2A receptors. Cereb. Cortex 19, 1678–1686 (2009).

    Article  Google Scholar 

  15. Roesch, M.R., Calu, D.J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615–1624 (2007).

    Article  CAS  Google Scholar 

  16. Margolis, E.B., Lock, H., Hjelmstad, G.O. & Fields, H.L. The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J. Physiol. (Lond.) 577, 907–924 (2006).

    Article  CAS  Google Scholar 

  17. Jin, X. & Costa, R.M. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457–462 (2010).

    Article  CAS  Google Scholar 

  18. Grace, A.A. & Bunney, B.S. The control of firing pattern in nigral dopamine neurons: burst firing. J. Neurosci. 4, 2877–2890 (1984).

    Article  CAS  Google Scholar 

  19. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate for prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  20. Schoenbaum, G., Roesch, M.R., Stalnaker, T.A. & Takahashi, Y.K. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885–892 (2009).

    Article  CAS  Google Scholar 

  21. Sul, J.H., Kim, H., Huh, N., Lee, D. & Jung, M.W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449–460 (2010).

    Article  CAS  Google Scholar 

  22. Chudasama, Y. & Robbins, T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 23, 8771–8780 (2003).

    Article  CAS  Google Scholar 

  23. Fellows, L.K. & Farah, M.J. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 126, 1830–1837 (2003).

    Article  Google Scholar 

  24. Gershman, S.J. & Niv, Y. Learning latent structure: carving nature at its joints. Curr. Opin. Neurobiol. 20, 251–256 (2010).

    Article  CAS  Google Scholar 

  25. Redish, A.D., Jensen, S., Johnson, A. & Kurth-Nelson, Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol. Rev. 114, 784–805 (2007).

    Article  Google Scholar 

  26. Gershman, S.J., Blei, D.M. & Niv, Y. Time, context and extinction. Psychol. Rev. 117, 197–209 (2010).

    Article  Google Scholar 

  27. Ramus, S.J. & Eichenbaum, H. Neural correlates of olfactory recognition memory in the rat orbitofrontal cortex. J. Neurosci. 20, 8199–8208 (2000).

    Article  CAS  Google Scholar 

  28. van Duuren, E., Lankelma, J. & Pennartz, C.M.A. Population coding of reward magnitude in the orbitofrontal cortex of the rat. J. Neurosci. 28, 8590–8603 (2008).

    Article  CAS  Google Scholar 

  29. van Duuren, E. et al. Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat. J. Neurosci. 29, 8965–8976 (2009).

    Article  CAS  Google Scholar 

  30. Walton, M.E., Behrens, T.E.J., Buckley, M.J., Rudebeck, P.H. & Rushworth, M.F.S. Separable learning systems in the macaque brain and the role of the orbitofrontal cortex in contingent learning. Neuron 65, 927–939 (2010).

    Article  CAS  Google Scholar 

  31. Tsuchida, A., Doll, B.B. & Fellows, L.K. Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J. Neurosci. 30, 16868–16875 (2010).

    Article  CAS  Google Scholar 

  32. Tsujimoto, S., Genovesio, A. & Wise, S.P. Monkey orbitofrontal cortex encodes response choices near feedback time. J. Neurosci. 29, 2569–2574 (2009).

    Article  CAS  Google Scholar 

  33. Feierstein, C.E., Quirk, M.C., Uchida, N., Sosulski, D.L. & Mainen, Z.F. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51, 495–507 (2006).

    Article  CAS  Google Scholar 

  34. Furuyashiki, T., Holland, P.C. & Gallagher, M. Rat orbitofrontal cortex separately encodes response and outcome information during performance of goal-directed behavior. J. Neurosci. 28, 5127–5138 (2008).

    Article  CAS  Google Scholar 

  35. Abe, H. & Lee, D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70, 731–741 (2011).

    Article  CAS  Google Scholar 

  36. Young, J.J. & Shapiro, M.L. Dynamic coding of goal-directed paths by orbital prefrontal cortex. J. Neurosci. 31, 5989–6000 (2011).

    Article  CAS  Google Scholar 

  37. O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

    Article  CAS  Google Scholar 

  38. Voorn, P., Vanderschuren, L.J.M.J., Groenewegen, H.J., Robbins, T.W. & Pennartz, C.M.A. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 27, 468–474 (2004).

    Article  CAS  Google Scholar 

  39. Lodge, D.J. The medial prefrontal and orbitofrontal cortices differentially regulate dopamine system function. Neuropsychopharmacology 36, 1227–1236 (2011).

    Article  CAS  Google Scholar 

  40. Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).

    Article  CAS  Google Scholar 

  41. Burke, K.A., Franz, T.M., Miller, D.N. & Schoenbaum, G. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 454, 340–344 (2008).

    Article  CAS  Google Scholar 

  42. Ostlund, S.B. & Balleine, B.W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning. J. Neurosci. 27, 4819–4825 (2007).

    Article  CAS  Google Scholar 

  43. Daw, N.D., Gershman, S.J., Seymour, B., Dayan, P. & Dolan, R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron (in the press).

  44. Simon, D.A. & Daw, N.D. Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. (in the press).

  45. Bromberg-Martin, E.S., Matsumoto, M., Hong, S. & Hikosaka, O. A pallidus-habenula-dopamine pathway signals inferred stimulus values. J. Neurophysiol. 104, 1068–1076 (2010).

    Article  Google Scholar 

  46. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057–1063 (2006).

    Article  CAS  Google Scholar 

  47. Hampton, A.N., Bossaerts, P. & O′Doherty, J.P. the role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 26, 8360–8367 (2006).

    Article  CAS  Google Scholar 

  48. Gläscher, J., Daw, N., Dayan, P. & O′Doherty, J.P. Prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585–595 (2010).

    Article  Google Scholar 

  49. McDannald, M.A., Lucantonio, F., Burke, K.A., Niv, Y. & Schoenbaum, G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700–2705 (2011).

    Article  CAS  Google Scholar 

  50. Niv, Y., Daw, N.D. & Dayan, P. Choice values. Nat. Neurosci. 9, 987–988 (2006).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This work was supported by grants from the US National Institute on Drug Abuse to G.S. and M.R. and from the US National Institute on Mental Health to Y.K.T., a Sloan Research Fellowship to Y.N. and a Binational United States-Israel Science Foundation grant to Y.N. and R.C.W.

Author information

Author notes

  1. Yael Niv and Geoffrey Schoenbaum: These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland, USA

    Yuji K Takahashi, Kathy Toreson, Patricio O'Donnell & Geoffrey Schoenbaum

  2. Department of Psychology, University of Maryland College Park, College Park, Maryland, USA

    Matthew R Roesch

  3. Program in Neuroscience and Cognitive Science, University of Maryland College Park, College Park, Maryland, USA

    Matthew R Roesch

  4. Department of Psychology, Princeton University, Princeton, New Jersey, USA

    Robert C Wilson & Yael Niv

  5. Neuroscience Institute, Princeton University, Princeton, New Jersey, USA

    Robert C Wilson & Yael Niv

  6. Department of Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland, USA

    Patricio O'Donnell & Geoffrey Schoenbaum

  7. National Institute on Drug Abuse Intramural Research Program, Baltimore, Maryland, USA

    Geoffrey Schoenbaum

Authors
  1. Yuji K Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew R Roesch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert C Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kathy Toreson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Patricio O'Donnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yael Niv
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Geoffrey Schoenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

G.S., M.R.R. and Y.K.T. conceived the initial unit recording study in awake rats; Y.K.T. and M.R.R. carried it out, and Y.K.T. and G.S. analyzed the data. Subsequently, G.S. approached P.O. and Y.N. regarding in vivo recording and computational modeling, respectively. K.T. conducted the in vivo experiments, and K.T. and P.O. analyzed the data. R.C.W. and Y.N. conceived the alternative computational models, R.C.W. carried out the modeling, and Y.N. and R.C.W. interpreted the experimental data in light of simulation results. Y.N., G.S. and Y.K.T. collaborated in writing the manuscript with assistance from the other team members.

Corresponding authors

Correspondence to Yuji K Takahashi or Geoffrey Schoenbaum.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–4 and Supplementary Results (PDF 769 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Takahashi, Y., Roesch, M., Wilson, R. et al. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat Neurosci 14, 1590–1597 (2011). https://doi.org/10.1038/nn.2957

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2957

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing