Abstract
The ability to predict favorable outcomes using environmental cues is an essential part of learned behavior. Dopamine neurons in the midbrain encode such stimulus-reward relationships in a manner consistent with contemporary learning models, but it is unclear how encoding this translates into actual dopamine release in target regions. Here, we sampled dopamine levels in the rat nucleus accumbens on a rapid (100 ms) timescale using electrochemical technology during a classical conditioning procedure. Early in conditioning, transient dopamine-release events signaled a primary reward, but not predictive cues. After repeated cue-reward pairings, dopamine signals shifted in time to predictive cue onset and were no longer observed at reward delivery. In the absence of stimulus-reward conditioning, there was no shift in the dopamine signal. Consistent with proposed roles in reward prediction and incentive salience, these results indicate that rapid dopamine release provides a reward signal that is dynamically modified by associative learning.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).
Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).
Sutton, R.S. & Barto, A.G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).
Montague, P.R., Hyman, S.E. & Cohen, J.D. Computational roles for dopamine in behavioural control. Nature 431, 760–767 (2004).
Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027 (1994).
Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).
Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057–1063 (2006).
Mogenson, G.J., Jones, D.L. & Yim, C.Y. From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69–97 (1980).
Day, J.J., Wheeler, R.A., Roitman, M.F. & Carelli, R.M. Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. Eur. J. Neurosci. 23, 1341–1351 (2006).
Roitman, M.F., Wheeler, R.A. & Carelli, R.M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597 (2005).
Yun, I.A., Wakabayashi, K.T., Fields, H.L. & Nicola, S.M. The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J. Neurosci. 24, 2923–2933 (2004).
Di Ciano, P., Cardinal, R.N., Cowell, R.A., Little, S.J. & Everitt, B.J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J. Neurosci. 21, 9471–9477 (2001).
Di Chiara, G. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav. Brain Res. 137, 75–114 (2002).
Dalley, J.W. et al. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc. Natl. Acad. Sci. USA 102, 6189–6194 (2005).
Zhang, H. & Sulzer, D. Frequency-dependent modulation of dopamine release by nicotine. Nat. Neurosci. 7, 581–582 (2004).
Cragg, S.J. Variable dopamine release probability and short-term plasticity between functional domains of the primate striatum. J. Neurosci. 23, 4378–4385 (2003).
Cragg, S.J. Meaningful silences: how dopamine listens to the ACh pause. Trends Neurosci. 29, 125–131 (2006).
Montague, P.R. et al. Dynamic gain control of dopamine delivery in freely moving animals. J. Neurosci. 24, 1754–1759 (2004).
Garris, P.A. et al. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature 398, 67–69 (1999).
Bassareo, V. & Di Chiara, G. Differential influence of associative and nonassociative learning mechanisms on the responsiveness of prefrontal and accumbal dopamine transmission to food stimuli in rats fed ad libitum. J. Neurosci. 17, 851–861 (1997).
Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).
Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042 (2004).
Margolis, E.B., Lock, H., Hjelmstad, G.O. & Fields, H.L. The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J Physiol 15, 907–924 (2006).
Phillips, P.E., Stuber, G.D., Heien, M.L., Wightman, R.M. & Carelli, R.M. Subsecond dopamine release promotes cocaine seeking. Nature 422, 614–618 (2003).
Heien, M.L., Johnson, M.A. & Wightman, R.M. Resolving neurotransmitters detected by fast-scan cyclic voltammetry. Anal. Chem. 76, 5697–5704 (2004).
Heien, M.L. et al. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc. Natl. Acad. Sci. USA 102, 10023–10028 (2005).
Everitt, B.J. & Robbins, T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489 (2005).
Robbins, T.W. & Everitt, B.J. Limbic-striatal memory systems and drug addiction. Neurobiol. Learn. Mem. 78, 625–636 (2002).
Uslaner, J.M., Acerbo, M.J., Jones, S.A. & Robinson, T.E. The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behav. Brain Res. 169, 320–324 (2006).
Cardinal, R.N. et al. Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav. Neurosci. 116, 553–567 (2002).
Parkinson, J.A., Olmstead, M.C., Burns, L.H., Robbins, T.W. & Everitt, B.J. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J. Neurosci. 19, 2401–2411 (1999).
Pan, W.X. & Hyland, B.I. Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J. Neurosci. 25, 4725–4732 (2005).
Dommett, E. et al. How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476–1479 (2005).
Kakade, S. & Dayan, P. Dopamine: generalization and bonuses. Neural Netw. 15, 549–559 (2002).
Parkinson, J.A. et al. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav. Brain Res. 137, 149–163 (2002).
Ikemoto, S. & Panksepp, J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Brain Res. Rev. 31, 6–41 (1999).
Roitman, M.F., Stuber, G.D., Phillips, P.E., Wightman, R.M. & Carelli, R.M. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24, 1265–1271 (2004).
Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).
Berridge, K.C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl.) 191, 391–431 (2006).
Berridge, K.C. & Robinson, T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998).
Setlow, B., Schoenbaum, G. & Gallagher, M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636 (2003).
Cepeda, C. & Levine, M.S. Dopamine and N-methyl-D-aspartate receptor interactions in the neostriatum. Dev. Neurosci. 20, 1–18 (1998).
Kerr, J.N. & Wickens, J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 85, 117–124 (2001).
Eyny, Y.S. & Horvitz, J.C. Opposing roles of D1 and D2 receptors in appetitive conditioning. J. Neurosci. 23, 1584–1587 (2003).
Watson, C.J., Venton, B.J. & Kennedy, R.T. In vivo measurements of neurotransmitters by microdialysis sampling. Anal. Chem. 78, 1391–1399 (2006).
Richfield, E.K., Penney, J.B. & Young, A.B. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30, 767–777 (1989).
Greengard, P. The neurobiology of slow synaptic transmission. Science 294, 1024–1030 (2001).
Phillips, P.E., Robinson, D.L., Stuber, G.D., Carelli, R.M. & Wightman, R.M. Real-time measurements of phasic changes in extracellular dopamine concentration in freely moving rats by fast-scan cyclic voltammetry. Methods Mol. Med. 79, 443–464 (2003).
Paxinos, G. & Watson, C. The Rat Brain in Stereotaxic Coordinates (Elsevier, New York, 2005).
Acknowledgements
The authors would like to thank R.A. Wheeler, B.J. Aragona, P.E.M. Phillips and J.L. Jones for helpful comments on this manuscript, and the University of North Carolina Department of Chemistry Electronics Facility for technical expertise. This research was supported by the US National Institute on Drug Abuse (DA 017318 to R.M.C., DA to 10900 R.M.W., DA 021979 to J.J.D. and DA 018298 to M.F.R.).
Author information
Authors and Affiliations
Contributions
J.J.D. and M.F.R. conducted the behavioral and electrochemical experiments. J.J.D. wrote the manuscript and conducted data and graphical analyses. M.F.R. contributed to the writing of the manuscript. R.M.C. and R.M.W. supervised the project and the writing of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Fig. 1
Individual differences in development of cue-related dopamine signals early in learning. (PDF 1400 kb)
Supplementary Fig. 2
Relationship between approach responses and dopamine signal. (PDF 366 kb)
Supplementary Fig. 3
-evoked dopamine release after extended conditioning. (PDF 345 kb)
Rights and permissions
About this article
Cite this article
Day, J., Roitman, M., Wightman, R. et al. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10, 1020–1028 (2007). https://doi.org/10.1038/nn1923
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn1923
This article is cited by
-
Glutamatergic dysfunction leads to a hyper-dopaminergic phenotype through deficits in short-term habituation: a mechanism for aberrant salience
Molecular Psychiatry (2023)
-
Disentangling the role of NAc D1 and D2 cells in hedonic eating
Molecular Psychiatry (2023)
-
Distinct sub-second dopamine signaling in dorsolateral striatum measured by a genetically-encoded fluorescent sensor
Nature Communications (2023)
-
Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues
Nature Communications (2022)
-
Dopamine signaling in the nucleus accumbens core mediates latent inhibition
Nature Neuroscience (2022)