Phasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs

doi:10.1016/j.biopsych.2010.03.026

Biological Psychiatry

Volume 68, Issue 3, 1 August 2010, Pages 306-309

https://doi.org/10.1016/j.biopsych.2010.03.026 Get rights and content

Background

Optimal decision-making requires that organisms correctly evaluate both the costs and benefits of potential choices. Dopamine transmission within the nucleus accumbens (NAc) has been heavily implicated in reward-learning and decision-making, but it is unclear how dopamine release might contribute to decisions that involve costs.

Methods

Cost-based decision-making was examined in rats trained to associate visual cues with either immediate or delayed rewards (delay manipulation) or low-effort or high-effort rewards (effort manipulation). After training, dopamine concentration within the NAc was monitored on a rapid time scale with fast-scan cyclic voltammetry.

Results

Animals exhibited a preference for immediate or low-effort rewards over delayed or high-effort rewards of equal magnitude. Reward-predictive cues but not response execution or reward delivery evoked increases in NAc dopamine concentration. When only one response option was available, cue-evoked dopamine release reflected the value of the future reward, with larger increases in dopamine signaling higher-value rewards. In contrast, when both options were presented simultaneously, dopamine signaled the better of two options, regardless of the future choice.

Conclusions

Phasic dopamine signals in the NAc reflect two different types of reward cost and encode potential rather than chosen value under choice situations.

Section snippets

Methods and Materials

The NAc dopamine concentration was monitored in rats (n = 10) trained in either effort-based or delay-based decision tasks (Figure S1 in Supplement 1 and Figure 1A;Supplementary Methods in Supplement 1). In these tasks, rewards of equal magnitude (45-mg sucrose pellets) were made available at either low or high value in pseudorandomly ordered trials, with 90 total trials/behavioral session. On forced-choice trials (60/session), distinct 5-sec cue lights signaled the available response option

Results

During behavioral sessions, animals readily overcame high-effort demands or long delays to obtain rewards (Figures 1B and 1E) and discriminated between reward-predictive cues to reduce errors on forced-choice trials (error rates significantly below chance levels; p < .0001 for all comparisons; Figures 1C and 1F). On free-choice trials, animals exhibited a marked preference for low-cost and immediate reward options over high-cost and delayed reward options (paired t test on choice allocation; p

Discussion

Dopamine neurons encode a reward prediction error signal in which cues that predict rewards evoke phasic increases in firing rate, whereas fully expected rewards do not alter dopamine activity (13). This signal is also sensitive to a number of features of the upcoming reward, because cues that predict larger, immediate, or more probable rewards evoke larger spikes in dopamine neuron activity than cues that predict smaller, delayed, or less probable rewards (9, 10, 11, 14). In this way, dopamine

References (18)

A. Scheres et al.
Temporal Reward discounting in attention-deficit/hyperactivity disorder: The contribution of symptom domains, reward magnitude, and session length
Biol Psychiatry
(2010)
J.D. Salamone et al.
Anhedonia or anergia?Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure
Behav Brain Res
(1994)
L. Green et al.
A discounting framework for choice with delayed and probabilistic rewards
Psychol Bull
(2004)
E.A. Heerey et al.
Delay discounting in schizophrenia
Cogn Neuropsychiatr
(2007)
P.L. Croxson et al.
Effort-based cost-benefit valuation and the human brain
J Neurosci
(2009)
J.D. Salamone et al.
Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: Implications for studies of natural motivation, psychiatry, and drug abuse
J Pharmacol Exp Ther
(2003)
R.N. Cardinal et al.
Impulsive choice induced in rats by lesions of the nucleus accumbens core
Science
(2001)
J.J. Day et al.
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Nat Neurosci
(2007)
M.F. Roitman et al.
Dopamine operates as a subsecond modulator of food seeking
J Neurosci
(2004)

There are more references available in the full text version of this article.

Cited by (124)

Accumbal adenosine A<inf>2A</inf> receptor inactivation biases for large and costly rewards in the effort- but not delay-based decision making
2023, Neuropharmacology
The cost-benefit decision-making (CBDM) is critical to normal human activity and a diminished willingness to expend effort to obtain rewards is a prevalent/noted characteristic of neuropsychiatric disorders such as schizophrenia, Parkinson's disease. Numerous studies have identified nucleus accumbens (NAc) as an important locus for CBDM control but their neuromodulatory and behavioral mechanisms remain largely under-explored. Adenosine A_2A receptors (A_2ARs), which are highly concentrated in the striatopallidal neurons, can integrate glutamate and dopamine signals for controlling effort-related choice behaviors. While the involvement of A_2ARs in effort-based decision making is well documented, the role of other decision variables (reward discrimination) in effort-based decision making and the role of A_2AR in delay-based decision making are less clear. In this study, we have developed a well-controlled CBDM behavioral paradigm to manipulate effort/cost and reward independently or in combination, allowing a dissection of four behavioral elements: effort-based CBDM (E-CBDM), delay-based CBDM (D-CBDM), reward discrimination (RD), effort discrimination (ED), and determined the effect of genetic knockdown (KD) of NAc A_2AR on the four behavioral elements. We found that A_2AR KD in NAc increased the choice for larger, more costly reward in the E-CBDM, but not D-CBDM. Furthermore, this high-effort/high-reward bias was attributable to the increased willingness to engage in effort but not the effect of discrimination of reward magnitude. Our findings substantiate an important role of the NAc A_2AR in control of E-CBDM and support that pharmacologically targeting NAc A_2ARs would be a useful strategy for treating the aberrant effort-based decision making in neuropsychiatric disorders.
The effects of early life stress on impulsivity
2022, Neuroscience and Biobehavioral Reviews
Elevated impulsivity is a symptom shared by various psychiatric disorders such as substance use disorder, bipolar disorder, and attention-deficit/hyperactivity disorder. However, impulsivity is not a unitary construct and impulsive behaviors fall into two subcategories: impulsive action and impulsive choice. Impulsive choice refers to the tendency to prefer immediate, small rewards over delayed, large rewards, whereas impulsive action involves difficulty inhibiting rash, premature, or mistimed behaviors. These behaviors are mediated by the mesocorticolimbic dopamine (DA) system, which consists of projections from the ventral tegmental area to the nucleus accumbens and prefrontal cortex. Early life stress (ELS) alters both impulsive choice and impulsive action in rodents. ELS also changes DA receptor expression, transmission, and activity within the mesocorticolimbic system. This review integrates the dopamine, impulsivity, and ELS literature to provide evidence that ELS alters impulsivity via inducing changes in the mesocorticolimbic DA system. Understanding how ELS affects brain circuits associated with impulsivity can help advance treatments aimed towards reducing impulsivity symptoms in a variety of psychiatric disorders.
Evolution of in vivo dopamine monitoring techniques
2021, Pharmacology Biochemistry and Behavior
The brain dopamine system is central to numerous behavioral processes, including movement, learning, and motivation. Accordingly, disruptions of this neural system underlie numerous neurological and psychiatric disorders. Current understanding of how dopamine neurotransmission contributes to behavior and its dysfunction has been driven by technological advancements that permit spatiotemporally-defined measurements of dopaminergic signaling in behaving animals. In this review, we will discuss the evolution of in vivo neural monitoring technologies for measuring dopamine neuron function. We focus on the dopamine system for two reasons: (1) the central role of dopamine neurotransmission in normal behavior and disease, and (2) dopamine neuron measurements have long been at the forefront of in vivo neural monitoring technologies. We will provide a brief overview of standard techniques for monitoring dopamine function, including electrophysiology, microdialysis, and voltammetry. Then, we will discuss recent advancements in optical technologies using genetically-encoded fluorescent proteins (GEFPs), including a critical evaluation of their advantages and limitations.
An Analysis of Decision under Risk in Rats
2019, Current Biology
Citation Excerpt :
Conditioned stimuli predicting rewards with different probabilities or magnitudes have been shown to elicit phasic dopamine responses reflecting the value of the expected reward [71–73]. In delay discounting tasks, the phasic dopamine response reflects discounted value of delayed rewards in monkeys and rats [74, 75]. Finally, recent work has shown that dopamine reward prediction errors reflect the shape of monkeys’ measured utility functions [76].
In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled “Prospect Theory: An Analysis of Decision under Risk,” which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists’ normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards. However, prospect theory assumes stable preferences in the absence of learning, an assumption at odds with alternative frameworks such as animal learning theory and reinforcement learning [3, 4, 5, 6, 7]. Rats also exhibited trial history effects, consistent with ongoing learning. A reinforcement learning model in which state-action values were updated by the subjective value of outcomes according to prospect theory reproduced rats’ nonlinear utility and probability weighting functions and also captured trial-by-trial learning dynamics.
The power of price compels you: Behavioral economic insights into dopamine-based valuation of rewarding and aversively motivated behavior
2019, Brain Research
Citation Excerpt :
An interdisciplinary combination of economic theory, computer science and mathematics led to value-based computational models that better align with the DA and behavioral observations we observe during action-outcome learning (Glimcher, 2011). A growing body of electrophysiological and electrochemical studies suggest that the concentration of dopamine evoked by reward predictive cues corresponds to the magnitude of reward predicted by the cue (Bayer and Glimcher, 2005; Day et al., 2010; Day et al., 2011; Enomoto et al., 2011; Lak et al., 2014; Schelp et al., 2017; Stauffer et al., 2014; Tobler et al., 2005). Imaging studies reveal similar observations in human subjects and further refine the output regions involved.
The mesocorticolimbic dopamine pathway is generally considered to be a reward pathway. While shortsighted, there is a strong basis for this view of dopamine function. Here, we first describe the role of phasic dopamine release events in reward seeking. We then explain why these release events are being reconsidered as value signals and how we applied behavioral economics to confirm they play a causal role in the valuation of reward. Just because dopamine release can function as a dopamine reward value signal however, does not imply that dopamine is solely a reward molecule. Rather, mesocorticolimbic dopamine appears to mediate many adaptive behaviors, including: reward seeking, avoidance, escape and fear-associated conditioned freezing. While more studies are needed before a consensus is reached on when, where and how dopamine mediates aversively-motivated behavior, its involvement is almost irrefutable. Thus, we next describe the role dopamine plays in these ethologically-relevant defensive behaviors. We conclude by describing our recent behavioral economics results that reveal a causal role for dopamine in the valuation of avoidance.
Accumbal Dopamine Release Tracks the Expectation of Dopamine Neuron-Mediated Reinforcement
2019, Cell Reports
Dopamine (DA) transmission in the nucleus accumbens (NAc) facilitates cue-reward associations and appetitive action. Reward-related accumbal DA release dynamics are traditionally ascribed to ventral tegmental area (VTA) DA neurons. Activation of VTA to NAc DA signaling is thought to reinforce action and transfer reward-related information to predictive cues, allowing cues to guide behavior and elicit dopaminergic activity. Here, we use optogenetics to control DA neuron activity and voltammetry to simultaneously record accumbal DA release in order to quantify how reinforcer-evoked dopaminergic activity shapes conditioned mesolimbic DA transmission. We find that cues predicting access to DA neuron self-stimulation elicit conditioned responding and NAc DA release. However, cue-evoked DA release does not reflect the cost or magnitude of DA neuron activation. Accordingly, conditioned accumbal DA release selectively tracks the expected availability of DA-neuron-mediated reinforcement. This work provides insight into how mesolimbic DA transmission drives and encodes appetitive action.

View all citing articles on Scopus

View full text

Brief ReportPhasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs

Background

Methods

Results

Conclusions

Section snippets

Methods and Materials

Results

Discussion

Biol Psychiatry

Behav Brain Res

A discounting framework for choice with delayed and probabilistic rewards

Psychol Bull

Delay discounting in schizophrenia

Cogn Neuropsychiatr

Effort-based cost-benefit valuation and the human brain

J Neurosci

Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: Implications for studies of natural motivation, psychiatry, and drug abuse

J Pharmacol Exp Ther

Impulsive choice induced in rats by lesions of the nucleus accumbens core

Science

Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

Nat Neurosci

Dopamine operates as a subsecond modulator of food seeking

J Neurosci

Brief Report
Phasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs