Neurobiological basis of individual variation in stimulus-reward learning
Introduction
Cues (conditional stimuli, CSs) that predict the impending delivery of biologically significant events (unconditional stimuli, USs), such as a food reward, acquire the ability to control behavior, or produce a conditioned response (CR), via Pavlovian learning mechanisms [1]. The same is true for stimuli associated with aversive events, but here we will focus only on cues associated with rewards. The ability of a CS to evoke simple reflexive CRs, such as salivation in the case of Pavlov’s dogs, is well known. It is less well appreciated, however, that CSs can also acquire the ability to evoke complex emotional and motivational states [2, 3, 4, 5]. This latter transformation is thought to occur if a CS is attributed with incentive salience and thus acquires the properties of an incentive stimulus [2, 3, 4, 6]. Incentive stimuli: (1) bias attention towards them and can elicit approach into close proximity with them; (2) become desirable themselves, in the sense that an animal will work for access to the stimulus alone (i.e., they act as conditioned reinforcers); and (3) can instigate or invigorate reward-seeking behavior (as in Pavlovian-to-instrumental transfer effects, PIT). Incentive stimuli can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, such cues may also serve as powerful motivators that lead to maladaptive patterns of behavior, as in over-eating and addiction. Importantly, there is considerable individual variation in the extent to which CSs act as incentive stimuli and gain motivational control over behavior [7].
In the laboratory, if a discrete and localizable Pavlovian CS, such as presentation of a lever, is reliably paired with presentation of a food reward, some rats come to approach the CS (Figure 1), even though no response is required for delivery of the reward [8]. This is called ‘sign-tracking’ [9, 10]. In contrast, upon presentation of the lever-CS, other rats go to the location of impending reward delivery (Figure 1) [8]. This CR is called ‘goal-tracking’ [11]. It is important to note that the lever-CS is an equally effective predictive stimulus (CS) in both sign-trackers (STs) and goal-trackers (GTs) – they learn their respective CRs at the same rate – but only in STs does the lever-CS acquire the properties of an incentive stimulus [12]. That is, for STs, the CS is more attractive and elicits approach towards it, is a more effective conditioned reinforcer, and is more effective at instigating reward-seeking behavior relative to GTs [12, 13, 14]. Importantly, this variation in the ability of a CS to acquire incentive salience is captured best by a localizable CS (i.e., lever or light) [15], and, in rats, is not apparent when the CS is a tone [16]. Furthermore, tone stimuli paired with a food reward are attributed with less incentive value than a lever-CS [16, 17].
Sign-trackers are also more resistant to extinction of their CR than GTs [18] and will continue to approach the CS even if contact with it results in omission of the reward [19], indicating that approach behavior is not supported by response reinforcement (i.e., by food delivery reinforcing an action already made) [20]. It has also been shown that sign-tracking behavior becomes more pronounced when the relationship between the CS and the reward is uncertain, such that the probability of the reward following CS presentation changes [21]. These findings provide further evidence for dissociation between the predictive vs. incentive value of the CS (see also Ref. [22]). Thus, a lever-CS acquires all of the properties of an incentive stimulus in some individuals (STs), but not others (GTs). Importantly, this individual variation in the propensity to attribute incentive salience to a discrete or localizable CS has been described not only for food predictive cues, but also for cues that predict drug rewards [7, 15]. The current article will focus on what we have learned about the neurobiological mechanisms of stimulus-reward learning by exploiting this individual variation.
Section snippets
Dopamine
There has been considerable research on the role of dopamine (DA) in stimulus-reward learning, and one popular hypothesis is that phasic DA signals serve as a prediction error signal necessary for learning associations (for recent review see Ref. [22]). Given that STs and GTs learn CS-US associations equally well, but differ in the degree to which they attribute incentive salience to the CS [12], we have used this animal model to parse the role of dopamine in stimulus-reward learning [23]. The
Conclusions
We have provided a review of converging data that implicate several brain regions and possible circuits in mediating the attribution of incentive salience to reward cues (Figure 2). Using the sign-tracker/goal-tracker animal model, we, and others, have demonstrated that the role of dopamine is to encode the incentive – and not the predictive – properties of reward cues. Furthermore, the cortico–striato–pallido–thalamic loops of the ‘motive circuit’ are engaged only when a reward cue is
Conflict of interest statement
Nothing declared.
Funding sources
Support for the authors and the studies reviewed in this manuscript is provided by grants from the National Institute on Drug Abuse (NIDA): P01 DA031656 (SBF, TER), P50 DA037844 (SBF, TER) and R01 DA038599 (SBF).
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
We would like to thank the former and present members of the Robinson and Flagel laboratories who contributed to some of the studies reviewed here and who prompted insightful discussions surrounding the topic.
References (59)
- et al.
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Neurosci Biobehav Rev
(2002) - et al.
On the motivational properties of reward cues: individual differences
Neuropharmacology
(2014) - et al.
Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction
Neuropharmacology
(2009) - et al.
Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences
Biol Psychiatry
(2009) - et al.
A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction
Biol Psychiatry
(2010) - et al.
Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues
Behav Brain Res
(2010) - et al.
Rats that sign-track are resistant to Pavlovian but not instrumental extinction
Behav Brain Res
(2016) - et al.
Reward uncertainty enhances incentive salience attribution as sign-tracking
Behav Brain Res
(2013) - et al.
Examining the role of dopamine D2 and D3 receptors in Pavlovian conditioned approach behaviors
Behav Brain Res
(2016) - et al.
Dopamine D2 modulation of sign and goal tracking in rats
Neuropsychopharmacology
(2015)
Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training
J Neurosci
The neural basis of addiction: a pathology of motivation and choice
Am J Psychiatry
A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico–striatal–thalamic brain regions
Neuroscience
A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses
Front Behav Neurosci
Cholinergic control over attention in rats prone to attribute incentive salience to reward cues
J Neurosci
The habenula governs the attribution of incentive salience to reward predictive cues
Front Hum Neurosci
Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats
Eur J Neurosci
Neural activity in the ventral pallidum encodes variation in the incentive value of a reward cue
J Neurosci
Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior
Front Behav Neurosci
Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex
Reward learning: reinforcement, incentives and expectations
How adaptive behavior is produced: a perceptual–motivation alternative to response reinforcement
Behav Brain Sci
Pavlovian conditioning: it’s not what you think it is
Am Psychol
The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction
Eur J Neurosci
Auto-shaping of the pigeon’s key-peck
J Exp Anal Behav
Sign-tracking: the stimulus-reinforcer relation and directed action
Monograph of the Psychonomic Society
A study of misbehavior: token reinforcement in the rat
J Exp Anal Behav
A classically conditioned cocaine cue acquires greater control over motivated behavior in rats prone to attribute incentive salience to a food cue
Psychopharmacology (Berl)
The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties
PLoS One
Cited by (77)
Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking
2024, Behavioural Brain ResearchSeparating desire from prediction of outcome value
2023, Trends in Cognitive SciencesMotivation and reward mechanisms in health behavior change processes
2021, Brain ResearchA translational rodent model of individual differences in sensitivity to the aversive properties of ethanol
2024, Alcohol, Clinical and Experimental Research