Research reportRats that sign-track are resistant to Pavlovian but not instrumental extinction
Introduction
In appetitive Pavlovian conditioning an initially neutral cue is paired with the delivery of a reward (the unconditioned stimulus; US), and as they become associated, the cue (conditioned stimulus; CS) may come to elicit a variety of conditioned responses (CRs) that anticipate reward delivery. CRs can take many forms, depending not only on the nature of the US, but also the properties of the CS [35], including whether the CS is attributed with incentive salience [2], [27], [43], [52], [54], [61]. For example, if presentation of a discrete, localizable CS is consistently followed by food reward in a response-independent manner some animals learn to approach the CS itself (termed “sign-trackers”; STs) [34], whilst others approach the site of food delivery (termed “goal trackers”; GTs) [6], [74]. Although both STs and GTs learn the predictive value of the CS – it comes to evoke an anticipatory CR in both – it has been suggested that the form of the CR differs in part because STs attribute greater incentive salience to the CS than GTs [1], [27], [44], [54], [61], [66]. Not only are STs more attracted to the CS, but it is a more effective conditioned reinforcer and more effective in renewing reward-seeking behavior in STs than GTs [54], [62], [71]. Furthermore, performance on the Pavlovian conditioned approach (PCA) task using food reward predicts the extent to which discrete drug cues acquire control over motivated behavior [45], [55], [61]. Thus, it appears that the cue is an equally effective CS in STs and GTs, but it acts as a more powerful, motivating incentive stimulus in STs.
Both food and drug cues engage so-called brain ‘motive circuits’ to a greater extent in STs than GTs [28], [29], [73], which may contribute to STs being more susceptible to develop impulse control disorders, such as addiction [4], [29], [61], [69]. Addiction-like behaviors tend to persist despite non-reinforcement [22], therefore, one might expect sign-tracking behavior to be more resistant to extinction than goal-tracking. However, in previous studies involving the extinction of instrumental responding for food [71] or cocaine [57], [59], [72], there was no difference between STs and GTs in the rate of instrumental extinction. However, in the studies above extinction was conducted in the absence of a reward cue (CS), or an explicit reward cue was not used in training. In another study presentation of an increasingly aversive outcome progressively decreased instrumental responding for cocaine at the same rate in STs and GTs, despite continued presence of a cue associated with cocaine delivery [62]. On the other hand, in one study in which extinction of cocaine self-administration behavior was conducted in the presence of the reward cue, STs were somewhat more resistant to extinction than GTs, but the effect was very small and was only evident on the first day (out of 28 days) of extinction training [58]. Thus, there is no compelling evidence that STs and GTs differ in extinction learning in an instrumental task, but we are not aware of any study comparing Pavlovian extinction in STs and GTs.
It is important to examine Pavlovian extinction because, although there are many similarities in the processes involved in instrumental and Pavlovian extinction, there are also differences (e.g., [65]). Beckmann and Chow did examine Pavlovian extinction using a 2-CS procedure, in which rats developed a ST CR to a lever-CS and a GT CR to a tone-CS, when both CSs were presented within the same session. They reported that when the lever and tone were no longer reinforced the GT CR extinguished faster than the ST CR [3]. However, in this study all rats showed a ST CR during lever-CS presentation; i.e., no rats showed a GT CR in response to the lever-CS. Therefore, it is still not known whether STs and GTs differ in Pavlovian extinction, when the same CS evokes two different CRs in different individuals.
Thus, we conducted a series of studies on extinction in STs and GTs. In the first study, after initial training, the US was simply withheld over 4-8 days of extinction training. Next, we examined within-session extinction and recovery of ST and GT CRs during alternating blocks of rewarded (R) and non-rewarded (NR) trials. The R and NR periods were signaled by either a diffuse and continuous change in illumination (experiment 2) or by spatially and temporally discrete light cues (experiment 3). These signals were intended to facilitate discrimination between R and NR phases by acting as occasion setters, indicating that the CS would be rewarded when accompanied by one signal but would not be rewarded when accompanied by the other [8], [9], [16], [36]. Finally, in a fourth experiment changes in illumination were used as discriminative stimuli [9], [11] signaling whether an instrumental response would be reinforced or not. STs were highly resistant to extinction, relative to GTs, but only in the Pavlovian setting, when the CS was present.
Section snippets
Subjects
A total of 133 male Sprague–Dawley rats purchased from Charles River or Harlan were used [26]. All rats weighed 250–275 g on arrival, and were singly housed in standard polycarbonate cages with food and water available ad libitum throughout experiments (i.e., the rats were not food deprived at any time). Rats were housed on a reverse 12-h light/12-h dark cycle, with all testing taking place during the dark phase of the cycle. Testing began after one week of acclimation and handling. All
PCA training: ST and GT groups
As described previously, individual rats differed in the form of conditioned approach behavior they developed during the PCA training phase, with some rats primarily approaching the lever and others the food cup during the 8-s CS periods [27], [44], [54]. Across the four experiments, 72 rats were classified as STs, and 61 were classified as GTs. Additional rats were classified as intermediates (INs), but these rats were not used further because we wanted to directly compare groups that varied
Discussion
In appetitive Pavlovian conditioning, when a CS is associated with delivery of a reward, animals may acquire many different anticipatory CRs depending on the form of both the US and the CS [20], [35]. However, as seen here, even when subjected to exactly the same training procedures, using the same CS and US, different individuals may acquire quite different CRs (also see [6], [27], [62], [74]. Thus, presentation of a lever-CS, paired with delivery of a food reward, evoked a ST CR (approach and
Acknowledgements
This study was supported by P01-DA031656 (TER), T32-DA007267 (AMA), T32-DA007268 (AMA, BFS), F32-DA038383 (BFS), K08-DA037912-01 (JDM), the University of Michigan Department of Psychiatry (U032826; JDM), and a Department of Defense (DoD) National Defense Science and Engineering Graduate (NDSEG) fellowship (CJF). We thank Dr. Stephan Anagnostaras for helpful comments on an earlier version of this paper.
References (74)
- et al.
Reward uncertainty enhances incentive salience attribution as sign-tracking
Behav. Brain Res.
(2013) - et al.
Novelty seeking, incentive salience and acquisition of cocaine self-administration in the rat
Behav. Brain Res.
(2011) ‘Liking’ and ‘wanting’ food rewards: brain substrates and roles in eating disorders
Physiol. Behav.
(2009)Context, ambiguity, and unlearning: sources of relapse after behavioral extinction
Biol. Psychiatry
(2002)- et al.
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Neurosci. Biobehav. Rev.
(2002) - et al.
Roles of nucleus accumbens and basolateral amygdala in autoshaped lever pressing
Neurobiol. Learn. Mem.
(2012) - et al.
Pavlovian valuation systems in learning and decision making
Curr. Opin. Neurobiol.
(2012) - et al.
Disruption of conditioned reward association by typical and atypical antipsychotics
Pharmacol. Biochem. Behav.
(2010) Issues in the extinction of specific stimulus-outcome associations in Pavlovian conditioning
Behav. Processes
(2012)- et al.
Which cue to ‘want’? Opioid stimulation of central amygdala makes goal-trackers show stronger goal-tracking, just as sign-trackers show stronger sign-tracking
Behav. Brain Res.
(2012)
Stepping outside the box in considering the C/T ratio?
Behav. Processes
Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction
Neuropharmacology
A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico-striatal-thalamic brain regions
Neuroscience
Antecedents and consequences of drug abuse in rats selectively bred for high and low response to novelty
Neuropharmacology
Occasion setting in Pavlovian conditioning
Hippocampus and context in classical conditioning
Curr. Opin. Neurobiol.
Inadequate early social experience increases the incentive salience of reward-related cues in adulthood
Behav. Brain Res.
Rats prone to attribute incentive salience to reward cues are also prone to impulsive action
Behav. Brain Res.
Individual variation in the propensity to attribute incentive salience to an appetitive cue predicts the propensity to attribute motivational salience to an aversive cue
Behav. Brain Res.
d-Cycloserine administered directly to infralimbic medial prefrontal cortex enhances extinction memory in sucrose-seeking animals
Neuroscience
Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues
Behav. Brain Res.
Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences
Biol. Psychiatry
On the motivational properties of reward cues: individual differences
Neuropharmacology
A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction
Biol. Psychiatry
Individual variation in resisting temptation: implications for addiction
Neurosci. Biobehav. Rev.
Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning
Neurobiol. Learn. Mem.
Individual differences in pavlovian autoshaping of lever pressing in rats predict stress-induced corticosterone release and mesolimbic levels of monoamines
Pharmacol. Biochem. Behav.
Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse
Brain Res. Rev.
Sign-tracking predicts increased choice of cocaine over food in rats
Behav. Brain Res.
Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues
Behav. Brain Res.
Incentive salience attribution under reward uncertainty: a Pavlovian model
Behav. Processes
Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence
Learn. Mem.
Performance on learning to associate a stimulus with positive reinforcement
Context and behavioral processes in extinction
Learn. Mem.
Contextual control of discriminated operant behavior
J. Exp. Psychol. Anim. Learn. Cogn.
Topography of spatially directed conditioned responding: effects of context and trial duration
J. Exp. Psychol. Anim. Behav. Process
Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training
J. Neurosci.
Cited by (70)
Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking
2024, Behavioural Brain ResearchEffects of predictive and incentive value manipulation on sign- and goal-tracking behavior
2023, Neurobiology of Learning and MemoryCO<inf>2</inf> reactivity is associated with individual differences in appetitive extinction memory
2023, Physiology and BehaviorA mechanical task for measuring sign- and goal-tracking in humans: A proof-of-concept study
2023, Behavioural Brain ResearchEstradiol and progesterone in female reward-learning, addiction, and therapeutic interventions
2023, Frontiers in Neuroendocrinology