Elsevier

Behavioural Brain Research

Volume 296, 1 January 2016, Pages 418-430
Behavioural Brain Research

Research report
Rats that sign-track are resistant to Pavlovian but not instrumental extinction

https://doi.org/10.1016/j.bbr.2015.07.055Get rights and content

Highlights

  • Sign trackers (STs) attribute more incentive salience to cues than goal trackers (GTs).

  • We examined STs and GTs when a Pavlovian cue was no longer consistently reinforced.

  • GTs showed faster and more complete extinction of conditioned responding than STs.

  • GTs showed greater behavioral flexibility than STs during intermittent phases of non-reward.

  • These ST-GT differences in extinction performance were seen in Pavlovian but not instrumental tasks.

Abstract

Individuals vary in the extent to which they attribute incentive salience to a discrete cue (conditioned stimulus; CS) that predicts reward delivery (unconditioned stimulus; US), which results in some individuals approaching and interacting with the CS (sign-trackers; STs) more than others (goal-trackers; GTs). Here we asked how periods of non-reinforcement influence conditioned responding in STs vs. GTs, in both Pavlovian and instrumental tasks. After classifying rats as STs or GTs by pairing a retractable lever (the CS) with the delivery of a food pellet (US), we introduced periods of non-reinforcement, first by simply withholding the US (i.e., extinction training; experiment 1), then by signaling alternating periods of reward (R) and non-reward (NR) within the same session (experiments 2 and 3). We also examined how alternating R and NR periods influenced instrumental responding for food (experiment 4). STs and GTs did not differ in their ability to discriminate between R and NR periods in the instrumental task. However, in Pavlovian settings STs and GTs responded to periods of non-reward very differently. Relative to STs, GTs very rapidly modified their behavior in response to periods of non-reward, showing much faster extinction and better and faster discrimination between R and NR conditions. These results highlight differences between Pavlovian and instrumental extinction learning, and suggest that if a Pavlovian CS is strongly attributed with incentive salience, as in STs, it may continue to bias attention toward it, and to facilitate persistent and relatively inflexible responding, even when it is no longer followed by reward.

Introduction

In appetitive Pavlovian conditioning an initially neutral cue is paired with the delivery of a reward (the unconditioned stimulus; US), and as they become associated, the cue (conditioned stimulus; CS) may come to elicit a variety of conditioned responses (CRs) that anticipate reward delivery. CRs can take many forms, depending not only on the nature of the US, but also the properties of the CS [35], including whether the CS is attributed with incentive salience [2], [27], [43], [52], [54], [61]. For example, if presentation of a discrete, localizable CS is consistently followed by food reward in a response-independent manner some animals learn to approach the CS itself (termed “sign-trackers”; STs) [34], whilst others approach the site of food delivery (termed “goal trackers”; GTs) [6], [74]. Although both STs and GTs learn the predictive value of the CS – it comes to evoke an anticipatory CR in both – it has been suggested that the form of the CR differs in part because STs attribute greater incentive salience to the CS than GTs [1], [27], [44], [54], [61], [66]. Not only are STs more attracted to the CS, but it is a more effective conditioned reinforcer and more effective in renewing reward-seeking behavior in STs than GTs [54], [62], [71]. Furthermore, performance on the Pavlovian conditioned approach (PCA) task using food reward predicts the extent to which discrete drug cues acquire control over motivated behavior [45], [55], [61]. Thus, it appears that the cue is an equally effective CS in STs and GTs, but it acts as a more powerful, motivating incentive stimulus in STs.

Both food and drug cues engage so-called brain ‘motive circuits’ to a greater extent in STs than GTs [28], [29], [73], which may contribute to STs being more susceptible to develop impulse control disorders, such as addiction [4], [29], [61], [69]. Addiction-like behaviors tend to persist despite non-reinforcement [22], therefore, one might expect sign-tracking behavior to be more resistant to extinction than goal-tracking. However, in previous studies involving the extinction of instrumental responding for food [71] or cocaine [57], [59], [72], there was no difference between STs and GTs in the rate of instrumental extinction. However, in the studies above extinction was conducted in the absence of a reward cue (CS), or an explicit reward cue was not used in training. In another study presentation of an increasingly aversive outcome progressively decreased instrumental responding for cocaine at the same rate in STs and GTs, despite continued presence of a cue associated with cocaine delivery [62]. On the other hand, in one study in which extinction of cocaine self-administration behavior was conducted in the presence of the reward cue, STs were somewhat more resistant to extinction than GTs, but the effect was very small and was only evident on the first day (out of 28 days) of extinction training [58]. Thus, there is no compelling evidence that STs and GTs differ in extinction learning in an instrumental task, but we are not aware of any study comparing Pavlovian extinction in STs and GTs.

It is important to examine Pavlovian extinction because, although there are many similarities in the processes involved in instrumental and Pavlovian extinction, there are also differences (e.g., [65]). Beckmann and Chow did examine Pavlovian extinction using a 2-CS procedure, in which rats developed a ST CR to a lever-CS and a GT CR to a tone-CS, when both CSs were presented within the same session. They reported that when the lever and tone were no longer reinforced the GT CR extinguished faster than the ST CR [3]. However, in this study all rats showed a ST CR during lever-CS presentation; i.e., no rats showed a GT CR in response to the lever-CS. Therefore, it is still not known whether STs and GTs differ in Pavlovian extinction, when the same CS evokes two different CRs in different individuals.

Thus, we conducted a series of studies on extinction in STs and GTs. In the first study, after initial training, the US was simply withheld over 4-8 days of extinction training. Next, we examined within-session extinction and recovery of ST and GT CRs during alternating blocks of rewarded (R) and non-rewarded (NR) trials. The R and NR periods were signaled by either a diffuse and continuous change in illumination (experiment 2) or by spatially and temporally discrete light cues (experiment 3). These signals were intended to facilitate discrimination between R and NR phases by acting as occasion setters, indicating that the CS would be rewarded when accompanied by one signal but would not be rewarded when accompanied by the other [8], [9], [16], [36]. Finally, in a fourth experiment changes in illumination were used as discriminative stimuli [9], [11] signaling whether an instrumental response would be reinforced or not. STs were highly resistant to extinction, relative to GTs, but only in the Pavlovian setting, when the CS was present.

Section snippets

Subjects

A total of 133 male Sprague–Dawley rats purchased from Charles River or Harlan were used [26]. All rats weighed 250–275 g on arrival, and were singly housed in standard polycarbonate cages with food and water available ad libitum throughout experiments (i.e., the rats were not food deprived at any time). Rats were housed on a reverse 12-h light/12-h dark cycle, with all testing taking place during the dark phase of the cycle. Testing began after one week of acclimation and handling. All

PCA training: ST and GT groups

As described previously, individual rats differed in the form of conditioned approach behavior they developed during the PCA training phase, with some rats primarily approaching the lever and others the food cup during the 8-s CS periods [27], [44], [54]. Across the four experiments, 72 rats were classified as STs, and 61 were classified as GTs. Additional rats were classified as intermediates (INs), but these rats were not used further because we wanted to directly compare groups that varied

Discussion

In appetitive Pavlovian conditioning, when a CS is associated with delivery of a reward, animals may acquire many different anticipatory CRs depending on the form of both the US and the CS [20], [35]. However, as seen here, even when subjected to exactly the same training procedures, using the same CS and US, different individuals may acquire quite different CRs (also see [6], [27], [62], [74]. Thus, presentation of a lever-CS, paired with delivery of a food reward, evoked a ST CR (approach and

Acknowledgements

This study was supported by P01-DA031656 (TER), T32-DA007267 (AMA), T32-DA007268 (AMA, BFS), F32-DA038383 (BFS), K08-DA037912-01 (JDM), the University of Michigan Department of Psychiatry (U032826; JDM), and a Department of Defense (DoD) National Defense Science and Engineering Graduate (NDSEG) fellowship (CJF). We thank Dr. Stephan Anagnostaras for helpful comments on an earlier version of this paper.

References (74)

  • M. Domjan

    Stepping outside the box in considering the C/T ratio?

    Behav. Processes

    (2003)
  • S.B. Flagel et al.

    Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction

    Neuropharmacology

    (2009)
  • S.B. Flagel et al.

    A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico-striatal-thalamic brain regions

    Neuroscience

    (2011)
  • S.B. Flagel et al.

    Antecedents and consequences of drug abuse in rats selectively bred for high and low response to novelty

    Neuropharmacology

    (2014)
  • P.C. Holland

    Occasion setting in Pavlovian conditioning

    (1992)
  • P.C. Holland et al.

    Hippocampus and context in classical conditioning

    Curr. Opin. Neurobiol.

    (1999)
  • A.M. Lomanowska et al.

    Inadequate early social experience increases the incentive salience of reward-related cues in adulthood

    Behav. Brain Res.

    (2011)
  • V. Lovic et al.

    Rats prone to attribute incentive salience to reward cues are also prone to impulsive action

    Behav. Brain Res.

    (2011)
  • J.D. Morrow et al.

    Individual variation in the propensity to attribute incentive salience to an appetitive cue predicts the propensity to attribute motivational salience to an aversive cue

    Behav. Brain Res.

    (2011)
  • J. Peters et al.

    d-Cycloserine administered directly to infralimbic medial prefrontal cortex enhances extinction memory in sucrose-seeking animals

    Neuroscience

    (2013)
  • M.J. Robinson et al.

    Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues

    Behav. Brain Res.

    (2014)
  • T.E. Robinson et al.

    Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences

    Biol. Psychiatry

    (2009)
  • T.E. Robinson et al.

    On the motivational properties of reward cues: individual differences

    Neuropharmacology

    (2014)
  • B.T. Saunders et al.

    A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction

    Biol. Psychiatry

    (2010)
  • B.T. Saunders et al.

    Individual variation in resisting temptation: implications for addiction

    Neurosci. Biobehav. Rev.

    (2013)
  • T.P. Todd et al.

    Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

    Neurobiol. Learn. Mem.

    (2014)
  • A. Tomie et al.

    Individual differences in pavlovian autoshaping of lever pressing in rats predict stress-induced corticosterone release and mesolimbic levels of monoamines

    Pharmacol. Biochem. Behav.

    (2000)
  • A. Tomie et al.

    Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse

    Brain Res. Rev.

    (2008)
  • B.J. Tunstall et al.

    Sign-tracking predicts increased choice of cocaine over food in rats

    Behav. Brain Res.

    (2015)
  • L.M. Yager et al.

    Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues

    Behav. Brain Res.

    (2010)
  • P. Anselme

    Incentive salience attribution under reward uncertainty: a Pavlovian model

    Behav. Processes

    (2014)
  • J.S. Beckmann et al.

    Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence

    Learn. Mem.

    (2015)
  • R.A. Boakes

    Performance on learning to associate a stimulus with positive reinforcement

  • M.E. Bouton

    Context and behavioral processes in extinction

    Learn. Mem.

    (2004)
  • M.E. Bouton et al.

    Contextual control of discriminated operant behavior

    J. Exp. Psychol. Anim. Learn. Cogn.

    (2014)
  • M. Burns et al.

    Topography of spatially directed conditioned responding: effects of context and trial duration

    J. Exp. Psychol. Anim. Behav. Process

    (2001)
  • J.J. Clark et al.

    Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training

    J. Neurosci.

    (2013)
  • Cited by (70)

    View all citing articles on Scopus
    View full text