Neurobiological basis of individual variation in stimulus-reward learning

https://doi.org/10.1016/j.cobeha.2016.12.004Get rights and content

Highlights

  • There is considerable individual variation in conditioned responses to reward cues.

  • Cues can evoke simple conditioned responses and/or complex motivational states.

  • When a reward cue becomes an incentive stimulus it can elicit maladaptive behavior.

  • We can exploit variation to elucidate the neurobiology of cue-elicited behavior.

  • Distinct neural circuits are engaged when a cue has incentive vs. predictive value.

Cues in the environment can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, cues in the environment may also serve as powerful motivators that contribute to maladaptive patterns of behavior, such as addiction. Importantly, and central to this article, there is considerable individual variation in the extent to which reward cues gain motivational control over behavior. Here we describe an animal model that captures this individual variation, allowing us to better understand the psychological and neurobiological processes that underlie cue-evoked behaviors. When a discrete cue is paired with a food reward in a Pavlovian manner it acquires greater control over motivated behavior in some rats (‘sign-trackers’, STs) than in others (‘goal-trackers’, GTs). We review studies that have exploited this animal model to parse the neurobiological mechanisms involved in learning associations between stimuli vs. those involved in attributing incentive salience to those same stimuli. The latter seems to be dependent on dopamine and subcortical circuits, whereas the former may engage more cortical ‘top-down’ mechanisms.

Introduction

Cues (conditional stimuli, CSs) that predict the impending delivery of biologically significant events (unconditional stimuli, USs), such as a food reward, acquire the ability to control behavior, or produce a conditioned response (CR), via Pavlovian learning mechanisms [1]. The same is true for stimuli associated with aversive events, but here we will focus only on cues associated with rewards. The ability of a CS to evoke simple reflexive CRs, such as salivation in the case of Pavlov’s dogs, is well known. It is less well appreciated, however, that CSs can also acquire the ability to evoke complex emotional and motivational states [2, 3, 4, 5]. This latter transformation is thought to occur if a CS is attributed with incentive salience and thus acquires the properties of an incentive stimulus [2, 3, 4, 6]. Incentive stimuli: (1) bias attention towards them and can elicit approach into close proximity with them; (2) become desirable themselves, in the sense that an animal will work for access to the stimulus alone (i.e., they act as conditioned reinforcers); and (3) can instigate or invigorate reward-seeking behavior (as in Pavlovian-to-instrumental transfer effects, PIT). Incentive stimuli can guide behavior in adaptive ways, leading one towards valuable resources such as food, water, or a potential mate. However, such cues may also serve as powerful motivators that lead to maladaptive patterns of behavior, as in over-eating and addiction. Importantly, there is considerable individual variation in the extent to which CSs act as incentive stimuli and gain motivational control over behavior [7].

In the laboratory, if a discrete and localizable Pavlovian CS, such as presentation of a lever, is reliably paired with presentation of a food reward, some rats come to approach the CS (Figure 1), even though no response is required for delivery of the reward [8]. This is called ‘sign-tracking’ [9, 10]. In contrast, upon presentation of the lever-CS, other rats go to the location of impending reward delivery (Figure 1) [8]. This CR is called ‘goal-tracking’ [11]. It is important to note that the lever-CS is an equally effective predictive stimulus (CS) in both sign-trackers (STs) and goal-trackers (GTs) – they learn their respective CRs at the same rate – but only in STs does the lever-CS acquire the properties of an incentive stimulus [12]. That is, for STs, the CS is more attractive and elicits approach towards it, is a more effective conditioned reinforcer, and is more effective at instigating reward-seeking behavior relative to GTs [12, 13, 14]. Importantly, this variation in the ability of a CS to acquire incentive salience is captured best by a localizable CS (i.e., lever or light) [15], and, in rats, is not apparent when the CS is a tone [16]. Furthermore, tone stimuli paired with a food reward are attributed with less incentive value than a lever-CS [16, 17].

Sign-trackers are also more resistant to extinction of their CR than GTs [18] and will continue to approach the CS even if contact with it results in omission of the reward [19], indicating that approach behavior is not supported by response reinforcement (i.e., by food delivery reinforcing an action already made) [20]. It has also been shown that sign-tracking behavior becomes more pronounced when the relationship between the CS and the reward is uncertain, such that the probability of the reward following CS presentation changes [21]. These findings provide further evidence for dissociation between the predictive vs. incentive value of the CS (see also Ref. [22]). Thus, a lever-CS acquires all of the properties of an incentive stimulus in some individuals (STs), but not others (GTs). Importantly, this individual variation in the propensity to attribute incentive salience to a discrete or localizable CS has been described not only for food predictive cues, but also for cues that predict drug rewards [7, 15]. The current article will focus on what we have learned about the neurobiological mechanisms of stimulus-reward learning by exploiting this individual variation.

Section snippets

Dopamine

There has been considerable research on the role of dopamine (DA) in stimulus-reward learning, and one popular hypothesis is that phasic DA signals serve as a prediction error signal necessary for learning associations (for recent review see Ref. [22]). Given that STs and GTs learn CS-US associations equally well, but differ in the degree to which they attribute incentive salience to the CS [12], we have used this animal model to parse the role of dopamine in stimulus-reward learning [23]. The

Conclusions

We have provided a review of converging data that implicate several brain regions and possible circuits in mediating the attribution of incentive salience to reward cues (Figure 2). Using the sign-tracker/goal-tracker animal model, we, and others, have demonstrated that the role of dopamine is to encode the incentive – and not the predictive – properties of reward cues. Furthermore, the cortico–striato–pallido–thalamic loops of the ‘motive circuit’ are engaged only when a reward cue is

Conflict of interest statement

Nothing declared.

Funding sources

Support for the authors and the studies reviewed in this manuscript is provided by grants from the National Institute on Drug Abuse (NIDA): P01 DA031656 (SBF, TER), P50 DA037844 (SBF, TER) and R01 DA038599 (SBF).

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

  • • of special interest

  • •• of outstanding interest

Acknowledgements

We would like to thank the former and present members of the Robinson and Flagel laboratories who contributed to some of the studies reviewed here and who prompted insightful discussions surrounding the topic.

References (59)

  • J.J. Clark et al.

    Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training

    J Neurosci

    (2013)
  • P.W. Kalivas et al.

    The neural basis of addiction: a pathology of motivation and choice

    Am J Psychiatry

    (2005)
  • S.B. Flagel et al.

    A food predictive cue must be attributed with incentive salience for it to induce c-fos mRNA expression in cortico–striatal–thalamic brain regions

    Neuroscience

    (2011)
  • J.L. Haight et al.

    A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses

    Front Behav Neurosci

    (2014)
  • G. Paolone et al.

    Cholinergic control over attention in rats prone to attribute incentive salience to reward cues

    J Neurosci

    (2013)
  • C.L. Danna et al.

    The habenula governs the attribution of incentive salience to reward predictive cues

    Front Hum Neurosci

    (2013)
  • S.E. Chang et al.

    Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats

    Eur J Neurosci

    (2015)
  • A.M. Ahrens et al.

    Neural activity in the ventral pallidum encodes variation in the incentive value of a reward cue

    J Neurosci

    (2016)
  • S. Garofalo et al.

    Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior

    Front Behav Neurosci

    (2015)
  • I. Pavlov

    Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex

    (1927)
  • K.C. Berridge

    Reward learning: reinforcement, incentives and expectations

  • D. Bindra

    How adaptive behavior is produced: a perceptual–motivation alternative to response reinforcement

    Behav Brain Sci

    (1978)
  • R.A. Rescorla

    Pavlovian conditioning: it’s not what you think it is

    Am Psychol

    (1988)
  • A.L. Milton et al.

    The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction

    Eur J Neurosci

    (2010)
  • P.L. Brown et al.

    Auto-shaping of the pigeon’s key-peck

    J Exp Anal Behav

    (1968)
  • E. Hearst et al.

    Sign-tracking: the stimulus-reinforcer relation and directed action

    Monograph of the Psychonomic Society

    (1974)
  • R.A. Boakes et al.

    A study of misbehavior: token reinforcement in the rat

    J Exp Anal Behav

    (1978)
  • L.M. Yager et al.

    A classically conditioned cocaine cue acquires greater control over motivated behavior in rats prone to attribute incentive salience to a food cue

    Psychopharmacology (Berl)

    (2013)
  • P.J. Meyer et al.

    The form of a conditioned stimulus can influence the degree to which it acquires incentive motivational properties

    PLoS One

    (2014)
  • Cited by (77)

    • Separating desire from prediction of outcome value

      2023, Trends in Cognitive Sciences
    View all citing articles on Scopus
    View full text