Instrumental learning in hyperdopaminergic mice
Introduction
Dopamine (DA) has a variety of effects on cortico-basal ganglia circuits. It is critical for the acquisition and modification of adaptive, purposive behaviors, though its effect at the level of neural systems remains controversial (Robinson and Berridge, 2003, Schultz, 1998a, West et al., 2003). In recent years, the role of DA in instrumental learning has also attracted much attention (Reynolds et al., 2001, Wickens and Koetter, 1995, Wickens et al., 2003). According to a popular account, DA serves to stamp in associations between stimulus and response during instrumental conditioning by facilitating heterosynaptic long-term plasticity in the striatum (Wickens et al., 2003). In support of this claim, it has been shown that DA innervation of the sensorimotor striatum is necessary for habit formation in instrumental conditioning (Faure, Haberland, Conde, & El Massioui, 2005). In vitro studies using brain slices have also demonstrated a critical role for DA in striatal plasticity (Kerr and Wickens, 2001, Lovinger et al., 2003).
Studies using either the water maze or the radial arm maze have also shown that the dorsal striatum plays a major role in tasks (e.g., win-stay) in which a discrete stimulus signals the location of the food and the response to be performed (Devan et al., 1999, Devan and White, 1999, Packard and McGaugh, 1992). More importantly, local injection of dopamine agonists into the dorsal striatum appears to enhance the acquisition of these tasks, suggesting a role for striatal dopamine in habit learning (Packard & White, 1991).
The idea that DA serves as the reinforcement signal in S–R habit learning is particularly interesting in light of the growing body of work in defining the neural substrates of instrumental learning. This work has shown that two largely independent neural systems control the learning and performance of instrumental actions such as lever pressing (Corbit and Balleine, 2003, Corbit et al., 2003, Yin et al., 2004, Yin et al., 2005a, Yin et al., 2005b). Initially, as animals learn to press the lever for food, they encode the specific relationship between their actions and the rewarding outcomes, and their behavior is controlled by encoded action–outcome associations. This action–outcome learning depends on the associative cortico-basal ganglia network, particularly that involving the dorsomedial striatum (Yin et al., 2005a, Yin et al., 2005b). After extended training, however, instrumental actions can become habitual, i.e., controlled by antecedent stimuli rather than by outcome expectancy (Dickinson & Balleine, 1993). This more gradual process of habit formation appears to depend on the sensorimotor cortico-basal ganglia network, particularly the dorsolateral striatum (Yin et al., 2004) and dopaminergic afferents to this area (Faure et al., 2005).
The present study used dopamine transporter knockdown (DAT KD) mice to assess the contribution of tonic DA to instrumental learning and performance. After release, DA is rapidly taken up by the high-affinity DAT, a protein expressed exclusively in brain regions where DA is synthesized (West et al., 2003). The DAT KD mice, which develop normally, have a 70% higher level of tonic DA, thus providing a useful tool for examining the effects of enhanced tonic DA on striatum-dependent learning (Pecina et al., 2003, Zhuang et al., 2001).
In particular, we tested the hypothesis that S–R habit learning might be enhanced in these hyperdopaminergic animals. If tonic DA is critical for habit learning, then given the same amount of training the instrumental performance of DAT KD mice should be predicted to be less sensitive to outcome devaluation than WT controls. Furthermore, given the evidence that habits depend for their performance on the motivating aspects of reward-related cues (Holland, 2004), instrumental performance in the DAT KD mice should be predicted to show increased sensitivity to the excitatory effects of Pavlovian cues. These two predictions were assessed in Experiments 1 and 2, respectively.
Section snippets
Subjects and apparatus
Eight wild-type mice and 6 KD mice (all males) were used for both experiments. The generation of DAT KD mice has been described in an earlier paper (Zhuang et al., 2001). Training and testing took place in 7 Med Associates (East Fairfield, VT) operant chambers housed within sound- and light-resistant walls. Each chamber was equipped with a pump fitted with a syringe that could deliver sucrose solution into a recessed magazine in the chamber, as well as a pellet dispenser that can deliver food
Enhanced motivation but normal acquisition of action–outcome learning
During the initial acquisition phase (Fig. 1A), both groups rapidly increased lever pressing over sessions. The acquisition of lever pressing for each reinforcer over 9 days of training was analyzed with a mixed two-way ANOVA. For pellets, there was a main effect of days (F8,12 = 25.2, p < .05), no main effect of genotype (F1,12 = 2.0, p > .05), and no significant interaction between days and genotype (F8,12 = 1.54, p > .05). For sucrose, there was a main effect of days (F8,12 = 15.1, p < .05), no main effect
Discussion
Experiment 1 assessed the content of instrumental learning in DAT KD mice using outcome devaluation, the canonical assay for detecting action–outcome encoding. Both DAT KD mice and their WT controls were able to acquire two actions each earning a different outcome. It should be noted that the instrumental training procedure used in this study is specifically designed to generate considerable sensitivity to outcome devaluation in the performance of the controls.
Although DAT KD mice showed
Acknowledgments
This research was supported by NIMH Grant MH56446 to B.W.B. and NIMH Grant MH66216 to X.Z.
References (38)
- et al.
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Neuron
(2005) - et al.
The role of prelimbic cortex in instrumental conditioning
Behavioural Brain Research
(2003) - et al.
Reward, motivation, and reinforcement learning
Neuron
(2002) - et al.
Effects of medial and lateral caudate-putamen lesions on place- and cue- guided behaviors in the water maze: relation to thigmotaxis
Behavioural Brain Research
(1999) - et al.
Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons
Neuron
(2004) - et al.
Neural mechanisms of reward-related motor learning
Current Opinion in Neurobiology
(2003) - et al.
Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer
Journal of Neuroscience
(2005) - et al.
The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell
Journal of Neuroscience
(2001) - et al.
Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats
European Journal of Neuroscience
(2003) Effects of several extinction treatments upon the integrity of Pavlovian stimulus-outcome associations
Animal Learning & Behavior
(1996)
Parallel information processing in the dorsal striatum: Relation to hippocampal function
Journal of Neuroscience
Actions and responses: The dual psychology of behaviour
Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists
Behavioral Neuroscience
Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation
Journal of Neuroscience
Differences between accumbens core and shell neurons exhibiting phasic firing patterns related to drug-seeking behavior during a discriminative-stimulus task
Journal of Neurophysiology
Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum
Journal of Neuroscience
Relations between Pavlovian-instrumental transfer and reinforcer devaluation
Journal of Experimental Psychology: Animal Behavior Processes
Reward-predicting activity of dopamine and caudate neurons—a possible mechanism of motivational control of saccadic eye movement
Journal of Neurophysiology
Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro
Journal of Neurophysiology
Cited by (59)
Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice
2023, Behavioural Brain ResearchResearch Domain Criteria versus DSM V: How does this debate affect attempts to model corticostriatal dysfunction in animals?
2017, Neuroscience and Biobehavioral ReviewsThe effects of reduced dopamine transporter function and chronic lithium on motivation, probabilistic learning, and neurochemistry in mice: Modeling bipolar mania
2017, NeuropharmacologyCitation Excerpt :Further examination of dose response effects on PRLT combined with extracellular dopamine level analyses are required to test this hypothesis. Previous studies using a Pavlovian association learning task also observed that learning for food was not enhanced nor impaired in DAT KD mice (Cagniard et al., 2006; Yin et al., 2006). Cagniard and colleagues also observed that DAT KD mice learned operant tasks at the same or faster rates than their WT littermates (Cagniard et al., 2006; Hironaka et al., 2004).
Individual variation in resisting temptation: Implications for addiction
2013, Neuroscience and Biobehavioral ReviewsCitation Excerpt :For example, dopamine is implicated in arousal, action selection, cognitive flexibility, and behavioral effort, particularly during instrumental conditioning (Beeler et al., 2012; Cools, 2008; Day et al., 2010; Redgrave et al., 1999; Robbins and Everitt, 1992; Salamone et al., 2007; Wassum et al., 2012). We should note, though, that even for instrumental behaviors, dopamine can modulate responding by scaling performance vigor, or by regulating PIT effects, independent of learning (Cagniard et al., 2006; Yin et al., 2006). Also, we have focused our discussion on appetitive cue processing, but dopamine is also involved in processing aversive or dysphoric motivational states (Aragona and Wang, 2009; Badrinarayan et al., 2012; Chaudhury et al., 2012; Faure et al., 2008; Kapur et al., 2005; Lemos et al., 2012; Oleson et al., 2012; Pezze and Feldon, 2004; Pezze et al., 2001; Richard and Berridge, 2011; Roitman et al., 2008; Tye et al., 2012).
The integrative functions of the basal ganglia
2023, The Integrative Functions of The Basal Ganglia