Reversal learning in Parkinson's disease depends on medication status and outcome valence
Introduction
The mesocortical and nigrostriatal dopamine (DA) systems are well known to play a role in cognitive and reward-related processing (Brozoski, Brown, Rosvold, & Goldman, 1979; Castner, Williams, & Goldman-Rakic, 2000; Goldman-Rakic, 1992; Hollerman & Schultz, 1998). Human disorders that implicate the DA system, such as Parkinson's disease (PD), attention-deficit/hyperactivity disorder (ADHD) and schizophrenia, are associated with a variety of cognitive deficits, ranging from impulsivity to inflexibility. Treatment with dopaminergic medication may alleviate some of these deficits. However, the relationship between DA and cognitive performance is complex (Arnsten, 1998; Williams & Goldman-Rakic, 1995; Zahrt, Taylor, Mathew, & Arnsten, 1997): Dopaminergic medication may improve or impair cognitive function depending on a number of factors, such as task demands and baseline DA levels in underlying neural circuitry (Arnsten, 1998; Cools, Barker, Sahakian, & Robbins, 2001; Kimberg, D’Esposito, & Farah, 1997; Mattay et al., 2003).
PD is associated with nigrostriatal, and to a lesser extent mesocorticolimbic DA depletion and subtle cognitive impairments even in the early disease stages (Owen et al., 1992; Taylor, Saint-Cyr, & Lang, 1986). Recent evidence indicates that administration of dopaminergic medication, which is known to ameliorate the motor deficits in PD, has more complex effects on cognitive function: Both beneficial and detrimental effects have been observed (Cools et al., 2001; Cools, Barker, Sahakian, & Robbins, 2003; Frank, 2005; Frank, Seeberger, & O’Reilly, 2004; Shohamy et al., 2005a, Shohamy et al., 2005b; Swainson et al., 2000). It has been hypothesized that these contrasting effects reflect an imbalance of DA in distinct regions of the striatum (Cools et al., 2001; Gotham, Brown, & Marsden, 1988; Swainson et al., 2000). In early PD, DA depletion is restricted to the dorsal striatum, whereas the ventral striatum is relatively intact (Farley, Price, & Hornykiewicz, 1977; Kish, Shannak, & Hornykiewicz, 1988). Thus, medication doses necessary to remedy depleted DA levels in the dorsal striatum may detrimentally ‘over-dose’ DA levels in the relatively intact ventral striatum. To test this, we have assessed performance of patients ON and OFF l-Dopa medication on two tasks associated with the dorsal and ventral striatum, respectively (Cools et al., 2001). Consistent with the hypothesis, we found that dopaminergic medication in mild PD remedied impairments in task-switching, associated with the lateral prefrontal cortex and its connections with the severely depleted dorsal striatum (Brass et al., 2003; Meyer et al., 1998; Sohn, Ursu, Anderson, Stenger, & Carter, 2000). Conversely, medication impaired probabilistic reversal learning (Cools et al., 2001, Swainson et al., 2000), associated with the relatively intact ventral striatum and its connections with the ventral prefrontal cortex (Cools, Clark, Owen, & Robbins, 2002; Dias, Robbins, & Roberts, 1996; Divac, Rosvold, & Szwarcbart, 1967; Iversen & Mishkin, 1970). A follow-up functional imaging study in mild PD patients has strengthened this ‘over-dose’ hypothesis by showing that dopaminergic medication modulated the ventral striatum (i.e. the nucleus accumbens), but not the dorsal striatum during the performance of a probabilistic reversal shifting paradigm (Cools et al., submitted for publication). The findings are consistent with observations from animal studies suggesting that the dopaminergic modulation of cognitive function adheres to an ‘inverted U’ function whereby excessive, as well as insufficient DA receptor stimulation impairs cognitive performance (Arnsten, 1998; Williams & Goldman-Rakic, 1995; Zahrt et al., 1997).
A recent study by Frank et al. (2004) extended the above-described contrasting effects of dopaminergic medication on cognitive flexibility to the domain of outcome-based learning.1 Frank et al. (2004) showed that relative to PD patients ON medication, PD patients OFF medication were better at learning from negative outcomes than at learning from positive outcomes. Thus, patients OFF medication exhibited an increased tendency towards ‘not-choosing’ (i.e. avoiding) a previously punished stimulus (an increased ‘NoGO’ bias) relative to patients ON medication. By contrast, patients ON medication learned more from positive than negative outcomes and accordingly, exhibited an increased ‘GO’ tendency towards choosing a previously rewarded stimulus (Frank et al., 2004). This profile was predicted by their computational model, which simulated transient changes in DA following positive and negative outcomes, and subsequent contrasting effects on the direct and indirect pathways within the basal ganglia system: DA was thought to excite the direct or ‘GO’ pathway, which facilitates rewarded responding, while inhibiting the indirect or ‘NoGO’ pathway, which suppresses non-rewarded responding. It was proposed that DA bursts, which occur when animals receive unexpected reward, increase plasticity in the direct pathway (supporting ‘GO’ learning). Conversely, plasticity in the indirect pathway (supporting ‘NoGO’ or avoidance learning) was proposed to be increased by DA dips, which occur when an expected reward is omitted. In the model, elevated (tonic) levels of DA following dopaminergic medication blocked the effects of normal phasic ‘DA dips’, which are thought to occur following reward omission (i.e. a form of punishment) (Hollerman & Schultz, 1998). The medication-induced attenuation of phasic ‘DA dips’ impaired reversal learning by diminishing the normal ‘NoGO’ bias in learning from punishment (Frank, 2005). Although this model did not explicitly take into account the spatiotemporal progression of DA depletion from the dorsal to the ventral striatum in PD, it did provide a mechanistic account of the detrimental effect of dopaminergic medication on outcome-related functioning associated with relatively intact ventral fronto-striatal circuitry.
These data raise the question whether the previously observed medication-induced deficit on reversal shifting is valence-specific; that is, restricted to conditions where the reversal is signaled by an unexpected negative outcome. The specific aim of the present study was to examine the hypothesis that dopaminergic medication in mild PD impairs reversal shifting based on unexpected negative, but not positive outcomes. Such a selective punishment-based reversal deficit would support the existence of different representations of reward- and punishment-based learning signals (Daw, Kakade, & Dayan, 2002; Frank et al., 2004; O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001; Seymour et al., 2005). We examined performance of 2 groups of 10 patients with mild PD, 1 ON medication and 1 OFF medication, as well as 12 control subjects using a novel paradigm that enabled the separate investigation of learning reversals, signaled by either negative or positive outcomes.
Section snippets
Patients
The study was approved by the UC Berkeley Committee for the Protection of Human Subjects and all subjects gave written informed consent.
Twenty-eight patients with mild PD were recruited from the movement disorders clinics at the Northern California Veterans Administration Medical Center and the University of California, San Francisco. All patients were diagnosed by a neurologist. Selected patients were contacted and a medical history was obtained. Patients with a significant neurological
Effects of medication withdrawal
Fig. 3 presents the total number of stages performed as a function of valence and group. Patients OFF medication and control subjects completed approximately the same number of stages in the unexpected punishment and the unexpected reward condition. By contrast, patients ON medication completed fewer stages in the unexpected punishment condition relative to the unexpected reward condition. While the omnibus ANOVA with a three-level group factor revealed a trend towards an interaction between
Discussion
The present findings concur with previous observations indicating that dopaminergic medication in mild PD patients impaired probabilistic and concurrent reversal learning in tasks where reversals were signaled by unexpected punishment (Cools et al., 2001, Swainson et al., 2000). Our results significantly extend these previous findings by showing that the medication-induced deficit on reversal shifting was restricted to conditions where reversals were signaled by unexpected punishment and did
Acknowledgements
This work was supported by the American Parkinson's Disease Association, NIH grants MH63901, NS40813 and the Veterans Administration Research Service. We thank Emily Jacobs for testing the age-matched control subjects and analyzing the behavioral data and Daphna Shohamy, Adam Aron, Richard Ivry and Luke Clark as well as three anonymous reviewers for helpful comments on previous drafts of the current manuscript. We are grateful to Dr. D Beckley from the Northern California Veterans
References (68)
- et al.
l-Dopa is converted to dopamine in serotonergic fibers of the striatum of the rat: A double-labeling immunofluorescence study
Neuroscience Letters
(1995) Differential behavioral effects in frontal lobe disease
Neuropsychologia
(1968)- et al.
Striatal dopamine and learning strategy-an 123I-FP-CIT SPECT study
Neuropsychologia
(2004) - et al.
Functional imaging of neural responses to expectancy and experience of monetary gains and losses
Neuron
(2001) - et al.
l-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease
Neuropsychologia
(2003) - et al.
Opponent interactions between serotonin and dopamine
Neural Network
(2002) - et al.
Compulsive use of dopamine replacement therapy in Parkinson's disease: Reward systems gone awry?
The Lancet
(2003) Selective effects on prefrontal cortex serotonin by dopamine D3 receptor agonism: Interaction with low-dose haloperidol
Progress in Neuropsychopharmacology and Biological Psychiatry
(1997)- et al.
Anxiolytic action on the behavioural inhibition system implies multiple types of arousal contribute to anxiety
Journal of Affective Disorders
(2000) - et al.
In vivo serotonin 5ht(2a) receptor binding and personality traits in healthy subjects: A positron emission tomography study
NeuroImage
(2002)
Dissociable roles of the ventral, medial and lateral striatum on the acquisition and performance of a complex visual stimulus-response habit
Behavioral Brain Research
The role of dopamine in cognitive sequence learning: Evidence from Parkinson's disease
Behavioral Brain Research
Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: Possible adverse effects of dopaminergic medication
Neuropsychologia
Anatomic and functional topography of the dorsal raphe nucleus
Annals of the New York Academy of Sciences
Catecholamine modulation of prefrontal cortical cognitive function
Trends in Cognitive Science
Decision making, impulse control and loss of willpower to resist drugs: A neurocognitive perspective
Nature Neuroscience
An inventory for measuring depression
Archives of General Psychiatry
When the same response has different meaning: Recoding the response meaning in the lateral prefrontal cortex
NeuroImage
Cognitive deficit caused by regional depletion of dopamine in the prefrontal cortex of rhesus monkeys
Science
Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The bis/bas scales
Journal of Personality and Social Psychology
Reversal of antipsychotic induced working memory deficits by short-term dopamine d1 receptor stimulation
Science
Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands
Cerebral Cortex
Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging
Journal of Neuroscience
Tracking the hemodynamic responses to reward and punishment in the striatum
Journal of Neurophysiology
Dissociation in prefrontal cortex of affective and attentional shifts
Nature
Behavioral effects of selective ablation of the caudate nucleus
Journal of Comparative Physiology and Psychology
Pathological gambling caused by drugs used to treat Parkinson's disease
Archives of Neurology
Pathological gambling associated with dopamine agonist therapy in Parkinson's disease
Neurology
Unified Parkinson's disease rating scale
Allelic variation of serotonin transporter function modulates the brain electrical response for error processing
Neuropsychopharmacology
Dopamine in the limbic regions of the human brain: Normal and abnormal
Advances in Biochemistry and Psychopharmacology
Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism
Journal of Cognitive Neuroscience
By carrot or by stick: Cognitive reinforcement learning in Parkinsonism
Science
Cited by (248)
Postnatal Phencyclidine-Induced Deficits in Decision Making Are Ameliorated by Optogenetic Inhibition of Ventromedial Orbitofrontal Cortical Glutamate Neurons
2024, Biological Psychiatry Global Open ScienceDopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients
2022, Neurobiology of Learning and MemoryCurrent concepts in treating mild cognitive impairment in Parkinson's disease
2022, NeuropharmacologyCitation Excerpt :In other studies, agonists such as PD128907, 7-OH-DPAT, BP-897, and pramipexole have shown either an improvement or exacerbation of symptoms, depending on the nature of the cognitive domains tested, the ability of these compounds to affect the D3R, and the involvement of D3R in those cognitive functions. However, most studies have concluded that agonist activity at D3R leads to worsening of cognitive symptoms (Smith et al., 1999; Cools et al., 2006; Svenningsson et al., 2012). Clinical trials for pramipexole to treat motor impairment in PD has shown an influence of the genetic polymorphism of the D3R at position 9 (Ser9Gly), with better efficacy in patients having the Ser/Ser allele as compared to Ser/Gly alleles (Liu et al., 2009; Bhathena et al., 2013).
Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease
2022, Progress in Brain ResearchCitation Excerpt :Having said that, we also know that the effects of serotonin depletion and dopaminergic medication in PD on relevant tasks are qualitatively different. For example, while dopaminergic medication in PD impairs learning from punishment (Cools et al., 2006), decreases in serotonin with acute tryptophan depletion in fact increases punishment learning (Cools et al., 2008; Robinson et al., 2012). The dopamine overdose hypothesis was put forward originally by Gotham et al. (1986), Gotham et al. (1988); see also Poewe et al. (1991), and extended by Swainson et al. (2000) and Cools et al. (2001a, 2003), Rowe et al. (2008).
Theranostic applications of metal–organic frameworks (MOFs)-based materials in brain disorders: Recent advances and challenges
2021, Inorganic Chemistry Communications