Reversal learning in Parkinson's disease depends on medication status and outcome valence

https://doi.org/10.1016/j.neuropsychologia.2006.03.030Get rights and content

Abstract

We investigated the role of dopamine in distinct forms of reversal shifting by comparing two groups of patients with mild Parkinson's disease (PD), one ON and one OFF their normal dopaminergic medication. In accordance with our previous work, patients ON medication exhibited impaired reversal shifting relative to patients OFF medication. The present results extend previous studies by showing that the medication-induced deficit on reversal shifting was restricted to conditions where reversals were signaled by unexpected punishment. By contrast, patients ON medication performed as well as patients OFF medication and controls when the reversal was signaled by unexpected reward. The medication-induced deficit was particularly pronounced in patients on the dopamine D3 receptor agonist pramipexole. These data indicate that dopaminergic medication in PD impairs reversal shifting depending on the motivational valence of unexpected outcomes.

Introduction

The mesocortical and nigrostriatal dopamine (DA) systems are well known to play a role in cognitive and reward-related processing (Brozoski, Brown, Rosvold, & Goldman, 1979; Castner, Williams, & Goldman-Rakic, 2000; Goldman-Rakic, 1992; Hollerman & Schultz, 1998). Human disorders that implicate the DA system, such as Parkinson's disease (PD), attention-deficit/hyperactivity disorder (ADHD) and schizophrenia, are associated with a variety of cognitive deficits, ranging from impulsivity to inflexibility. Treatment with dopaminergic medication may alleviate some of these deficits. However, the relationship between DA and cognitive performance is complex (Arnsten, 1998; Williams & Goldman-Rakic, 1995; Zahrt, Taylor, Mathew, & Arnsten, 1997): Dopaminergic medication may improve or impair cognitive function depending on a number of factors, such as task demands and baseline DA levels in underlying neural circuitry (Arnsten, 1998; Cools, Barker, Sahakian, & Robbins, 2001; Kimberg, D’Esposito, & Farah, 1997; Mattay et al., 2003).

PD is associated with nigrostriatal, and to a lesser extent mesocorticolimbic DA depletion and subtle cognitive impairments even in the early disease stages (Owen et al., 1992; Taylor, Saint-Cyr, & Lang, 1986). Recent evidence indicates that administration of dopaminergic medication, which is known to ameliorate the motor deficits in PD, has more complex effects on cognitive function: Both beneficial and detrimental effects have been observed (Cools et al., 2001; Cools, Barker, Sahakian, & Robbins, 2003; Frank, 2005; Frank, Seeberger, & O’Reilly, 2004; Shohamy et al., 2005a, Shohamy et al., 2005b; Swainson et al., 2000). It has been hypothesized that these contrasting effects reflect an imbalance of DA in distinct regions of the striatum (Cools et al., 2001; Gotham, Brown, & Marsden, 1988; Swainson et al., 2000). In early PD, DA depletion is restricted to the dorsal striatum, whereas the ventral striatum is relatively intact (Farley, Price, & Hornykiewicz, 1977; Kish, Shannak, & Hornykiewicz, 1988). Thus, medication doses necessary to remedy depleted DA levels in the dorsal striatum may detrimentally ‘over-dose’ DA levels in the relatively intact ventral striatum. To test this, we have assessed performance of patients ON and OFF l-Dopa medication on two tasks associated with the dorsal and ventral striatum, respectively (Cools et al., 2001). Consistent with the hypothesis, we found that dopaminergic medication in mild PD remedied impairments in task-switching, associated with the lateral prefrontal cortex and its connections with the severely depleted dorsal striatum (Brass et al., 2003; Meyer et al., 1998; Sohn, Ursu, Anderson, Stenger, & Carter, 2000). Conversely, medication impaired probabilistic reversal learning (Cools et al., 2001, Swainson et al., 2000), associated with the relatively intact ventral striatum and its connections with the ventral prefrontal cortex (Cools, Clark, Owen, & Robbins, 2002; Dias, Robbins, & Roberts, 1996; Divac, Rosvold, & Szwarcbart, 1967; Iversen & Mishkin, 1970). A follow-up functional imaging study in mild PD patients has strengthened this ‘over-dose’ hypothesis by showing that dopaminergic medication modulated the ventral striatum (i.e. the nucleus accumbens), but not the dorsal striatum during the performance of a probabilistic reversal shifting paradigm (Cools et al., submitted for publication). The findings are consistent with observations from animal studies suggesting that the dopaminergic modulation of cognitive function adheres to an ‘inverted U’ function whereby excessive, as well as insufficient DA receptor stimulation impairs cognitive performance (Arnsten, 1998; Williams & Goldman-Rakic, 1995; Zahrt et al., 1997).

A recent study by Frank et al. (2004) extended the above-described contrasting effects of dopaminergic medication on cognitive flexibility to the domain of outcome-based learning.1 Frank et al. (2004) showed that relative to PD patients ON medication, PD patients OFF medication were better at learning from negative outcomes than at learning from positive outcomes. Thus, patients OFF medication exhibited an increased tendency towards ‘not-choosing’ (i.e. avoiding) a previously punished stimulus (an increased ‘NoGO’ bias) relative to patients ON medication. By contrast, patients ON medication learned more from positive than negative outcomes and accordingly, exhibited an increased ‘GO’ tendency towards choosing a previously rewarded stimulus (Frank et al., 2004). This profile was predicted by their computational model, which simulated transient changes in DA following positive and negative outcomes, and subsequent contrasting effects on the direct and indirect pathways within the basal ganglia system: DA was thought to excite the direct or ‘GO’ pathway, which facilitates rewarded responding, while inhibiting the indirect or ‘NoGO’ pathway, which suppresses non-rewarded responding. It was proposed that DA bursts, which occur when animals receive unexpected reward, increase plasticity in the direct pathway (supporting ‘GO’ learning). Conversely, plasticity in the indirect pathway (supporting ‘NoGO’ or avoidance learning) was proposed to be increased by DA dips, which occur when an expected reward is omitted. In the model, elevated (tonic) levels of DA following dopaminergic medication blocked the effects of normal phasic ‘DA dips’, which are thought to occur following reward omission (i.e. a form of punishment) (Hollerman & Schultz, 1998). The medication-induced attenuation of phasic ‘DA dips’ impaired reversal learning by diminishing the normal ‘NoGO’ bias in learning from punishment (Frank, 2005). Although this model did not explicitly take into account the spatiotemporal progression of DA depletion from the dorsal to the ventral striatum in PD, it did provide a mechanistic account of the detrimental effect of dopaminergic medication on outcome-related functioning associated with relatively intact ventral fronto-striatal circuitry.

These data raise the question whether the previously observed medication-induced deficit on reversal shifting is valence-specific; that is, restricted to conditions where the reversal is signaled by an unexpected negative outcome. The specific aim of the present study was to examine the hypothesis that dopaminergic medication in mild PD impairs reversal shifting based on unexpected negative, but not positive outcomes. Such a selective punishment-based reversal deficit would support the existence of different representations of reward- and punishment-based learning signals (Daw, Kakade, & Dayan, 2002; Frank et al., 2004; O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001; Seymour et al., 2005). We examined performance of 2 groups of 10 patients with mild PD, 1 ON medication and 1 OFF medication, as well as 12 control subjects using a novel paradigm that enabled the separate investigation of learning reversals, signaled by either negative or positive outcomes.

Section snippets

Patients

The study was approved by the UC Berkeley Committee for the Protection of Human Subjects and all subjects gave written informed consent.

Twenty-eight patients with mild PD were recruited from the movement disorders clinics at the Northern California Veterans Administration Medical Center and the University of California, San Francisco. All patients were diagnosed by a neurologist. Selected patients were contacted and a medical history was obtained. Patients with a significant neurological

Effects of medication withdrawal

Fig. 3 presents the total number of stages performed as a function of valence and group. Patients OFF medication and control subjects completed approximately the same number of stages in the unexpected punishment and the unexpected reward condition. By contrast, patients ON medication completed fewer stages in the unexpected punishment condition relative to the unexpected reward condition. While the omnibus ANOVA with a three-level group factor revealed a trend towards an interaction between

Discussion

The present findings concur with previous observations indicating that dopaminergic medication in mild PD patients impaired probabilistic and concurrent reversal learning in tasks where reversals were signaled by unexpected punishment (Cools et al., 2001, Swainson et al., 2000). Our results significantly extend these previous findings by showing that the medication-induced deficit on reversal shifting was restricted to conditions where reversals were signaled by unexpected punishment and did

Acknowledgements

This work was supported by the American Parkinson's Disease Association, NIH grants MH63901, NS40813 and the Veterans Administration Research Service. We thank Emily Jacobs for testing the age-matched control subjects and analyzing the behavioral data and Daphna Shohamy, Adam Aron, Richard Ivry and Luke Clark as well as three anonymous reviewers for helpful comments on previous drafts of the current manuscript. We are grateful to Dr. D Beckley from the Northern California Veterans

References (68)

  • P.J. Reading et al.

    Dissociable roles of the ventral, medial and lateral striatum on the acquisition and performance of a complex visual stimulus-response habit

    Behavioral Brain Research

    (1991)
  • D. Shohamy et al.

    The role of dopamine in cognitive sequence learning: Evidence from Parkinson's disease

    Behavioral Brain Research

    (2005)
  • R. Swainson et al.

    Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: Possible adverse effects of dopaminergic medication

    Neuropsychologia

    (2000)
  • J.K. Abrams et al.

    Anatomic and functional topography of the dorsal raphe nucleus

    Annals of the New York Academy of Sciences

    (2004)
  • A.F.T. Arnsten

    Catecholamine modulation of prefrontal cortical cognitive function

    Trends in Cognitive Science

    (1998)
  • A. Bechara

    Decision making, impulse control and loss of willpower to resist drugs: A neurocognitive perspective

    Nature Neuroscience

    (2005)
  • A.T. Beck et al.

    An inventory for measuring depression

    Archives of General Psychiatry

    (1961)
  • M. Brass et al.

    When the same response has different meaning: Recoding the response meaning in the lateral prefrontal cortex

    NeuroImage

    (2003)
  • T.J. Brozoski et al.

    Cognitive deficit caused by regional depletion of dopamine in the prefrontal cortex of rhesus monkeys

    Science

    (1979)
  • C. Carver et al.

    Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The bis/bas scales

    Journal of Personality and Social Psychology

    (1994)
  • S.A. Castner et al.

    Reversal of antipsychotic induced working memory deficits by short-term dopamine d1 receptor stimulation

    Science

    (2000)
  • R. Cools et al.

    Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands

    Cerebral Cortex

    (2001)
  • R. Cools et al.

    Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging

    Journal of Neuroscience

    (2002)
  • Cools, R., Lewis, S., Clark, L., Barker, R., & Robbins, T. (submitted for publication). l-Dopa disrupts activity in the...
  • M.R. Delgado et al.

    Tracking the hemodynamic responses to reward and punishment in the striatum

    Journal of Neurophysiology

    (2000)
  • R. Dias et al.

    Dissociation in prefrontal cortex of affective and attentional shifts

    Nature

    (1996)
  • I. Divac et al.

    Behavioral effects of selective ablation of the caudate nucleus

    Journal of Comparative Physiology and Psychology

    (1967)
  • M.L. Dodd et al.

    Pathological gambling caused by drugs used to treat Parkinson's disease

    Archives of Neurology

    (2005)
  • E. Driver-Dunckley et al.

    Pathological gambling associated with dopamine agonist therapy in Parkinson's disease

    Neurology

    (2003)
  • S. Fahn et al.

    Unified Parkinson's disease rating scale

  • A.J. Fallgatter et al.

    Allelic variation of serotonin transporter function modulates the brain electrical response for error processing

    Neuropsychopharmacology

    (2004)
  • I. Farley et al.

    Dopamine in the limbic regions of the human brain: Normal and abnormal

    Advances in Biochemistry and Psychopharmacology

    (1977)
  • M.J. Frank

    Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism

    Journal of Cognitive Neuroscience

    (2005)
  • M.J. Frank et al.

    By carrot or by stick: Cognitive reinforcement learning in Parkinsonism

    Science

    (2004)
  • Cited by (248)

    • Current concepts in treating mild cognitive impairment in Parkinson's disease

      2022, Neuropharmacology
      Citation Excerpt :

      In other studies, agonists such as PD128907, 7-OH-DPAT, BP-897, and pramipexole have shown either an improvement or exacerbation of symptoms, depending on the nature of the cognitive domains tested, the ability of these compounds to affect the D3R, and the involvement of D3R in those cognitive functions. However, most studies have concluded that agonist activity at D3R leads to worsening of cognitive symptoms (Smith et al., 1999; Cools et al., 2006; Svenningsson et al., 2012). Clinical trials for pramipexole to treat motor impairment in PD has shown an influence of the genetic polymorphism of the D3R at position 9 (Ser9Gly), with better efficacy in patients having the Ser/Ser allele as compared to Ser/Gly alleles (Liu et al., 2009; Bhathena et al., 2013).

    • Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease

      2022, Progress in Brain Research
      Citation Excerpt :

      Having said that, we also know that the effects of serotonin depletion and dopaminergic medication in PD on relevant tasks are qualitatively different. For example, while dopaminergic medication in PD impairs learning from punishment (Cools et al., 2006), decreases in serotonin with acute tryptophan depletion in fact increases punishment learning (Cools et al., 2008; Robinson et al., 2012). The dopamine overdose hypothesis was put forward originally by Gotham et al. (1986), Gotham et al. (1988); see also Poewe et al. (1991), and extended by Swainson et al. (2000) and Cools et al. (2001a, 2003), Rowe et al. (2008).

    View all citing articles on Scopus
    View full text