Elsevier

Neuroscience

Volume 345, 14 March 2017, Pages 12-26
Neuroscience

Review
The neural basis of reversal learning: An updated perspective

https://doi.org/10.1016/j.neuroscience.2016.03.021Get rights and content

Highlights

  • Reversal learning is a widely used test of cognitive flexibility across species.

  • The idea that this learning primarily measures response inhibition has been revised.

  • We describe how it is measured and present new definitions for its construct validity.

  • We also present an update of the brain regions and neurotransmitters that support it.

Abstract

Reversal learning paradigms are among the most widely used tests of cognitive flexibility and have been used as assays, across species, for altered cognitive processes in a host of neuropsychiatric conditions. Based on recent studies in humans, non-human primates, and rodents, the notion that reversal learning tasks primarily measure response inhibition, has been revised. In this review, we describe how cognitive flexibility is measured by reversal learning and discuss new definitions of the construct validity of the task that are serving as a heuristic to guide future research in this field. We also provide an update on the available evidence implicating certain cortical and subcortical brain regions in the mediation of reversal learning, and an overview of the principal neurotransmitter systems involved.

Introduction

Cognitive flexibility, the ability to rapidly change behavior in the face of changing circumstances, is disrupted in many psychiatric and neurological disorders. Determining the neural basis of cognitive flexibility is therefore important for understanding the pathophysiology of these disorders and potentially developing treatments. To study the neural substrates of cognitive flexibility in rodents, nonhuman primates, and humans, researchers have often used a set of paradigms collectively referred to as reversal learning. Across species, these paradigms are subtly different, but importantly they all assess cognitive flexibility by evaluating adaptive responding in the face of changing stimulus-outcome (S-O) or response-outcome (R-O) contingencies.

Over the years, reversal learning has become a pre-eminent test of cognitive flexibility and has been used to characterize altered cognitive processes in a host of neuropsychiatric disorders, including substance abuse, obsessive compulsive disorder, psychopathy, Parkinson’s disease, schizophrenia, and to assess cognition at certain developmental time periods such as adolescence (Swainson et al., 2000, Remijnse et al., 2006, Finger et al., 2008, Brigman et al., 2009, Leeson et al., 2009, van der Schaaf et al., 2011, Izquierdo and Jentsch, 2012). Despite its long history of use, reversal learning continues to be an essential experimental paradigm for assessing cognitive function. Indeed, recent years have seen a precipitous rise in the number of published studies using reversal learning, with almost equal focus on rodent, monkey and human subjects (Fig. 1).

While the literature on the neural basis of reversal and the interpretation of findings using this task have been reviewed elsewhere (Clark et al., 2004, Izquierdo and Jentsch, 2012, Costa et al., 2015, Hamilton and Brigman, 2015, Wassum and Izquierdo, 2015), our aim here is to: (1) consider how reversal can be measured and compare different versions of the paradigm across species; (2) provide an updated perspective on the construct validity of reversal learning paradigms; (3) discuss current thinking on the major neural circuits mediating the ability to flexibly change behavior; and (4) review the neurochemical modulation of the cognitive processes engaged during reversal learning.

Section snippets

Reversal learning paradigms across species

In the classic reversal learning paradigm used in humans (Fellows and Farah, 2003a), monkeys (Butter, 1969) and rodents (Schoenbaum et al., 2000), subjects are trained to discriminate between two visual stimuli or spatial locations, one of which is rewarded every time it is chosen and the other which is not. After successful discrimination learning has been demonstrated by reaching a criterion level of performance, the outcomes associated with the two stimuli are reversed and subjects are again

What does reversal learning measure?

Reversal learning requires a subject to flexibly adjust their behavior when the reward-related contingencies that they have previously learned are reversed. For some time, a widely voiced idea was that reversal leaning paradigms primarily measured inhibitory control of responding (Jones and Mishkin, 1972). Based on experiments in humans, monkeys, and rodents (Fellows and Farah, 2005, Chudasama et al., 2007, Schoenbaum et al., 2009a), this view has been revisited and other tasks have been

Cortical regions

Neuroimaging studies report increased activity in orbitofrontal cortex (OFC) and medial prefrontal cortex (mPFC) in human subjects performing reversals (Nagahama et al., 2001, Cools et al., 2002, Kringelbach and Rolls, 2003, Remijnse et al., 2005, Ghahremani et al., 2010), and patients with lesions of these regions exhibit reversal learning deficits (Rolls et al., 1994b, Fellows and Farah, 2003b, Hornak et al., 2004). Furthermore, a consistent finding from over fifty years of work in nonhuman

Neurochemical modulation of reversal

The molecular and neurochemical factors influencing reversal learning and the associated cognitive domains affected are yet to be fully understood (Izquierdo and Jentsch, 2012). Here we consider what have been the most intensively studied neurotransmitter systems in reversal learning – serotonin, dopamine and glutamate.

Concluding remarks

In this review we summarized evidence that reversal learning paradigms generally test the ability to learn specific S-O associations, to estimate the likelihood that reversals can occur given accumulated evidence, and/or to generate a representation of option or task space. More investigation is needed to design behavioral tasks to extract and further describe the neural mechanisms of these individual processes, which should then allow even better comparisons between species and across

Conflict of interest

The authors report no conflicts of interest.

Acknowledgments

We are very grateful to Dr. Russel Morton for constructing some of the figures and helpful comments from the Izquierdo lab on a previous version of this manuscript. AI is supported by the UCLA Division of Life Sciences Recruitment and Retention fund. JLB is supported by National Institute on Alcohol Abuse and Alcoholism grant 1P50AA022534-01. PHR is supported by generous seed funds from the Icahn School of Medicine at Mount Sinai and a Brain and Behavior Research Foundation Young Investigator

References (224)

  • B. Cagniard et al.

    Dopamine scales performance in the absence of new learning

    Neuron

    (2006)
  • X. Cai et al.

    Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation

    Neuron

    (2014)
  • P. Calabresi et al.

    Dopamine-mediated regulation of corticostriatal synaptic plasticity

    Trends Neurosci

    (2007)
  • A. Castañé et al.

    Selective lesions of the dorsomedial striatum impair serial spatial reversal learning in rats

    Behav Brain Res

    (2010)
  • B.K. Chau et al.

    Contrasting roles for orbitofrontal cortex and amygdala in credit assignment and learning in macaques

    Neuron

    (2015)
  • L. Clark et al.

    The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning

    Brain Cogn

    (2004)
  • C.A. Cordova et al.

    Impaired executive function following ischemic stroke in the rat medial prefrontal cortex

    Behav Brain Res

    (2014)
  • N.D. Daw et al.

    The computational neurobiology of learning and reward

    Curr Opin Neurobiol

    (2006)
  • D.A. De Steno et al.

    Role for dopamine D2 receptors in reversal learning

    Neuroscience

    (2009)
  • H.E. den Ouden et al.

    Dissociable effects of dopamine and serotonin on reversal learning

    Neuron

    (2013)
  • K. Doya

    What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?

    Neural Netw

    (1999)
  • L. Fellini et al.

    Re-evaluating the PCP challenge as a pre-clinical model of impaired cognitive flexibility in schizophrenia

    Eur Neuropsychopharmacol

    (2014)
  • S.B. Floresco et al.

    Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure

    Behav Brain Res

    (2008)
  • S. Ghods-Sharifi et al.

    Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning

    Neurobiol Learn Memory

    (2008)
  • S.M. Groman et al.

    Monoamine levels within the orbitofrontal cortex and putamen interact to predict reversal learning performance

    Biol Psychiatry

    (2013)
  • A.N. Hampton et al.

    Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex

    Neuron

    (2007)
  • C. Ineichen et al.

    Establishing a probabilistic reversal learning test in mice: evidence for the processes mediating reward-stay and punishment-shift behaviour and for their modulation by serotonin

    Neuropharmacology

    (2012)
  • A. Izquierdo et al.

    Impaired reward learning and intact motivation after serotonin depletion in rats

    Behav Brain Res

    (2012)
  • A. Izquierdo et al.

    Genetic and dopaminergic modulation of reversal learning in a touchscreen-based operant procedure for mice

    Behav Brain Res

    (2006)
  • Z. Abdul-Monim et al.

    Sub-chronic psychotomimetic phencyclidine induces deficits in reversal learning and alterations in parvalbumin-immunoreactive expression in the rat

    J Psychopharmacol

    (2007)
  • A.R. Adamantidis et al.

    Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior

    J Neurosci

    (2011)
  • J. Alsio et al.

    The role of 5-HT2C receptors in touchscreen visual reversal learning in the rat: a cross-site study

    Psychopharmacology

    (2015)
  • D.M. Bannerman et al.

    GluR-A-deficient mice display normal acquisition of a hippocampus-dependent spatial reference memory task but are impaired during spatial reversal

    Behav Neurosci

    (2003)
  • A. Bari et al.

    Serotonin modulates sensitivity to reward and negative feedback in a probabilistic reversal learning task in rats

    Neuropsychopharmacology

    (2010)
  • R.L. Barlow et al.

    Markers of serotonergic function in the orbitofrontal cortex and dorsal raphe nucleus predict individual variation in spatial-discrimination serial reversal learning

    Neuropsychopharmacology

    (2015)
  • K. Battig et al.

    Comparison of the effects of frontal and caudate lesions on discrimination learning in monkeys

    J Comp Physiol Psychol

    (1962)
  • M.A. Bermudez et al.

    Responses of amygdala neurons to positive reward-predicting stimuli depend on background reward (contingency) rather than stimulus-reward pairing (contiguity)

    J Neurophysiol

    (2010)
  • J.M. Birrell et al.

    Medial frontal cortex mediates perceptual attentional set shifting in the rat

    J Neurosci

    (2000)
  • G.B. Bissonette et al.

    Double dissociation of the effects of medial and orbital prefrontal cortical lesions on attentional and affective shifts in mice

    J Neurosci

    (2008)
  • V. Boulougouris et al.

    Dopamine D2/D3 receptor agonist quinpirole impairs spatial reversal learning in rats: investigation of D3 receptor involvement in persistent behavior

    Psychopharmacology

    (2009)
  • V. Boulougouris et al.

    Dissociable effects of selective 5-HT2A and 5-HT2C receptor antagonists on serial spatial reversal learning in rats

    Neuropsychopharmacology

    (2008)
  • V. Boulougouris et al.

    Enhancement of spatial reversal learning by 5-HT2C receptor antagonism is neuroanatomically specific

    J Neurosci

    (2010)
  • J.L. Brigman et al.

    GluN2B in corticostriatal circuits governs choice learning and choice shifting

    Nat Neurosci

    (2013)
  • J.L. Brigman et al.

    Impaired discrimination learning in mice lacking the NMDA receptor NR2A subunit

    Learn Mem

    (2008)
  • J.L. Brigman et al.

    Predictably irrational: assaying cognitive inflexibility in mouse models of schizophrenia

    Front Neurosci

    (2010)
  • J.L. Brigman et al.

    Effects of subchronic phencyclidine (PCP) treatment on social behaviors, and operant discrimination and reversal learning in C57BL/6J mice

    Front Behav Neurosci

    (2009)
  • J.L. Brigman et al.

    Pharmacological or genetic inactivation of the serotonin transporter improves reversal learning in mice

    Cereb Cortex

    (2010)
  • H.D. Brown et al.

    The selective serotonin reuptake inhibitor, escitalopram, enhances inhibition of prepotent responding and spatial reversal learning

    J Psychopharmacol

    (2012)
  • K.A. Burke et al.

    Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with ‘disinhibition’ of responding for previously unrewarded cues

    Eur J Neurosci

    (2009)
  • T.J. Bussey et al.

    Triple dissociation of anterior cingulate, posterior cingulate, and medial frontal cortices on visual discrimination tasks using a touchscreen testing procedure for the rat

    Behav Neurosci

    (1997)
  • Cited by (362)

    View all citing articles on Scopus
    View full text