ReviewThe neural basis of reversal learning: An updated perspective
Introduction
Cognitive flexibility, the ability to rapidly change behavior in the face of changing circumstances, is disrupted in many psychiatric and neurological disorders. Determining the neural basis of cognitive flexibility is therefore important for understanding the pathophysiology of these disorders and potentially developing treatments. To study the neural substrates of cognitive flexibility in rodents, nonhuman primates, and humans, researchers have often used a set of paradigms collectively referred to as reversal learning. Across species, these paradigms are subtly different, but importantly they all assess cognitive flexibility by evaluating adaptive responding in the face of changing stimulus-outcome (S-O) or response-outcome (R-O) contingencies.
Over the years, reversal learning has become a pre-eminent test of cognitive flexibility and has been used to characterize altered cognitive processes in a host of neuropsychiatric disorders, including substance abuse, obsessive compulsive disorder, psychopathy, Parkinson’s disease, schizophrenia, and to assess cognition at certain developmental time periods such as adolescence (Swainson et al., 2000, Remijnse et al., 2006, Finger et al., 2008, Brigman et al., 2009, Leeson et al., 2009, van der Schaaf et al., 2011, Izquierdo and Jentsch, 2012). Despite its long history of use, reversal learning continues to be an essential experimental paradigm for assessing cognitive function. Indeed, recent years have seen a precipitous rise in the number of published studies using reversal learning, with almost equal focus on rodent, monkey and human subjects (Fig. 1).
While the literature on the neural basis of reversal and the interpretation of findings using this task have been reviewed elsewhere (Clark et al., 2004, Izquierdo and Jentsch, 2012, Costa et al., 2015, Hamilton and Brigman, 2015, Wassum and Izquierdo, 2015), our aim here is to: (1) consider how reversal can be measured and compare different versions of the paradigm across species; (2) provide an updated perspective on the construct validity of reversal learning paradigms; (3) discuss current thinking on the major neural circuits mediating the ability to flexibly change behavior; and (4) review the neurochemical modulation of the cognitive processes engaged during reversal learning.
Section snippets
Reversal learning paradigms across species
In the classic reversal learning paradigm used in humans (Fellows and Farah, 2003a), monkeys (Butter, 1969) and rodents (Schoenbaum et al., 2000), subjects are trained to discriminate between two visual stimuli or spatial locations, one of which is rewarded every time it is chosen and the other which is not. After successful discrimination learning has been demonstrated by reaching a criterion level of performance, the outcomes associated with the two stimuli are reversed and subjects are again
What does reversal learning measure?
Reversal learning requires a subject to flexibly adjust their behavior when the reward-related contingencies that they have previously learned are reversed. For some time, a widely voiced idea was that reversal leaning paradigms primarily measured inhibitory control of responding (Jones and Mishkin, 1972). Based on experiments in humans, monkeys, and rodents (Fellows and Farah, 2005, Chudasama et al., 2007, Schoenbaum et al., 2009a), this view has been revisited and other tasks have been
Cortical regions
Neuroimaging studies report increased activity in orbitofrontal cortex (OFC) and medial prefrontal cortex (mPFC) in human subjects performing reversals (Nagahama et al., 2001, Cools et al., 2002, Kringelbach and Rolls, 2003, Remijnse et al., 2005, Ghahremani et al., 2010), and patients with lesions of these regions exhibit reversal learning deficits (Rolls et al., 1994b, Fellows and Farah, 2003b, Hornak et al., 2004). Furthermore, a consistent finding from over fifty years of work in nonhuman
Neurochemical modulation of reversal
The molecular and neurochemical factors influencing reversal learning and the associated cognitive domains affected are yet to be fully understood (Izquierdo and Jentsch, 2012). Here we consider what have been the most intensively studied neurotransmitter systems in reversal learning – serotonin, dopamine and glutamate.
Concluding remarks
In this review we summarized evidence that reversal learning paradigms generally test the ability to learn specific S-O associations, to estimate the likelihood that reversals can occur given accumulated evidence, and/or to generate a representation of option or task space. More investigation is needed to design behavioral tasks to extract and further describe the neural mechanisms of these individual processes, which should then allow even better comparisons between species and across
Conflict of interest
The authors report no conflicts of interest.
Acknowledgments
We are very grateful to Dr. Russel Morton for constructing some of the figures and helpful comments from the Izquierdo lab on a previous version of this manuscript. AI is supported by the UCLA Division of Life Sciences Recruitment and Retention fund. JLB is supported by National Institute on Alcohol Abuse and Alcoholism grant 1P50AA022534-01. PHR is supported by generous seed funds from the Icahn School of Medicine at Mount Sinai and a Brain and Behavior Research Foundation Young Investigator
References (224)
- et al.
Do GluA1 knockout mice exhibit behavioral abnormalities relevant to the negative or cognitive symptoms of schizophrenia and schizoaffective disorder?
Neuropharmacology
(2012) - et al.
Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala
Neuron
(2007) - et al.
Orbital prefrontal cortex and guidance of instrumental behaviour in rats under reversal conditions
Behav Brain Res
(2003) - et al.
Effects of orbitofrontal, infralimbic and prelimbic cortical lesions on serial spatial reversal learning in the rat
Behav Brain Res
(2007) - et al.
The dorsomedial striatum mediates flexible choice behavior in spatial tasks
Behav Brain Res
(2011) - et al.
Stimulus specific deficit on visual reversal learning after lesions of medial prefrontal cortex in the mouse
Behav Brain Res
(2008) - et al.
Stress facilitates late reversal learning using a touchscreen-based visual discrimination procedure in male Long Evans rats
Behav Brain Res
(2015) - et al.
Effects of dorsal and ventral striatal lesions on delayed matching trained with retractable levers
Behav Brain Res
(2001) Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in Macaca mulatta
Physiol Behav
(1969)- et al.
Conditioning and extinction of a food-rewarded response after selective ablations of frontal cortex in rhesus monkeys
Exp Neurol
(1963)
Dopamine scales performance in the absence of new learning
Neuron
Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation
Neuron
Dopamine-mediated regulation of corticostriatal synaptic plasticity
Trends Neurosci
Selective lesions of the dorsomedial striatum impair serial spatial reversal learning in rats
Behav Brain Res
Contrasting roles for orbitofrontal cortex and amygdala in credit assignment and learning in macaques
Neuron
The neuropsychology of ventral prefrontal cortex: decision-making and reversal learning
Brain Cogn
Impaired executive function following ischemic stroke in the rat medial prefrontal cortex
Behav Brain Res
The computational neurobiology of learning and reward
Curr Opin Neurobiol
Role for dopamine D2 receptors in reversal learning
Neuroscience
Dissociable effects of dopamine and serotonin on reversal learning
Neuron
What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?
Neural Netw
Re-evaluating the PCP challenge as a pre-clinical model of impaired cognitive flexibility in schizophrenia
Eur Neuropsychopharmacol
Inactivation of the medial prefrontal cortex of the rat impairs strategy set-shifting, but not reversal learning, using a novel, automated procedure
Behav Brain Res
Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning
Neurobiol Learn Memory
Monoamine levels within the orbitofrontal cortex and putamen interact to predict reversal learning performance
Biol Psychiatry
Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex
Neuron
Establishing a probabilistic reversal learning test in mice: evidence for the processes mediating reward-stay and punishment-shift behaviour and for their modulation by serotonin
Neuropharmacology
Impaired reward learning and intact motivation after serotonin depletion in rats
Behav Brain Res
Genetic and dopaminergic modulation of reversal learning in a touchscreen-based operant procedure for mice
Behav Brain Res
Sub-chronic psychotomimetic phencyclidine induces deficits in reversal learning and alterations in parvalbumin-immunoreactive expression in the rat
J Psychopharmacol
Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior
J Neurosci
The role of 5-HT2C receptors in touchscreen visual reversal learning in the rat: a cross-site study
Psychopharmacology
GluR-A-deficient mice display normal acquisition of a hippocampus-dependent spatial reference memory task but are impaired during spatial reversal
Behav Neurosci
Serotonin modulates sensitivity to reward and negative feedback in a probabilistic reversal learning task in rats
Neuropsychopharmacology
Markers of serotonergic function in the orbitofrontal cortex and dorsal raphe nucleus predict individual variation in spatial-discrimination serial reversal learning
Neuropsychopharmacology
Comparison of the effects of frontal and caudate lesions on discrimination learning in monkeys
J Comp Physiol Psychol
Responses of amygdala neurons to positive reward-predicting stimuli depend on background reward (contingency) rather than stimulus-reward pairing (contiguity)
J Neurophysiol
Medial frontal cortex mediates perceptual attentional set shifting in the rat
J Neurosci
Double dissociation of the effects of medial and orbital prefrontal cortical lesions on attentional and affective shifts in mice
J Neurosci
Dopamine D2/D3 receptor agonist quinpirole impairs spatial reversal learning in rats: investigation of D3 receptor involvement in persistent behavior
Psychopharmacology
Dissociable effects of selective 5-HT2A and 5-HT2C receptor antagonists on serial spatial reversal learning in rats
Neuropsychopharmacology
Enhancement of spatial reversal learning by 5-HT2C receptor antagonism is neuroanatomically specific
J Neurosci
GluN2B in corticostriatal circuits governs choice learning and choice shifting
Nat Neurosci
Impaired discrimination learning in mice lacking the NMDA receptor NR2A subunit
Learn Mem
Predictably irrational: assaying cognitive inflexibility in mouse models of schizophrenia
Front Neurosci
Effects of subchronic phencyclidine (PCP) treatment on social behaviors, and operant discrimination and reversal learning in C57BL/6J mice
Front Behav Neurosci
Pharmacological or genetic inactivation of the serotonin transporter improves reversal learning in mice
Cereb Cortex
The selective serotonin reuptake inhibitor, escitalopram, enhances inhibition of prepotent responding and spatial reversal learning
J Psychopharmacol
Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with ‘disinhibition’ of responding for previously unrewarded cues
Eur J Neurosci
Triple dissociation of anterior cingulate, posterior cingulate, and medial frontal cortices on visual discrimination tasks using a touchscreen testing procedure for the rat
Behav Neurosci
Cited by (362)
Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making
2024, Journal of Mathematical Psychology