Habits under stress: mechanistic insights across different types of learning

Highlights

• Stress favors habitual over cognitive behavior in distinct types of learning.

• The bias toward habits is critically mediated by noradrenaline and glucocorticoids.

• Amygdala orchestrates engagement of cognitive versus habit learning under stress.

Learning can be controlled by reflective, ‘cognitive’ or reflexive, ‘habitual’ systems. An essential question is what factors determine which system governs behavior. Here we review recent evidence from navigation, classification, and instrumental learning, demonstrating that stressful events induce a shift from cognitive to habitual control of learning. We propose that this shift, mediated by noradrenaline and glucocorticoids acting through mineralocorticoid receptors, is orchestrated by the amygdala. Although generally adaptive for coping with acute stress, the bias toward habits comes at the cost of reduced flexibility of learning and may ultimately contribute to stress-related psychopathologies.

Introduction

Adaptive behavior requires an intricate balance of thoughtful processing and efficient responding. Whereas the deliberate evaluation of our environment enables behavioral flexibility, crystallizing repeatedly successful actions into habits promotes behavioral autonomy that frees up cognitive resources. The idea that behavior can be controlled by more reflective or more reflexive processes is shared by several lines of scientific inquiry (Figure 1). Research on navigational learning, dating back to early work of Edward Tolman [1], distinguished between a hippocampus-dependent spatial (‘cognitive’) memory system that uses the relationship between multiple cues in the environment to build a ‘cognitive map’ and a dorsal striatum-dependent stimulus-response (S-R; ‘habit’) memory system that learns associations between responses and single stimuli [2, 3, 4] (Figure 1a). Inspired by neuropsychological data, a similar distinction was made between a hippocampus-dependent ‘cognitive’ system and a dorsal-striatum-dependent ‘habit’ system in probabilistic classification learning [5] (Figure 1b). A parallel strand of research in instrumental learning developed a set of elegant paradigms, allowing the experimental dissociation of goal-directed learning that processes the causal relationship between an action and its consequences, and habitual learning that associates responses with the preceding stimuli without links to the consequences [6] (Figure 1c). Although originally studied in rodents, these modes of instrumental control were shown in humans as well. Corroborating previous lesion data in rodents, these human studies identified the orbitofrontal cortex and dorsomedial striatum as key regions for goal-directed action and the dorsolateral striatum as a locus of habitual responding [7, 8]. Most recently, the concepts of goal-directed and habitual behavior were further developed by computational models suggesting a distinction between model-based and model-free learning [9, 10] (Figure 1d). Model-based control, dependent on prefrontal cortex (PFC) areas, is characterized by a collection of flexible but complex strategies which build an internal model of the environment that aids future planning of actions and their potential outcomes. The dorsal striatum-dependent model-free system involves inflexible and rigid strategies that are driven solely by past outcomes. Specifically, in model-based learning approaches, participants use the task structure to maximize their rewards, whereas in model-free learning, choices are guided by recent experiences instead of the overall task structure.

While these different research traditions are only partly overlapping and important differences exist (e.g. with respect to the operational definition of a habit or to the neural underpinnings of the two modes of behavioral control, in particular the involvement of the dorsolateral versus dorsomedial striatum in habitual forms of behavior; [7, 11, 12]) a key question for all of these conceptualizations is how the ‘cognitive’ and ‘habit’ systems are coordinated. In other words, what factors determine which system may dominate behavior? Overtraining and dual-tasking are known to bias behavior toward the ‘habit system’ [13, 14]. In addition, there is accumulating evidence that stress may be a factor that critically modulates the balance of ‘cognitive’ and ‘habit’ behavior, putatively by accelerating the shift that would otherwise occur after extensive practice [15^••]. Stressful events are known to influence a broad range of cognitive functions, including attention, memory and decision-making [16, 17, 18]. These stress effects are mediated through the actions of neurotransmitters and hormones, such as noradrenaline and glucocorticoids (mainly cortisol in humans). In particular, noradrenaline, released within seconds after a stressful event from noradrenergic brain stem nuclei and the adrenal medulla, triggers a reorientation of large scale networks toward the processing of salient stimuli, at the expense of executive control processes [19]. Cortisol, acting via brain glucocorticoid (GRs) and mineralocorticoid receptors (MRs), initially enhances and later reverses the effects of noradrenaline [20].

Here, we review recent evidence showing that stress may modulate the preferential engagement of ‘cognitive’ and ‘habitual’ systems in different domains of learning and memory, ranging from navigational and classification to instrumental learning. We will argue that stressful events promote, mediated through the actions of noradrenaline and glucocorticoids, a shift from flexible ‘cognitive’ toward more rigid ‘habit’ behavior. The implications of this shift will be briefly discussed.