The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization

Highlights

• The feedback-related negativity (FRN) is associated with surprise signals.

• The FRN is rather associated with absolute than signed reward prediction errors.

• EEG-informed fMRI consistently locates the FRN in the dorsal anterior cingulum.

• Surprise signals are directly projected to the dorsal anterior cingulum.

Abstract

Changes in response contingencies require adjusting ones assumptions about outcomes of behaviors. Such adaptation processes are driven by reward prediction error (RPE) signals which reflect the inadequacy of expectations. Signals resembling RPEs are known to be encoded by mesencephalic dopamine neurons projecting to the striatum and frontal regions. Although regions that process RPEs, such as the dorsal anterior cingulate cortex (dACC), have been identified, only indirect evidence links timing and network organization of RPE processing in humans. In electroencephalography (EEG), which is well known for its high temporal resolution, the feedback-related negativity (FRN) has been suggested to reflect RPE processing. Recent studies, however, suggested that the FRN might reflect surprise, which would correspond to the absolute, rather than the signed RPE signals. Furthermore, the localization of the FRN remains a matter of debate.

In this simultaneous EEG–functional magnetic resonance imaging (fMRI) study, we localized the FRN directly using the superior spatial resolution of fMRI without relying on any spatial constraint or other assumption. Using two different single-trial approaches, we consistently found a cluster within the dACC. One analysis revealed additional activations of the salience network. Furthermore, we evaluated the effect of signed RPEs and surprise signals on the FRN amplitude. We considered that both signals are usually correlated and found that only surprise signals modulate the FRN amplitude. Last, we explored the pathway of RPE signals using dynamic causal modeling (DCM). We found that the surprise signals are directly projected to the source region of the FRN. This finding contradicts earlier theories about the network organization of the FRN, but is in line with a recent theory stating that dopamine neurons also encode surprise-like saliency signals.

Our findings crucially advance the understanding of the FRN. We found compelling evidence that the FRN originates from the dACC. Furthermore, we clarified the functional role of the FRN, and determined the role of the dACC within the RPE network. These findings should enable us to study the processing of surprise and adjustment signals in the dACC in healthy and also in psychiatric patients.

Introduction

The ability to adapt to changes in the environment is essential for survival. A growing corpus of neuroscientific literature strongly suggests that adaptation involves similar learning processes in decision making, perception and movement control and is driven by (reward) prediction error (RPE) signals (Friston, 2010). RPEs signal the difference between expected and received outcomes of a behavior and are assumed to update the expectations about stimulus–outcome relations (Rescorla and Wagner, 1972). Classical reinforcement learning theories (Sutton and Barto, 1998) assume that learning is driven by signed RPEs (sRPEs). This means that the value of an object increases after a positive RPE (outcome for this object is better than expected), whereas the value decreases after a negative RPE (outcome is worse than expected). Other theories (Courville et al., 2006, Hayden et al., 2011, Pearce and Hall, 1980) focus on the aspect that learning is mainly driven by the surprisingness of an event, regardless of its valence, which can be characterized by absolute RPEs (|RPEs|). |RPEs| are a measure for the unsigned deviance of an expected outcome, which is often called surprise (cf Hayden et al., 2011). Dopamine (DA) neurons of the mesencephalon have been found to display sRPE-like signals (Schultz et al., 1997). More recent findings show that these dopamine neurons also encode |RPE|-like salience signals (Bromberg-Martin et al., 2010a, Bromberg-Martin et al., 2010b, Matsumoto and Hikosaka, 2009). These neurons are known to project to a variety of sub- and neocortical areas (Bromberg-Martin et al., 2010b, Oades and Halliday, 1987). A wealth of studies in human and non-human primates showed that these projected RPEs are also processed in the striatum, the orbitofrontal/ventromedial prefrontal cortex (OFC/vmPFC), the dorsal anterior cingulate cortex (dACC) and the dorsolateral prefrontal cortex (dlPFC) and the amygdalae (Gläscher et al., 2010, Glimcher, 2011, Haber and Knutson, 2010, Hare et al., 2008, Rutledge et al., 2010). In humans, however, we have only very limited knowledge about the temporal evolution of RPE signals in the brain, because of the poor temporal resolution of conventional neuroimaging methods, such as functional magnetic resonance imaging (fMRI; Meyer-Lindenberg, 2010).

Electroencephalography (EEG) is a method in human neuroimaging, which has an excellent temporal, but lacks in spatial resolution (Meyer-Lindenberg, 2010). In a seminal work, Holroyd and Coles (2002) suggested an event-related potential (ERP) of the EEG, the feedback-related negativity (FRN), to reflect sRPE processing. The FRN is commonly computed as the difference wave between rewards and punishments at mid-central sites, such as the vertex electrode Cz and peaks between 200 and 300 ms after feedback onset (Miltner et al., 1997, Nieuwenhuis et al., 2004). Holroyd and Coles (2002) assumed that the neuronal populations which elicit the FRN are usually inhibited by the tonic firing of the dopamine neurons. A decrease in tonic DA, as reflected by negative sRPEs, would therefore disinhibit these neuronal populations causing the widespread neuronal activity which is detectable on the scalp as FRN. Over the last decade, various studies examined the relation between the FRN and sRPEs (for review cf Walsh and Anderson, 2012). Most approaches which evaluated the association between the FRN amplitude and sRPEs (e.g., Bellebaum and Daum, 2008, Holroyd et al., 2004, Pfabigan et al., 2011) relied on static paradigms and averaged ERPs. The averaging of multiple trials using ERPs has the advantage of increasing the physiologically poor signal-to-noise-ratio (SNR) of single-trial EEG data. Unfortunately, such approaches sometimes underestimate the dynamic, context-specific nature of RPE signals in the framework of reinforcement learning. Additional evidence which directly links RPEs with the FRN amplitudes by also considering their covariation over time, such as in single-trial analyses, would therefore be important to complement the current knowledge.

Furthermore, also the localization of the FRN remains a matter of debate. Although many source estimation studies localized the FRN in the large area of the ACC (for review cf Walsh and Anderson, 2012), several studies also located the origin of the FRN in the posterior cingulate cortex (Badgaiyan and Posner, 1998, Cohen and Ranganath, 2007, Hewig et al., 2007, Müller et al., 2005, Nieuwenhuis et al., 2005b). Others (Carlson et al., 2011, Foti et al., 2011a, Foti et al., 2011b, Martin et al., 2009, Nieuwenhuis et al., 2005b) found the source within the basal ganglia, which is also known to crucially process RPEs. All source localization studies so far face the unsolvable inverse problem, which states that the same topographical maps measured at the scalp surface can be caused by several different sources and/or source configurations (Helmholtz, 1853). Therefore, current dipole localization studies heavily rely on prior assumptions of the experimenters, such as the number of dipoles and their (alternative) locations (Luck, 2005). Other methods, such as standardized low resolution brain electromagnetic tomography (sLORETA, Pascual-Marqui, 2002) overcome some of these problems, but show a very limited spatial resolution (see e.g., Cohen and Ranganath, 2007). Recently, multimodal imaging techniques have been introduced, such as simultaneous EEG–fMRI (Rosa et al., 2010). These methods can overcome the above mentioned limitations of EEG source localization by associating the variability of the EEG signal with the fluctuations of the fMRI (Debener et al., 2006, Huster et al., 2012, Lüchinger et al., 2011, Lüchinger et al., 2012). To our knowledge, the localization of the FRN has not yet been investigated using concurrent EEG–fMRI.

An increasing number of studies contradict the assumption by Holroyd and Coles (2002) that the FRN amplitude reflects sRPEs (Alexander and Brown, 2011, Chase et al., 2010, Oliveira et al., 2007, Talmi et al., 2012). Rather, these studies suggest that the FRN reflects surprise or unexpectedness, which is known as the unsigned, absolute RPE signal (Hayden et al., 2011, Pearce and Hall, 1980). In a seminal study by Oliveira et al. (2007), the authors showed that the FRN is elicited by surprising, rather than by negative feedback. In many studies which evaluate the FRN, the negative feedback appears with a frequency well below 50% and therefore, a clear differentiation between valence (positive vs. negative feedback) and surprise is not possible. In a recent study by Chase et al. (2010), the authors tried to link RPEs to single-trial amplitudes and found indications that larger negative sRPEs reflect larger FRN amplitudes. For positive sRPEs, the authors found a marginally significant negative relationship with FRN amplitudes (greater positive sRPEs predicted more negative amplitudes; Chase et al., 2010). This finding indicates that the FRN amplitude might be modulated by the absolute value of the RPEs rather than by sRPEs. Further evidence comes also from a functional model of the ACC by Alexander and Brown (2011), which successfully simulated the FRN deflection based on surprise-signals.

In their model, Holroyd and Coles (2002) suggested that the FRN-source lies within a widespread network of cortical and subcortical areas, such as the striatum, the amygdalae, the dlPFC and the OFC/vmPFC. They assumed that the FRN-source receives different motor programs from these areas and selects the best among them using the sRPE signal. Whether such a network is functionally plausible in the framework of FRN processing has not yet been evaluated. Furthermore, it remains unclear whether the FRN-source directly receives dopaminergic RPE signals or whether they are transmitted via other areas such as the striatum or the vmPFC.

In order to localize the origin of the FRN, in this study, we simultaneously recorded EEG and fMRI from 15 healthy adults which performed a probabilistic reversal learning task. For the localization of the FRN, we used two complementary approaches. In the first analysis, we aimed for the maximal consistency across subjects and therefore defined the FRN latency based on the grand average. In the second analysis, we aimed for maximal individual temporal sensitivity and defined the FRN latency based on the individual difference waves. In both analyses, we found a cluster in the dorsal anterior cingulate cortex. In the second analysis, we additionally found activations in the salience network. Furthermore, we evaluated the relation between the FRN amplitude and model-derived RPEs using a single-trial approach. We found that the single-trial variability was better explained by surprise (encoded as |RPEs|) than by sRPEs. Additionally, we investigated the effective functional connectivity of the RPE-network surrounding the FRN source. Using dynamic causal modeling (DCM; Friston et al., 2003), we found that it is most likely that the FRN source directly receives |RPE|-inputs rather than signals which are first processed in another area.