Elsevier

NeuroImage

Volume 53, Issue 2, 1 November 2010, Pages 629-637
NeuroImage

EEG oscillations reflect visual short-term memory processes for the change detection in human faces

https://doi.org/10.1016/j.neuroimage.2010.06.057Get rights and content

Abstract

People often fail to notice a large change in the visual scene when the change occurs during a brief interruption of the viewing. Since the change is well above perceptual threshold in continuous viewing, the failure (termed change blindness) has been attributed to abnormal visual short-term memory (VSTM). However, it is still unclear where the abnormality lies among the phases in VSTM, namely, encoding, maintenance, and retrieval-comparison. EEG oscillations, especially the gamma activity, have been suggested as neural signatures of VSTM, but have not been examined in the context of change blindness. Thus, we asked in the present study whether change detection or failure is correlated with EEG oscillatory activities and, if so, whether the timing and the spatial distribution of the oscillations could pin-point the abnormal phase of VSTM in change blindness. While on EEG recording, subjects watched morphed pictures of human faces in trials which consisted of a 200-ms initial image display, a 500-ms blank period, and a 200-ms comparison image display. The two images were either the same or clearly different above threshold. Trials with different images were classified as hit or missed, based on subjects' responses, and EEG data were compared between the two types of trials. Enhanced gamma activity was observed in the right temporal-parietal region during all periods in the hit trials compared to the missed ones. Frontal theta activity was increased during initial image encoding, whereas beta activity was decreased during maintenance and retrieval-comparison in the hit trials. These results point to weak encoding of initial images as the culprit for a later failure in change detection, while abnormal processing in subsequent phases of VSTM may result from the weak encoding and also contribute to change blindness.

Introduction

Change detection, the ability to detect changes in scenery, is one of the most fundamental cognitive skills for survival. In everyday life, our perception of the environment continuously changes, and failure to detect these changes could cause serious problems. However, we sometimes miss large changes in our environment. The term “change blindness” is used to describe the surprising difficulty observers have in noticing large changes to visual scenes because of saccades, blinks, blank screens, movie cuts, and other interruptions (Simons and Levin, 1998, Simons and Rensink, 2005). Change blindness can occur while viewing simple geometric shapes or colors, complex scenes, or familiar objects such as faces (Pourtois et al., 2006). Several factors, such as attention and short-term memory, could influence the change detection task. For instance, researchers studying one model of attention argued that observers can represent the details of only a few attended objects in a scene, suggesting the possibility that the representation of unattended objects is relatively sparse (Treisman, 1993). Although attention might be necessary for conscious change detection, it might not be sufficient (Simons and Rensink, 2005); observers often fail to detect changes even when attention is focused directly on the changing object (Levin and Simons, 1997, Simons and Levin, 1998). Other studies suggested that the key factor underlying change blindness is the limited processing capacity of the brain, resulting in the inability to retain an intact representation of sensory information over time and across brief interruptions in input (Pourtois et al., 2006, Simons and Rensink, 2005).

Theoretically, successful change detection in the change blindness paradigm depends on the initial encoding of a visual representation from the initial view, the retention of that representation in visual short-term memory (VSTM), and the comparison of the representation to relevant perceptual information in the changed image (Hollingworth, 2003). A deficit in any of these processes can induce change blindness, and each possibility has been suggested as the mechanism for it. First, change blindness may occur if insufficient information is encoded from the first image (the encoding-failure hypothesis), as when focused attention is not directed to the changing object (Rensink, 2002). Second, even though sufficient information is encoded, change blindness may still occur if that information is not retained (the retention-failure hypothesis). The initial representation may simply be lost during the blank interval or overwritten by the second image (Shapiro et al., 1997). Finally, although the initial image is encoded and retained successfully, changes may not be detected if comparison with the subsequent image fails (the comparison-failure hypothesis). Subjects can sometimes remember and describe the visual characteristics of a deleted object even when they fail to detect the deletion (Simons et al., 2002). This suggests that implicit VSTM of the representation from an initial image is preserved and change blindness may be caused by comparison failure.

Previous functional magnetic resonance imaging (fMRI) studies have shown increased activity in the frontal and parietal lobes during change detection tasks (Beck et al., 2001, Pessoa and Ungerleider, 2004). In particular, Beck et al. (2006) suggested that the right parietal region might play a causal role in change detection of human faces in their study using repetitive transcranial magnetic stimulation (rTMS). However, since fMRI relies on the detection of relatively slow hemodynamic responses, fMRI techniques cannot evaluate what VSTM processes result in change blindness. On the other hand, several event-related potential (ERP) studies have been carried out to investigate the electrophysiological correlates of conscious change detection with faster temporal resolution (Eimer and Mazza, 2005, Koivisto and Revonsuo, 2003, Pourtois et al., 2006, Schankin and Wascher, 2007). These studies have shown that a negative amplitude shift around 200–300 ms detected at posterior sites after a change in the stimulus (i.e., N2) is associated with visual change awareness (Koivisto and Revonsuo, 2003). However, most of these ERP studies focused on neural correlates of change detection that occur after the second stimulus is presented (Eimer and Mazza, 2005, Koivisto and Revonsuo, 2003, Schankin and Wascher, 2007). Thus, specific processes of VSTM responsible for change blindness were not identified in these studies. Furthermore, ERP may not capture information about the dynamics of cell assembly formation, activation, and subsequent uncoupling, which may play a prominent role in different types of memory operations (Bastiaansen and Hagoort, 2003).

Complex cognitive processes could be implemented by the synchronization of neurons into transient oscillatory assemblies (Pesonen et al., 2007, Varela et al., 2001). Gamma band oscillations are particularly reported to be correlated with encoding, maintenance, matching, and retrieval of VSTM (Herrmann et al., 2004, Jensen et al., 2007, Jokisch and Jensen, 2007, Osipova et al., 2006, Tallon-Baudry et al., 1998). These hypotheses are in accordance with the proposal that neuronal representations can be sustained by the persistent firing of recurrently connected neurons (Hebb, 1949, Jensen et al., 2007). In addition, the gamma band and other frequency bands (e.g., theta and beta) have been reported to be related to memory processing tasks. For example, increased theta oscillation at the frontal region is closely correlated with encoding of VSTM (Jensen et al., 2002, Sederberg et al., 2003). Recent evidence also suggests that decreased beta activity reflects increased short-term memory load (Pesonen et al., 2006, Pesonen et al., 2007).

Although the limited capacity of VSTM is assumed to play an essential role in change blindness, and theta/beta/gamma synchronization is critically related to VSTM, to the best of our knowledge, no EEG oscillation studies that are directly related to change detection or change blindness tasks have been performed. Therefore, in the present study, we investigated whether the change detection phenomenon is reflected in EEG oscillatory activities by using morphed pictures of human faces as stimuli. Note that each of the three hypotheses (the encoding-failure hypothesis, the retention-failure hypothesis, and the comparison-failure hypothesis) makes a different prediction on the timing of neural activity differences when the change detection and the change blindness trials are compared, and in the current study we asked which prediction was actually supported by the data. More specifically, the encoding-failure hypothesis suggests that differences in neural activities between the change detection and change blindness trials will be observed during the memory-encoding period (i.e., when the first image is shown). In contrast, the retention-failure hypothesis predicts that neural activities will not differ during the encoding period but they will during the retention period (i.e., inter-stimulus interval between two images). Finally, according to the comparison-failure hypothesis, differences between two conditions will be detected after the changed image is presented (i.e., when the second image is shown).

Furthermore, it should be noted that the hypotheses may be tested using features of neural oscillations. For instance, the spatial distribution and frequency band of an observed difference in brain oscillation between the change detection and the change blindness trials will indicate which cognitive process differs between the trial types. On the basis of the current view on brain oscillations, one may predict that in successful change detection trials, gamma and theta synchronizations will increase before the second stimulus is presented, as they are closely associated with encoding and maintenance in VSTM (Herrmann et al., 2004, Jensen et al., 2007, Sederberg et al., 2003). One may also predict that beta activities will decrease in the change detection trials compared to the missed ones because they tend to decrease when memory load increases (Pesonen et al., 2006, Pesonen et al., 2007). In addition, enhanced gamma activity after the appearance of the second image is expected for successful change detection, since gamma oscillations have been implicated in the matching and utilization of VSTM (Herrmann et al., 2004).

Section snippets

Participants

Ten healthy subjects with no history of neurological disorders participated in the experiment (mean age, 26.5 ± 4.4 years; 1 female; 1 left-handed) after providing written informed consent in accordance with the Declaration of Helsinki. All participants had normal or corrected-to-normal vision. Two subjects were excluded from data analysis because of their low accuracy rates (< 25%).

Stimuli

Six non-morphed and 147 morphed pictures of human faces were used as visual stimuli. These stimuli were frontal view

Behavioral data

During the pre-experiment, we measured the threshold-morph level (TML) in each subject. The mean value of TML for the 3 sets was 28 ± 2%, 26 ± 6%, and 24 ± 4%, respectively. The middle column of Fig. 1 shows one typical picture morphed by the degree of TML in each set.

During the recording sessions, the stimulus was changed according to the degree of TML, and subjects produced a balanced rate of change detection (hit) and change blindness (miss) trials (mean 51% and 49%, respectively). In addition,

Discussion

In the present study, we investigated the EEG oscillatory activity produced while subjects detected changes in morphed human face pictures. Successful change detections were accompanied by increased gamma power in the right temporo-parietal region. Moreover, enhancement of gamma activity was already observed when the first stimulus was presented, and this activity was sustained until the subjects responded. We also observed increased theta activity in the frontal region when the first stimulus

Acknowledgments

The work was supported by a grant from the National Research Foundation of Korea (No. 800-2009-0003). The authors would like to thank the Swartz Center for Computational Neuroscience (SCCN) and the Donders Institute for Brain, Cognition and Behavior (DCCN) for their generous gift of analysis toolboxes, Professor Ione Fine for kindly providing the pictures used as stimuli, and Dr. Hyon Lee for editorial help.

References (56)

  • B.K. Min et al.

    Prestimulus EEG alpha activity reflects prestimulus top-down processing

    Neurosci. Lett.

    (2007)
  • I. Nieuwenhuis et al.

    Visual areas become less engaged in associative recall following memory stabilization

    Neuroimage

    (2008)
  • S. Palva et al.

    New vistas for alpha-frequency band oscillations

    Trends Neurosci.

    (2007)
  • M. Pesonen et al.

    Brain oscillatory 1–30 Hz EEG ERG/ERS responses during the different stages of an auditory memory search task

    Neurosci. Lett.

    (2006)
  • M. Pesonen et al.

    Brain oscillatory 4–30 Hz responses during a visual n-back memory task with varying memory load

    Brain Res.

    (2007)
  • K.L. Shapiro et al.

    The attentional blink: a view on attention and a glimpse on consciousness

    Trends Cogn. Sci.

    (1997)
  • D.J. Simons et al.

    Change blindness: past, present, and future

    Trends Cogn. Sci.

    (2005)
  • D.J. Simons et al.

    Evidence for preserved representations in change blindness

    Conscious. Cogn.

    (2002)
  • C. Tallon-Baudry et al.

    Oscillatory gamma activity in humans and its role in object representation

    Trends Cogn. Sci.

    (1999)
  • E. Zion-Golumbic et al.

    Human face preference in gamma-frequency EEG activity

    Neuroimage

    (2008)
  • D.M. Beck et al.

    Neural correlates of change detection and change blindness

    Nat. Neurosci.

    (2001)
  • D.M. Beck et al.

    Right parietal cortex plays a critical role in change blindness

    Cereb. Cortex

    (2006)
  • D.H. Brainard

    The Psychophysics Toolbox

    Spat. Vis.

    (1997)
  • T.N. Cornsweet

    The staircrase-method in psychophysics

    Am. J. Psychol.

    (1962)
  • M. Eimer et al.

    Electrophysiological correlates of change detection

    Psychophysiology

    (2005)
  • P. Fries et al.

    Modulation of oscillatory neuronal synchronization by selective visual attention

    Science

    (2001)
  • D.M. Green et al.

    Signal Detection Theory and Psychophysics

    (1966)
  • D.O. Hebb

    The Organization of Behavior: A Neuropsychological Theory

    (1949)
  • Cited by (19)

    • Electrical devices for visual restoration

      2022, Survey of Ophthalmology
      Citation Excerpt :

      Change detection is dependent on visual short-term memory and requires intact visual and attention circuitry. Low frequency gamma band oscillations are enhanced in the right temporal and parietal regions during visual change detection tasks in the normally sighted.40 In addition to this, increased theta activity aids in the maintenance and retrieval of visual memories, while decreased beta activity in the occipital lobe may reflect intake of a high memory load.40

    • Short-range and long-range neuronal oscillatory coupling in multiple frequency bands during face perception

      2020, International Journal of Psychophysiology
      Citation Excerpt :

      Although these studies demonstrated meaningful findings, they did not demonstrate the neuronal oscillatory activity over or between these face-related brain regions in detail. Furthermore, the neuronal oscillatory activity in the human brain has been considered as potential neural mechanism of face processing (Brenner et al., 2014; Clayton et al., 2018; Furl et al., 2017; Matsuzaki et al., 2015; Okazaki et al., 2014; Park et al., 2010; Schneider et al., 2019). Neuronal activity in different frequency bands had been found over some facial areas.

    • The neural mechanisms of percept-memory comparison in visual working memory

      2012, Biological Psychology
      Citation Excerpt :

      This result is not surprising. Because ample research has revealed that the frontal theta has a close relationship with working memory in general (e.g., Jensen and Tesche, 2002; Park et al., 2010; Raghavachari et al., 2001, 2006; Rainer and Lee, 2004; Sarnthein et al., 1998; Tesche and Karhu, 2000), and has a functional link with the executive control of working memory and encoding new information into memory in particular (Klimesch, 1999; Park et al., 2010; Sauseng and Klimesch, 2008; Sauseng et al., 2004, 2005, 2008; Wang, 2010). Therefore, while the current study provides the first evidence showing that the frontal theta plays an important role at the VWM comparison stage, it offers converging evidence supporting that the frontal theta has a tight relationship with working memory.

    View all citing articles on Scopus
    View full text