The timing of individual face recognition in the brain
Introduction
Numerous behavioral and neuropsychological studies have provided evidence that adults’ perception of faces is different from their processing of other visual stimuli. Adults rapidly detect that a stimulus is a face even when realistic features are not physically present, so long as the face-like configuration of two eyes above a nose above a mouth can be inferred. For example, they can see a face in paintings by Archimbaldo consisting of only fruits and vegetables and in two-tone Mooney stimuli in which black and white shadows lead to the perception of a face. Adults process faces holistically (Tanaka and Farah, 1993, Young et al., 1987) and are sensitive to small differences among faces in the shape of individual features and the spacing among them (Freire et al., 2000, Mondloch et al., 2002; reviewed in Maurer, Le Grand, & Mondloch, 2002). Collectively, these behavioral skills allow adults to extract a wealth of information each time they encounter a human face (e.g., age, race, gender, emotional expression). Most notably, adults are able to recognize the identity of hundreds of faces at a glance and they can do so under poor lighting conditions, from numerous viewpoints, and after a face has aged by several years, at least for the kinds of faces they encounter on a daily basis (e.g., upright same-race faces).
Although behavioral researchers have investigated each aspect of face perception quite thoroughly, neural research on face processing with both humans and non-human primates has primarily investigated the neural markers that distinguish faces from non-face objects (Allison et al., 1994, Bentin et al., 1996, Bötzel et al., 1995, Desimone et al., 1984, Kanwisher et al., 1997, McCarthy et al., 1997, Perrett et al., 1982, Puce et al., 1995, Rolls and Baylis, 1986, Tsao et al., 2006). As a result, several brain regions (e.g., middle fusiform gyrus, inferior occipital gyrus, and superior temporal sulcus) and electrophysiological signals (e.g., the intracortical N200 and the scalp-recorded N170) have been found to respond more strongly to faces than to non-face objects. In contrast to the face versus non-face distinction, however, much less is known about the neural mechanisms underlying the perception and recognition of individual faces within the face category.
To account for adults’ expertise in face recognition, Valentine (1991) proposed a norm-based coding mechanism, a process by which individual faces are compared to a norm (prototype) that represents the average of all faces previously encountered. In his model, each face is represented as a point in a multi-dimensional face space; the origin of the face space corresponds to the prototypical face and the location of each face represents how and how much that face deviates from the average. Faces near the norm are rated as more typical/attractive than faces that are far from the norm and they are categorized more quickly in a face/nonface task. In contrast, faces far from the norm are recognized more quickly than typical faces, perhaps because they reside in a less populated area of face space (Valentine & Bruce, 1986; but see Burton & Vokey, 1998, for evidence suggesting that distance from the norm and local density may be sufficient to produce differences in the recognition of typical versus distinctive faces).
Recently, two studies have tested Valentine's norm-based model at a neural level. Their results indicate that neural activity increases as a function of face identity strength, i.e., as a function of how much individual faces differ from an average face (Leopold et al., 2006, Loffler et al., 2005). In each study, identity strength was manipulated by varying the relative weighting of an individual face versus the average face. Using synthetic faces of different identities, Loffler et al. (2005) reported greater BOLD responses from fusiform face area (FFA) as identity strength increased. Loffler et al. concluded that the BOLD response elicited by a face reflects the distance of the face from the average face because (a) the BOLD response did not increase as a function of the distance from a non-average face and (b) adaptation to a single facial identity reduced the BOLD response to other faces along the same identity trajectory, but not to faces along different trajectories. Similarly, in a single-cell recording study with monkeys, Leopold et al. (2006) found that neural responses from anterior inferotemporal cortex became stronger as face identity strength increased. Although these studies indicate that the magnitude of neural activity may code for identity strength, which is consistent with Valentine's norm-based coding model, the temporal parameters of individual face perception remain unclear. The goal of the current study was to examine the timing of brain responses to face identity strength using scalp-recorded ERPs.
There is some evidence that the face-sensitive N170 component, traditionally interpreted as a neural marker for structural encoding of faces (Eimer, 2000b), may also be sensitive to visual face identities. When the same identity is presented on consecutive trials, the amplitude of the N170 is reduced relative to when two different identities are presented (Caharel et al., 2009b, Campanella et al., 2000, Jacques and Rossion, 2006, Jacques et al., 2007; but see Schweinberger et al., 2004, Schweinberger et al., 2002), even when the viewpoints are different across presentations (Caharel, d’Arripe, Ramon, Jacques, & Rossion, 2009). This adaptation effect on N170 may occur as early as 160 ms post face onset (Caharel et al., 2009a, Caharel et al., 2009b, Jacques et al., 2007). In addition, when a face discrimination task was made more difficult by rotating the faces away from their canonical upright orientation, N170 amplitude increased along with error rates and reaction times (Jacques & Rossion, 2007).
However, these immediate repetition effects on the N170 may not indicate that the N170 reflects individual face recognition; rather, the N170 may reflect the processing of individual facial characteristics (Eimer, 2000b, Zheng et al., 2011), and adaptation to the face category (Eimer, Kiss, & Nicholas, 2010). First, although N170 adaptation for upright faces is larger when the adaptor is a face than when the adaptor is a house, N170 adaptation occurs when the test face is preceded by an adaptor stimulus of a different facial identity and the magnitude of this effect is independent of whether the adaptor is an upright face, an inverted face, a face without eyes, or eyes only (Eimer et al., 2010; see Harris and Nakayama, 2007, Harris and Nakayama, 2008, for similar results using MEG technology). Second, N170 is not influenced by face identity when faces are presented in a random order; under these conditions only later ERP components are influenced by identity (Bentin and Deouell, 2000, Eimer, 2000a, Gosling and Eimer, 2011, Kaufmann et al., 2008, Rossion et al., 1999, Tanaka et al., 2006). Given that the priming effects on later ERP components are robust even when the prime (e.g., a picture of Nancy Reagan) shares no facial characteristics with the target (e.g., Ronald Reagan) (Schweinberger, Pfütze, & Sommer, 1995), it is thus possible that modulation of the N170 reflects brain processes related to the encoding of facial characteristics (Eimer, 2000b, Zheng et al., 2011) and that modulation of later components (e.g., N250) reflects brain processes related to the visual recognition of a face.
To further explore the temporal parameters of visual face recognition we adopted a method previously used to study the influence of face identity on the BOLD signal (Loffler et al., 2005) and single-cell activity (Leopold et al., 2006). Specifically, we investigated the influence of variation in identity strength relative to an average face on the magnitude of early ERP components (P1, N170, P2, N250). We manipulated face identity strength by first constructing an “average” face based on 32 individual female faces; each individual face was then morphed with this “average” face to produce continua of face identity (Fig. 1a and b). The relative weighting of an original face in these morphed faces ranged from 100% to 0% in 10% decrements. Participants performed a face identification task, in which they were instructed to press a button whenever they felt that they had detected a target face or a face that looked like a target face (see Fig. 1c). We predicted that the amplitude of one or more ERP components would increase with identity strength. Our primary question was whether this effect would be observed as early as in the N170 or only in later components.
Section snippets
Participants
Seventeen Caucasian female undergraduate students (mean age = 20.4 ± 1.5 years) at Brock University participated in the current ERP study for either a research credit or a $15 honorarium. All participants were right-handed native English speakers with normal or corrected-to-normal vision. No participants reported any neurological disorders, psychiatric history, or attentional problems. The experimental procedures were approved by Brock University Research Ethics Board, and written informed consent
Behavioral results
Averaged across all participants, accuracy in detecting target faces followed a cubic function with face identity strength (Fig. 3) (for target face 1, p = .010; for target face 2, p = .014; for target face 3, p = .001; for target face 4, p = .001; overall, for all target faces, p < .001).1
Discussion
Recent studies with humans and non-human primates (Leopold et al., 2006, Loffler et al., 2005) have found that neural responses from face-sensitive regions are sensitive to the strength of face identities relative to an average face. In the present study we investigated the timing of these neural events (i.e., we investigated when the effect of face identity strength occurs in the brain). Face identity strength did not affect the amplitude of the P100 (80–130 ms) or the N170 (140–190 ms) or the
Acknowledgements
We thank M.D. Vida (Department of Psychology, Neuroscience & Behaviour, McMaster University, Hamilton, ON) for data collection, and M. Nishimura (Department of Psychology, Carnegie Mellon University, Pittsburgh, PA) for providing the original face stimuli. We also thank three anonymous reviewers for their comments and suggestions in helping us clarify some important issues. The work was supported by Natural Sciences and Engineering Research Council of Canada (C.J.M. and S.J.S.) and Canada
References (56)
- et al.
Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: Evidence from the N170 ERP component
Neuropsychologia
(2009) - et al.
Recognizing an individual face: 3D shape contributes earlier than 2D surface reflectance information
NeuroImage
(2009) Event-related brain potentials distinguish processing stages involved in face perception and recognition
Clinical Neurophysiology
(2000)- et al.
An event-related brain potential study of explicit face recognition
Neuropsychologia
(2011) - et al.
Recognition of unfamiliar faces
Trends in Cognitive Sciences
(2000) - et al.
Early electrophysiological responses to multiple face orientations correlate with individual discrimination performance in humans
NeuroImage
(2007) - et al.
Distortions in the brain? ERP effects of caricaturing familiar and unfamiliar faces
Brain Research
(2008) - et al.
The many faces of configural processing
Trends in Cognitive Sciences
(2002) - et al.
IFCN standards for digital recording of clinical EEG
Electroencephalography and Clinical Neurophysiology
(1998) - et al.
Spherical splines for scalp potential and current density mapping
Electroencephalography and Clinical Neurophysiology
(1989)