Regular articleEffect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus
Introduction
Humans have the remarkable ability to extract information regarding mental state, direction of attention, and intentions from facial expression and direction of gaze. These abilities are critical components of normal human social interaction. The acquisition of information from faces presumably requires several hierarchical processing stages, including recognizing the constellation of facial features, identifying the face, determining facial expression and direction of gaze, and then placing this information into a social context Bruce and Young 1986, Halgren et al 1994, Sergent 1995.
Research on automatic processing of low-level facial features has provided consistent evidence of the role of the fusiform gyrus (FG) in the representation of fixed facial features. This research was prompted by the discovery that prosopagnosic patients who display specific deficits in identifying faces almost always have lesions in the ventral temporal lobe De Renzi et al 1991, Meadows 1974, Sergent and Signoret 1992. Neuroimaging, surface and intracranial EEG, and MEG studies of face perception have provided supportive evidence of the role of the FG in face processing Clark et al 1996, Eimer and McCarthy 1999, George et al 1996, Halgren et al 1994, Halgren et al 2000, Kanwisher et al 1997, McCarthy et al 1997, McCarthy et al 1999, Puce et al 1995, Sergent et al 1992, Sergent and Signoret 1992.
More recently, the posterior superior temporal sulcus (STS) also has been implicated in processing facial features, particularly the more changeable features like gaze direction. In the monkey, neurons have been found in the STS that respond selectively to face and gaze direction (Perrett et al., 1985). Neuroimaging studies have shown similar involvement of the STS in face processing in humans. Several studies have shown STS activation when viewing static faces George et al 2001, Halgren et al 1999, Hoffman and Haxby 2000, Kanwisher et al 1997. STS activation also has been shown during perception of eye and mouth movement Puce et al 1998, Wicker et al 1998.
The distinct roles of the FG and STS in face processing are not known. Hoffman and Haxby (2000) have attempted to differentiate the functions of these regions by looking at variations in task instructions. They found that subjects had greater activation in the FG when instructed to pay attention to the identity of the person and greater activation in the STS when instructed to attend to the direction of gaze (Hoffman and Haxby, 2000). The present study attempts to further elucidate the roles of the FG and STS by investigating activation in these regions in response to viewing multiple face and gaze orientations.
The effect of head and gaze orientation on brain activation is unclear from current literature. Monkey studies have indicated that there is a differential response in STS cells to different face and gaze orientations Perrett et al 1982, Perrett et al 1985. These monkey studies revealed that cells with preference for a specific face direction also prefer the corresponding gaze direction (i.e., cells that respond to face forward also show greatest response to direct gaze) (Perrett et al., 1985). Furthermore, Perrett et al. (1985) found that sensitivity to gaze direction could override sensitivity to head orientation, which provided supported for a model in which gaze direction could affect perception of head orientation, but not vice versa.
Only two studies with limited subject numbers (N · 7) have attempted to differentiate brain activation in humans for specific views of the face George et al 2001, Tong et al 2000. George et al., (2001) used a block-design functional MRI (fMRI) study to investigate brain activation when subjects viewed varying face and gaze orientations (a 2 × 2 factorial design of head and gaze orientation). In the study, subjects were asked to determine the gender of the faces they viewed. Although both FG and STS activation was observed during the task, no difference in activation in either of these regions was revealed for the comparison of head orientation, regardless of gaze orientation. The only effect of head orientation was found in the posterior striate and extrastriate regions (V1/V2) and the left motor cortex, which showed greater activation when viewing frontal versus angled head orientations, regardless of gaze angle. Tong et al. (2000) used fMRI to look specifically at the fusiform face area and found no differences in activation for frontal versus profile view of the head, when subjects passively viewed the faces or performed a one-back matching task. However, the study revealed decreasing FG activation as the head orientation was rotated further away from view, with the least activation for a view of the back of the head.
Additional functional imaging studies have tried to differentiate the response to direct gaze versus averted gaze, but there is no evidence of a consistent distinct response to either of these gaze orientations. One study found greater activation in the right amygdala for direct gaze versus averted gaze in a gaze discrimination task (Kawashima et al., 1999). In a second study, subjects who passively viewed faces showed greater activation in the left STS and bilaterally in the intraparietal sulcus (IPS) to averted gaze in comparison to direct gaze (Hoffman and Haxby, 2000). A recent study showed greater activation in the FG, but not the STS, for forward gaze compared to averted gaze during a gender determination task (George et al., 2001).
To further investigate these inconsistencies of brain activation in response to variations in head and gaze orientation, especially with respect to activation in the FG and STS, we used event-related fMRI with a 2 × 2 factorial design of head and gaze orientation. Since there is clear evidence that the FG is involved in featural processing of the face, and since angled head orientation provide altered views of facial features, we hypothesized that FG activation would be modulated by variations in head orientation. Since the STS has recently been implicated in processing changeable aspects of the face such as gaze direction and since monkey studies revealed cells responsive to face and gaze direction, we hypothesized that STS activation would be modulated by variations in both head and gaze direction.
Section snippets
Subjects
A total of 18 healthy, right-handed subjects (9 female, 9 male), ages 18–24 (mean 20.9) years, participated in the study after giving written informed consent. The human subjects committee at Stanford University School of Medicine approved all protocols used in this study.
Stimuli
The stimuli consisted of 60 unique static pictures of faces, as well as 45 isoluminant scrambled images. Pictures of faces of college-aged models were acquired with a digital camera against a common background at a distance of
Task performance
Accuracy for all four experimental conditions was high (99 ± 4%) and repeated-measures ANOVA revealed no significant difference in the accuracy for any of the four conditions. Repeated-measures ANOVA demonstrated a significant effect on RT of both head orientation [F (1,17) = 54.27, P < 0.0001] and gaze direction [F (1,17) = 10.04, P < 0.006] (mean AA reaction time, 872 ms; mean AF reaction time, 877 ms; mean FA reaction time, 841 ms; mean FF reaction time, 766 ms). There also was a significant
Discussion
To our knowledge, this is the first study to use an event-related design to investigate FG and posterior STS activation during gaze processing. This study provides evidence that both head and gaze orientation significantly affect gaze processing. The 2 × 2 factorial design, with head orientation (face forward vs. face angled) as one factor and gaze orientation (gaze forward vs. gaze angled) as a second factor, allowed investigation of effects of both head and gaze orientation on gaze
Acknowledgements
Supported by NIH grants MH50047, MH01142, HD31715, and HD40761.
References (61)
- et al.
Social perception from visual cuesrole of the STS region
Trends Cogn. Sci.
(2000) - et al.
Functional magnetic resonance imaging of human visual cortex during face matchinga comparison with positron emission tomography
Neuroimage
(1996) Brainstem, cerebellar and limbic neuroanatomical abnormalities in autism [published erratum appears in Curr. Opin. Neurobiol. 1997 Aug; 7(4):568]
Curr. Opin. Neurobiol.
(1997)- et al.
Apperceptive and associative forms of prosopagnosia
Cortex
(1991) - et al.
Stochastic designs in event-related fMRI
Neuroreport
(1999) - et al.
Brain events related to normal and moderately scrambled faces
Brain Res. Cogn. Brain Res.
(1996) - et al.
Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing
Neuroimage
(2001) - et al.
Spatio-temporal stages in face and word processing. I. Depth-recorded potentials in the human occipital, temporal and parietal lobes [corrected] [published erratum appears in J. Physiol. (Paris) 1994; 88(2):following 151]
J. Physiol. (Paris)
(1994) - et al.
Generalisability, random effects & population inference
Neuroimage
(1998) - et al.
The role of the fusiform gyrus in successful encoding of face stimuli
Neuroimage
(1999)
Neurons in the amygdala of the monkey with responses selective for faces
Behav. Brain Res.
Heterogeneity of cingulate contributions to spatial attention
Neuroimage
Limbic and sensory connections of the inferior parietal lobule (area PG) in the rhesus monkeya study with a new method for horseradish peroxidase histochemistry
Brain Res.
Combining spatial extent and peak intensity to test for activations in functional imaging
Neuroimage
ERP evidence of developmental changes in processing of faces
Clin. Neurophysiol
Brain regions involved in the perception of gazea PET study
Neuroimage
MRI volumes of amygdala and hippocampus in non-mentally retarded autistic adolescents and adults
Neurology
Specification of the neurobehavioral phenotype in males with fragile X syndrome
Pediatrics
Understanding face recognition
Br. J. Psychol.
Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI
Neuroreport
fMRI study of face perception and memory using random stimulus sequences
J. Neurophysiol.
PsyScopea new graphic interactive environment for designing psychology experiments
Behav. Res. Methods Instr. Comp.
Fortoparietal cortical networks for directing attention and the eye to visual locationsIdentical, independent, or overlapping neural systems
Proc. Natl. Acad. Sci.
Voluntary orienting is dissociated from target detection in human posterior parietal cortex
Nature Neurosci.
Optimal experimental design for event-related fMRI
Hum. Brain Mapp.
The Human BrainSurface, Three-Dimensional Sectional Anatomy with MRI, and Blood Supply. 2nd completely rev. and enl. ed.
Prosopagnosia and structural encoding of faces; evidence from event-related potentials
Neuroreport
A cortical representation of the local visual environment
Nature
Statistical parametric maps in functional imaginga general linear approach
Hum. Brain Mapp.
Self-navigated spiral fMRIinterleaved versus single-shot
Magn. Res. Med.
Cited by (52)
Perception of direct vs. averted gaze in portrait paintings: An fMRI and eye-tracking study
2018, Brain and CognitionCitation Excerpt :This is fully in accord with influential models of face perception, which propose that FFA is involved in expression of invariant processing of faces (Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000), as well as with more recent integrative models of face perception, which argue that the FFA contributes to the perception of changeable aspects of faces, through a broad sensitivity to shape information (Bernstein, Erez, & Yovel, 2018; Duchaine & Yovel, 2015). Enhanced activity in the fusiform gyrus during direct gaze perception corresponds to several studies using photographic stimuli (George, Driver, & Dolan, 2001; Pageler et al., 2003). It has been suggested that a higher activation of the fusiform gyrus in these studies, which - much like our own study - involved presentation of many different faces, may reflect the need for a “deeper” processing and encoding of novel faces indicating that the interaction between a beholder and the direct gaze of the subjects in the painted portraits could be associated with the increasing feeling of novelty (Boyarskaya, Sebastian, Bauermann, Hecht, & Tuscher, 2015).
Processing of gaze direction within the N170/M170 time window: A combined EEG/MEG study
2017, Neuropsychologia