Elsevier

NeuroImage

Volume 20, Issue 1, September 2003, Pages 318-329
NeuroImage

Regular article
Effect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus

https://doi.org/10.1016/S1053-8119(03)00229-5Get rights and content

Abstract

We used functional MRI with an event-related design to dissociate the brain activation in the fusiform gyrus (FG) and posterior superior temporal sulcus (STS) for multiple face and gaze orientations. The event-related design allowed for concurrent behavioral analysis, which revealed a significant effect of both head and gaze orientation on the speed of gaze processing, with the face and gaze forward condition showing the fastest reaction times. In conjunction with this behavioral finding, the FG responded with the greatest activation to face and gaze forward, perhaps reflecting the unambiguous social salience of congruent face and gaze directed toward the viewer. Random effects analysis showed greater activation in both the FG and posterior STS when the subjects viewed a direct face compared to an angled face, regardless of gaze direction. Additionally, the FG showed greater activation for forward gaze compared to angled gaze, but only when the face was forward. Together, these findings suggest that head orientation has a significant effect on gaze processing and these effects are manifest not only in the STS, but also the FG.

Introduction

Humans have the remarkable ability to extract information regarding mental state, direction of attention, and intentions from facial expression and direction of gaze. These abilities are critical components of normal human social interaction. The acquisition of information from faces presumably requires several hierarchical processing stages, including recognizing the constellation of facial features, identifying the face, determining facial expression and direction of gaze, and then placing this information into a social context Bruce and Young 1986, Halgren et al 1994, Sergent 1995.

Research on automatic processing of low-level facial features has provided consistent evidence of the role of the fusiform gyrus (FG) in the representation of fixed facial features. This research was prompted by the discovery that prosopagnosic patients who display specific deficits in identifying faces almost always have lesions in the ventral temporal lobe De Renzi et al 1991, Meadows 1974, Sergent and Signoret 1992. Neuroimaging, surface and intracranial EEG, and MEG studies of face perception have provided supportive evidence of the role of the FG in face processing Clark et al 1996, Eimer and McCarthy 1999, George et al 1996, Halgren et al 1994, Halgren et al 2000, Kanwisher et al 1997, McCarthy et al 1997, McCarthy et al 1999, Puce et al 1995, Sergent et al 1992, Sergent and Signoret 1992.

More recently, the posterior superior temporal sulcus (STS) also has been implicated in processing facial features, particularly the more changeable features like gaze direction. In the monkey, neurons have been found in the STS that respond selectively to face and gaze direction (Perrett et al., 1985). Neuroimaging studies have shown similar involvement of the STS in face processing in humans. Several studies have shown STS activation when viewing static faces George et al 2001, Halgren et al 1999, Hoffman and Haxby 2000, Kanwisher et al 1997. STS activation also has been shown during perception of eye and mouth movement Puce et al 1998, Wicker et al 1998.

The distinct roles of the FG and STS in face processing are not known. Hoffman and Haxby (2000) have attempted to differentiate the functions of these regions by looking at variations in task instructions. They found that subjects had greater activation in the FG when instructed to pay attention to the identity of the person and greater activation in the STS when instructed to attend to the direction of gaze (Hoffman and Haxby, 2000). The present study attempts to further elucidate the roles of the FG and STS by investigating activation in these regions in response to viewing multiple face and gaze orientations.

The effect of head and gaze orientation on brain activation is unclear from current literature. Monkey studies have indicated that there is a differential response in STS cells to different face and gaze orientations Perrett et al 1982, Perrett et al 1985. These monkey studies revealed that cells with preference for a specific face direction also prefer the corresponding gaze direction (i.e., cells that respond to face forward also show greatest response to direct gaze) (Perrett et al., 1985). Furthermore, Perrett et al. (1985) found that sensitivity to gaze direction could override sensitivity to head orientation, which provided supported for a model in which gaze direction could affect perception of head orientation, but not vice versa.

Only two studies with limited subject numbers (N · 7) have attempted to differentiate brain activation in humans for specific views of the face George et al 2001, Tong et al 2000. George et al., (2001) used a block-design functional MRI (fMRI) study to investigate brain activation when subjects viewed varying face and gaze orientations (a 2 × 2 factorial design of head and gaze orientation). In the study, subjects were asked to determine the gender of the faces they viewed. Although both FG and STS activation was observed during the task, no difference in activation in either of these regions was revealed for the comparison of head orientation, regardless of gaze orientation. The only effect of head orientation was found in the posterior striate and extrastriate regions (V1/V2) and the left motor cortex, which showed greater activation when viewing frontal versus angled head orientations, regardless of gaze angle. Tong et al. (2000) used fMRI to look specifically at the fusiform face area and found no differences in activation for frontal versus profile view of the head, when subjects passively viewed the faces or performed a one-back matching task. However, the study revealed decreasing FG activation as the head orientation was rotated further away from view, with the least activation for a view of the back of the head.

Additional functional imaging studies have tried to differentiate the response to direct gaze versus averted gaze, but there is no evidence of a consistent distinct response to either of these gaze orientations. One study found greater activation in the right amygdala for direct gaze versus averted gaze in a gaze discrimination task (Kawashima et al., 1999). In a second study, subjects who passively viewed faces showed greater activation in the left STS and bilaterally in the intraparietal sulcus (IPS) to averted gaze in comparison to direct gaze (Hoffman and Haxby, 2000). A recent study showed greater activation in the FG, but not the STS, for forward gaze compared to averted gaze during a gender determination task (George et al., 2001).

To further investigate these inconsistencies of brain activation in response to variations in head and gaze orientation, especially with respect to activation in the FG and STS, we used event-related fMRI with a 2 × 2 factorial design of head and gaze orientation. Since there is clear evidence that the FG is involved in featural processing of the face, and since angled head orientation provide altered views of facial features, we hypothesized that FG activation would be modulated by variations in head orientation. Since the STS has recently been implicated in processing changeable aspects of the face such as gaze direction and since monkey studies revealed cells responsive to face and gaze direction, we hypothesized that STS activation would be modulated by variations in both head and gaze direction.

Section snippets

Subjects

A total of 18 healthy, right-handed subjects (9 female, 9 male), ages 18–24 (mean 20.9) years, participated in the study after giving written informed consent. The human subjects committee at Stanford University School of Medicine approved all protocols used in this study.

Stimuli

The stimuli consisted of 60 unique static pictures of faces, as well as 45 isoluminant scrambled images. Pictures of faces of college-aged models were acquired with a digital camera against a common background at a distance of

Task performance

Accuracy for all four experimental conditions was high (99 ± 4%) and repeated-measures ANOVA revealed no significant difference in the accuracy for any of the four conditions. Repeated-measures ANOVA demonstrated a significant effect on RT of both head orientation [F (1,17) = 54.27, P < 0.0001] and gaze direction [F (1,17) = 10.04, P < 0.006] (mean AA reaction time, 872 ms; mean AF reaction time, 877 ms; mean FA reaction time, 841 ms; mean FF reaction time, 766 ms). There also was a significant

Discussion

To our knowledge, this is the first study to use an event-related design to investigate FG and posterior STS activation during gaze processing. This study provides evidence that both head and gaze orientation significantly affect gaze processing. The 2 × 2 factorial design, with head orientation (face forward vs. face angled) as one factor and gaze orientation (gaze forward vs. gaze angled) as a second factor, allowed investigation of effects of both head and gaze orientation on gaze

Acknowledgements

Supported by NIH grants MH50047, MH01142, HD31715, and HD40761.

References (61)

  • C.M. Leonard et al.

    Neurons in the amygdala of the monkey with responses selective for faces

    Behav. Brain Res.

    (1985)
  • M.-M. Mesulam et al.

    Heterogeneity of cingulate contributions to spatial attention

    Neuroimage

    (2001)
  • M.-M. Mesulam et al.

    Limbic and sensory connections of the inferior parietal lobule (area PG) in the rhesus monkeya study with a new method for horseradish peroxidase histochemistry

    Brain Res.

    (1977)
  • J.B. Poline et al.

    Combining spatial extent and peak intensity to test for activations in functional imaging

    Neuroimage

    (1997)
  • M.J. Taylor et al.

    ERP evidence of developmental changes in processing of faces

    Clin. Neurophysiol

    (1999)
  • B. Wicker et al.

    Brain regions involved in the perception of gazea PET study

    Neuroimage

    (1998)
  • E.H. Aylward et al.

    MRI volumes of amygdala and hippocampus in non-mentally retarded autistic adolescents and adults

    Neurology

    (1999)
  • T. Baumgardner et al.

    Specification of the neurobehavioral phenotype in males with fragile X syndrome

    Pediatrics

    (1995)
  • V. Bruce et al.

    Understanding face recognition

    Br. J. Psychol.

    (1986)
  • M.A. Burock et al.

    Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI

    Neuroreport

    (1998)
  • V.P. Clark et al.

    fMRI study of face perception and memory using random stimulus sequences

    J. Neurophysiol.

    (1998)
  • J.D. Cohen et al.

    PsyScopea new graphic interactive environment for designing psychology experiments

    Behav. Res. Methods Instr. Comp.

    (1993)
  • M. Corbetta

    Fortoparietal cortical networks for directing attention and the eye to visual locationsIdentical, independent, or overlapping neural systems

    Proc. Natl. Acad. Sci.

    (1998)
  • M. Corbetta et al.

    Voluntary orienting is dissociated from target detection in human posterior parietal cortex

    Nature Neurosci.

    (2000)
  • A. Dale

    Optimal experimental design for event-related fMRI

    Hum. Brain Mapp.

    (1999)
  • H.M. Duvernoy et al.

    The Human BrainSurface, Three-Dimensional Sectional Anatomy with MRI, and Blood Supply. 2nd completely rev. and enl. ed.

    (1999)
  • M. Eimer et al.

    Prosopagnosia and structural encoding of faces; evidence from event-related potentials

    Neuroreport

    (1999)
  • R. Epstein et al.

    A cortical representation of the local visual environment

    Nature

    (1998)
  • K.J. Friston et al.

    Statistical parametric maps in functional imaginga general linear approach

    Hum. Brain Mapp.

    (1995)
  • G.H. Glover et al.

    Self-navigated spiral fMRIinterleaved versus single-shot

    Magn. Res. Med.

    (1998)
  • Cited by (52)

    • Perception of direct vs. averted gaze in portrait paintings: An fMRI and eye-tracking study

      2018, Brain and Cognition
      Citation Excerpt :

      This is fully in accord with influential models of face perception, which propose that FFA is involved in expression of invariant processing of faces (Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000), as well as with more recent integrative models of face perception, which argue that the FFA contributes to the perception of changeable aspects of faces, through a broad sensitivity to shape information (Bernstein, Erez, & Yovel, 2018; Duchaine & Yovel, 2015). Enhanced activity in the fusiform gyrus during direct gaze perception corresponds to several studies using photographic stimuli (George, Driver, & Dolan, 2001; Pageler et al., 2003). It has been suggested that a higher activation of the fusiform gyrus in these studies, which - much like our own study - involved presentation of many different faces, may reflect the need for a “deeper” processing and encoding of novel faces indicating that the interaction between a beholder and the direct gaze of the subjects in the painted portraits could be associated with the increasing feeling of novelty (Boyarskaya, Sebastian, Bauermann, Hecht, & Tuscher, 2015).

    View all citing articles on Scopus
    View full text