Elsevier

Neuropsychologia

Volume 48, Issue 2, January 2010, Pages 477-490
Neuropsychologia

Neural responses to rigidly moving faces displaying shifts in social attention investigated with fMRI and MEG

https://doi.org/10.1016/j.neuropsychologia.2009.10.005Get rights and content

Abstract

A widely adopted neural model of face perception (Haxby, Hoffman, & Gobbini, 2000) proposes that the posterior superior temporal sulcus (STS) represents the changeable features of a face, while the face-responsive fusiform gyrus (FFA) encodes invariant aspects of facial structure. ‘Changeable features’ of a face can include rigid and non-rigid movements. The current study investigated neural responses to rigid, moving faces displaying shifts in social attention. Both functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) were used to investigate neural responses elicited when participants viewed video clips in which actors made a rigid shift of attention, signalled congruently from both the eyes and head. These responses were compared to those elicited by viewing static faces displaying stationary social attention information or a scrambled video displaying directional motion. Both the fMRI and MEG analyses demonstrated heightened responses along the STS to turning heads compared to static faces or scrambled movement conditions. The FFA responded to both turning heads and static faces, showing only a slight increase in response to the dynamic stimuli. These results establish the applicability of the Haxby model to the perception of rigid face motions expressing changes in social attention direction. Furthermore, the MEG beamforming analyses found an STS response in an upper frequency band (30–80 Hz) which peaked in the right anterior region. These findings, derived from two complementary neuroimaging techniques, clarify the contribution of the STS during the encoding of rigid facial action patterns of social attention, emphasising the role of anterior sulcal regions alongside previously observed posterior areas.

Introduction

In functional magnetic resonance imaging (fMRI) studies, face-responsive activations are frequently seen in the posterior superior temporal sulcus (STS), the fusiform gyrus (the Fusiform Face Area or FFA) and posterior sections of the lateral occipital lobes (the Occipital Face Area or OFA; Andrews and Ewbank, 2004, Hoffman and Haxby, 2000, Kanwisher et al., 1997, Puce et al., 1998). These regions have been proposed as a core neural system for face perception by Haxby, Hoffman, and Gobbini (2000), who describe a fully integrated system in which the OFA, FFA and STS respond to the presentation of a face but with a division of labour between the FFA and posterior STS.

The dissociation of function between these ventral (FFA) and dorsal (STS) face-responsive regions has been demonstrated in various ways; for example by the finding that selective attention to the identity of a face increases activation in the FFA, whereas attention to the static gaze direction in the same stimulus produces a preferential activation of the posterior STS (Hoffman & Haxby, 2000). Based on such evidence, Haxby et al. (2000) proposed that invariant aspects of facial structure, used for purposes such as face recognition, are encoded in the face-responsive fusiform region, while changeable aspects of a face (e.g. the eyes and mouth), needed for social communicative functions, are represented in the posterior STS. As Haxby et al. note, this division between aspects of face perception signalled primarily from changeable and non-changeable cues parallels a distinction commonly made in cognitive models of face perception (Bruce & Young, 1986).

The perception and interpretation of changeable facial aspects is integral to social communication. In particular, perceiving another individual’s gaze direction provides information about what is important to them in the surrounding environment, from which one can extrapolate to their thoughts, motivations and intentions within the current circumstances (Baron-Cohen, 1995, for a recent review see; Frischen, Bayliss, & Tipper, 2007). Gobbini and Haxby (2007) incorporated these processes into a revision of their neural model of face perception, suggesting that the involvement of the posterior STS extends beyond a basic visual analysis of the face to extract the intentional information conveyed by these changeable features.

In everyday interaction, the changeable aspects of a face form a continuous display of dynamic social signals. These facial movements can be either rigid or non-rigid. Both types of motion convey salient social information. Rigid head motions provide insight in the direction of attention of an individual and provide a different view of the face (Pike, Kemp, Towell, & Phillips, 1997), while non-rigid motions of internal face features provide visual information relating to speech, expression and eye movement (O’Toole, Roark, & Abdi, 2002). Gaze direction, or social attention, can be determined from both non-rigid internal eye motions and rigid head motions Langton, 2000, Langton et al., 2004, Perrett et al., 1992. Previous studies examining the neural basis of the perception of social attention have focused on internal eye-gaze direction, either in a static face Engell and Haxby, 2007, Hoffman and Haxby, 2000, Materna et al., 2008, Taylor et al., 2001, Wicker et al., 1998 or through a non-rigid motion of the eyes Conty et al., 2007, Pelphrey et al., 2003, Pelphrey et al., 2004.

Head orientation also represents an important cue of social attention direction. Information about head orientation and eye direction are integrated in the perception of gaze (Langton, 2000, Langton et al., 2004, Perrett et al., 1992). Single cell studies in the macaque STS have indicated that individual cells respond to a conjunction of information from the eyes, head and body when computing the direction of social attention (Perrett et al., 1992). Furthermore, research with human participants has shown that when directional information from eye-gaze direction and head orientation is in conflict recognition of social attention is slowed (Langton, 2000, Langton et al., 2004) and the fMRI-indexed STS response may be reduced (George, Driver, & Dolan, 2001). However, with rigid head motion, these head and eye cues are inherently congruous and thus potentially elicit an increased STS response.

The first experiment of the current study was therefore designed to investigate the applicability of the Haxby model of face perception (Haxby et al., 2000) to faces that move rigidly to convey a congruent head-eye shift in social attention. FMRI was used to identify the spatial profile of the haemodynamic response to dynamic shifts in social attention conveyed by rigid face movements when contrasted with static social attention stimuli and non-social directional motion. By carrying out a concurrent functional localiser scan, designed to activate the core system of Haxby et al.’s face perception model, the activations identified in the main experimental contrasts could be defined both in terms of their anatomical location and their functional role within current conceptions of face perception Gobbini and Haxby, 2007, Haxby et al., 2000. Furthermore, a functional region of interest (fROI) analysis allowed for an examination of the main experimental contrasts within functionally defined regions of visual cortex (for discussion see; Saxe, Brett, & Kanwisher, 2006).

Having established the spatial profile of the haemodynamic response to rigidly moving social attention stimuli, the second experiment was designed to increase understanding of these activations by employing a direct measure of neural activity, magnetoencephalography (MEG). Recent advances in MEG beamforming source localisation potentially deliver spatial results of a similar resolution to fMRI Hillebrand et al., 2005, Singh, 2006 and have successfully been used to investigate cognitive function Bayless et al., 2006, Cornelissen et al., 2009, Itier et al., 2006, Pammer et al., 2004, Pammer et al., 2006, Singh et al., 2002, Singh et al., 2003. However, this body of research remains in its youth, therefore the above-described fMRI experiment provided a spatial structure around which to frame source localisations identified through MEG beamforming before gaining further insight from the multidimensional MEG signal.

Due to its excellent temporal resolution, MEG can be used to investigate both the time course and frequency content of neural responses. MEG has been employed to examine the time course of neural responses to faces Itier et al., 2006, Liu et al., 2002, Sato et al., 2008, Taylor et al., 2001, but as of yet little is known about the frequencies of neural oscillation which contribute to the neural system underlying face perception. Coherent object perception has been associated with an increase in oscillatory power in frequencies above 30 Hz in EEG studies Rodriguez et al., 1999, Tallon-Baudry and Bertrand, 1999 and a decrease in oscillatory power in frequencies below 30 Hz in both EEG and MEG studies Lachaux et al., 2005, Maratos et al., 2007. Decreased oscillatory power in frequencies below 30 Hz has also been observed with MEG when participants viewed point-light displays of biological motion (Singh et al., 2002). On this basis, MEG beamforming was carried out in two distinct frequency bands, a lower band (4–30 Hz) and an upper band (30–80 Hz), so as to examine the spatial distribution of neural oscillations within these frequency ranges during the perception of dynamic social attention stimuli.

In both the fMRI and MEG experiments, neural responses to dynamic face stimuli which conveyed social attention through a rigid and congruous head and eye shift (Turning Heads) were investigated. These activations were compared to those elicited by a static averted face displaying stationary social attention information (Static Heads) and also to a moving scramble video, conveying a directional shift but in a non-social domain (Moving Scrambles). The Turning Heads stimuli included turns that communicated both a shift towards the participant as if to engage in mutual gaze (Mutual Head Turns) and a shift away from the participant to averted gaze (Averted Head Turns). Traditionally defined face-responsive regions were identified by contrasting fMRI activations to static face and place stimuli Andrews and Ewbank, 2004, Downing et al., 2006, Kanwisher et al., 1997. These face-responsive regions were identified with a separate localiser scan for two reasons. In the fMRI whole-brain analysis the brain activations elicited by the main experimental conditions could be ascribed a functional label relating to current models of face perception Gobbini and Haxby, 2007, Haxby et al., 2000. Additionally, analyses could be restricted to these face-responsive regions using a fROI approach which benefits from increased statistical power and allows for an examination of the main experimental effects within functionally defined brain regions.

In the fMRI experiment, Turning Heads were contrasted with Moving Scrambles to identify regions that were responsive to faces which display social attention information, while controlling for the neural response to directional movement. Activations were anticipated in the STS, FFA and OFA of the Haxby model (Haxby et al., 2000), and would therefore be thought to represent a response to the basic perceptual analysis of a face which may be augmented by the social attention conveyed by the dynamic face stimulus (Gobbini & Haxby, 2007). By then contrasting Turning Heads with Static Heads, the neural activations were narrowed to those that appear to dynamic social attention over and above static social attention. In this contrast, activations were again anticipated in the STS to both the dynamic and intentional components of the Turning Heads stimuli, while any residual FFA and OFA activation might be ascribed to the increase in facial structural information available from the changing face angle O’Toole et al., 2002, Schultz and Pilz, 2009. Contrasts between Mutual and Averted Head Turns were expected to reveal differential activations in the STS.

In the MEG experiment, the Turning Heads (incorporating both Mutual and Averted Head Turns), Static Heads and Moving Scrambles conditions, each comprising identical stimuli to those used in the fMRI experiment, were again examined. MEG beamforming source localisations were used to both spatially identify neural responses and examine the frequency of neural oscillations contributing to these responses. It was anticipated that frequencies below and above 30 Hz (the lower band, 4–30 Hz, and upper band, 30–80 Hz, respectively) would contribute differentially to the perception of dynamic, rigid social attention.

Section snippets

Participants

Seventeen healthy volunteers (seven males, ten females, mean age = 24.94, s.d. = 4.16) participated in the fMRI experiment. The participants were right handed and had normal or corrected-to-normal vision. All gave informed consent to participate in the study. Ethical approval for this study was obtained from York Neuroimaging Centre (YNiC) and York University Department of Psychology.

Localiser scan

The localiser scan was carried out prior to the main experimental scan. It was designed to activate face-responsive

Experiment 2—an MEG investigation of neural responses to moving faces displaying shifts in social attention

The second experiment was designed using MEG to augment understanding of the cortical regions involved in the perception of moving faces signaling a rigid change in social attention direction. The above-reported fMRI results provided a spatial framework for MEG source localisations identified with an MEG beamforming technique (Hillebrand et al., 2005, Singh, 2006). In the current study, MEG beamforming was used to gain insight into the spatial distribution of neural oscillations within two

References (80)

  • A.D. Engell et al.

    Facial expression and gaze-direction in human superior temporal sulcus

    Neuropsychologia

    (2007)
  • I.P. Fawcett et al.

    The temporal frequency tuning of human visual cortex investigated using synthetic aperture magnetometry

    Neuroimage

    (2004)
  • P. Fries et al.

    The gamma cycle

    Trends in Neurosciences

    (2007)
  • K.J. Friston et al.

    A critique of functional localisers

    Neuroimage

    (2006)
  • N. George et al.

    Seen gaze-direction modulates fusiform activity and its coupling with other brain areas during face processing

    Neuroimage

    (2001)
  • M.I. Gobbini et al.

    Neural systems for recognition of familiar faces

    Neuropsychologia

    (2007)
  • K. Grill-Spector et al.

    fMR-adaptation: A tool for studying the functional properties of human cortical neurons

    Acta Psychology (Amsterdam)

    (2001)
  • A. Hadjipapas et al.

    Assessing interactions of linear and nonlinear neuronal sources using MEG beamformers: A proof of concept

    Clinical Neurophysiology

    (2005)
  • J. Haxby et al.

    The distributed human neural system for face perception

    Trends in Cognitive Sciences

    (2000)
  • R.J. Itier et al.

    Inversion and contrast-reversal effects on face processing assessed by meg

    Brain Research

    (2006)
  • M. Jenkinson et al.

    A global optimisation method for robust affine registration of brain images

    Medical Image Analysis

    (2001)
  • K. Kinsey et al.

    Cortical oscillatory activity associated with the perception of illusory and real visual contours

    International Journal of Psychophysiology

    (2009)
  • D. Kozinska et al.

    Automatic alignment of EEG/MEG and MRI data sets

    Clinical Neurophysiology

    (2001)
  • J-P. Lachaux et al.

    The many faces of the gamma band response to complex visual stimuli

    Neuroimage

    (2005)
  • F.A. Maratos et al.

    The spatial distribution and temporal dynamics of brain regions activated during the perception of object and non-object patterns

    Neuroimage

    (2007)
  • S. Materna et al.

    The posterior superior temporal sulcus is involved in social communication not specific for the eyes

    Neuropsychologia

    (2008)
  • A.J. O’Toole et al.

    Recognizing moving faces: A psychological and neural synthesis

    Trends in Cognitive Sciences

    (2002)
  • N.M. Pageler et al.

    Effect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus

    Neuroimage

    (2003)
  • K. Pammer et al.

    Attentional shifting and the role of the dorsal pathway in visual word recognition

    Neuropsychologia

    (2006)
  • K. Pammer et al.

    Visual word recognition: The first half second

    Neuroimage

    (2004)
  • K.A. Pelphrey et al.

    Brain activation evoked by perception of gaze shifts: The influence of context

    Neuropsychologia

    (2003)
  • G. Pfurtscheller et al.

    Event-related EEG/MEG synchronization and desynchronization: Basic principles

    Clinical Neurophysiology

    (1999)
  • R. Saxe et al.

    Divide and conquer: A defense of functional localizers

    Neuroimage

    (2006)
  • R. Saxe et al.

    A region of right posterior superior temporal sulcus responds to observed intentional actions

    Neuropsychologia

    (2004)
  • K.D. Singh et al.

    Group imaging of task-related changes in cortical synchronisation using nonparametric permutation testing

    Neuroimage

    (2003)
  • K.D. Singh et al.

    Task-related changes in cortical synchronization are spatially coincident with the hemodynamic response

    Neuroimage

    (2002)
  • C. Tallon-Baudry et al.

    Oscillatory gamma activity in humans and its role in object representation

    Trends in Cognitive Sciences

    (1999)
  • M.J. Taylor et al.

    Magnetoencephalographic evidence of early processing of direction of gaze in humans

    Neuroscience Letters

    (2001)
  • M.J. Taylor et al.

    Face processing in adolescents with and without epilepsy

    International Journal of Psychophysiology

    (2008)
  • R.B.H. Tootell et al.

    From retinotopy to recognition: fMRI in human visual cortex

    Trends in Cognitive Sciences

    (1998)
  • Cited by (48)

    • Perceiving emotional expressions in others: Activation likelihood estimation meta-analyses of explicit evaluation, passive perception and incidental perception of emotions

      2016, Neuroscience and Biobehavioral Reviews
      Citation Excerpt :

      We found that the pSTS was active during both passive perception and explicit evaluation of emotional versus neutral stimuli. Extensive research has shown that this major sulcal landmark in the temporal lobe is sensitive to gaze orientation (Hooker et al., 2003) and the movement of the human body (Kourtzi and Kanwisher, 2000; Saxe et al., 2004b), hands (Gallagher and Frith, 2004; Holle et al., 2010) and face (Lee et al., 2010), either through direct perception (Allison et al., 2000) or implied movement (David and Senior, 2000). However, the pSTS is not sensitive to just any kind of movement as significantly more activation occurs when people view others perform complex versus simple actions (Castelli et al., 2000), physically possible movements compared to impossible movements (Stevens et al., 2000), meaningful versus meaningless movements (Decety et al., 1997; Schultz et al., 2004), and when people observe the transition between meaningful actions within a larger goal-directed activity such as cleaning the kitchen (Zacks et al., 2001).

    • Two neural pathways of face processing: A critical evaluation of current models

      2015, Neuroscience and Biobehavioral Reviews
      Citation Excerpt :

      Overall current data suggest that the face areas in the STS show much greater response to dynamic than static face stimuli. The FFA, on the other hand, show no or a much smaller difference in response to dynamic and static face stimuli (Fox et al., 2009a; Furl et al., 2013, 2014; Lee et al., 2010; Pitcher et al., 2011a; Schultz and Pilz, 2009), suggesting that it does not extract the additional motion information conveyed by the dynamic facial stimuli. The only type of dynamic stimuli that have been shown to activate the FFA more than static stimuli are those that convey morphed facial expressions (Arsalidou et al., 2011; LaBar et al., 2003; Pelphrey et al., 2007; Sato et al., 2004), however, these findings may reflect the extraction of form information rather than dynamic information in the FFA.

    View all citing articles on Scopus

    This work was supported by a PhD studentship grant from 4D Neuroimaging (San Diego) and the University of York. We thank Katherine Newling for her help in conducting the functional region-of-interest analysis.

    View full text