Elsevier

NeuroImage

Volume 133, June 2016, Pages 484-497
NeuroImage

Interpersonal frontopolar neural synchronization in group communication: An exploration toward fNIRS hyperscanning of natural interactions

https://doi.org/10.1016/j.neuroimage.2016.03.059Get rights and content

Highlights

  • fNIRS hyperscannning of quadratic communication reveals enhanced frontopolar INS.

  • The communication effect on neural synchronization shows timescale dependence.

  • Skin blood flow removal fosters significant improvement in capturing INS changes.

  • Caution is needed for preprocessing to avoid excessive neural fluctuation removal.

Abstract

Research of interpersonal neural synchronization (INS) using functional near-infrared spectroscopy (fNIRS) hyperscanning is an expanding nascent field. This field still requires the accumulation of findings and establishment of analytic standards. In this study, we therefore intend to extend fNIRS-based INS research in three directions: (1) verifying the enhancement of frontopolar INS by natural and unstructured verbal communication involving more than two individuals; (2) examining timescale dependence of the INS modulation; and (3) evaluating the effects of artifact reduction methods in capturing INS. We conducted an fNIRS hyperscanning study while 12 groups of four subjects were engaged in cooperative verbal communication. Corresponding to the three objectives, our analyses of the data (1) confirmed communication-enhanced frontopolar INS, as expected from the region's roles in social communication; (2) revealed the timescale dependency in the INS modulation, suggesting the merit of evaluating INS in fine timescale bins; and (3) determined that removal of the skin blood flow component engenders substantial improvement in sensitivity to communication-enhanced INS and segregation from artifactual synchronization, and that caution for artifact reduction preprocessing is needed to avoid excessive removal of the neural fluctuation component. Accordingly, this study provides a prospective technical basis for future hyperscanning studies during daily communicative activities.

Introduction

The functional study of the human brain has recently addressed both conventional function mapping and functional connectivity of regions within an individual brain, and the dynamic relationship of neural activities between multiple brains of interacting individuals. The rapidly growing interest in interpersonal neural synchronization (INS) by measuring socially communicating multiple brains, termed “hyperscanning”, has been reflected in the number of review articles in the field (Babiloni and Astolfi, 2014, Hari and Kujala, 2009, Hasson et al., 2012, Koike et al., 2015, Konvalinka and Roepstorff, 2012, Schilbach et al., 2013). Functional magnetic resonance imaging (fMRI) studies (Bilek et al., 2015, Koike et al., 2016, Schippers et al., 2010, Stephens et al., 2010, Tanabe et al., 2012) and electroencephalography (EEG) studies (Astolfi et al., 2010, De Vico Fallani et al., 2010, Dumas et al., 2010, Kawasaki et al., 2013, Konvalinka et al., 2014, Lindenberger et al., 2009, Sanger et al., 2012) have shown that verbal and nonverbal communication induces interpersonal relationships among neural activities. Remarkably, it has been shown that the level of the interpersonal neural synchronization (INS) correlates with the level of understanding between partners (Stephens et al., 2010), a psychometric measure of social expertise (Bilek et al., 2015), and enhancement of a behavioral index of shared attention (Koike et al., 2016), which underscores the importance of investigating INS for facilitating communication and interactions.

Functional near-infrared spectroscopy (fNIRS) is a promising method for investigating the relationship between interpersonal interactions in natural settings and their neural activities on account of its cost-effectiveness, low constraints on measurements, and relatively high tolerance of head/body motion (Egetemeir et al., 2011, Scholkmann et al., 2013). Indeed, an increasing number of fNIRS hyperscanning studies have been recently published. These studies investigate INS during interactions, both verbal (Holper et al., 2013, Jiang et al., 2012, Jiang et al., 2015), semi-verbal (cooperative singing/humming (Osaka et al., 2015)), and nonverbal types (Cui et al., 2012, Dommer et al., 2012, Funane et al., 2011, Holper et al., 2012). Consistent with other studies using fMRI or EEG, these studies have revealed a general increase of INS during communications or cooperative and engaging interactions. This suggests the possible contribution of INS to successful communication.

In exploiting the strength of fNIRS and applying the hyperscanning method to various forms of unconstrained, real-life communications/interactions, however, both further evidence accumulation and methodological refinements are required. The present study therefore intends to extend fNIRS-based research of neural synchronization during communication in three directions: (1) verifying the enhancement of frontopolar INS by natural and unstructured verbal communication involving more than two individuals; (2) examining timescale dependence of the INS modulation induced by communication; and (3) evaluating the effects of artifact reduction methods in capturing INS using fNIRS.

Regarding the first objective, although some recent studies have begun extending it into natural triadic communication (Jiang et al., 2015) and interactions involving additional individuals (Duan et al., 2015), most previous studies have focused on dyadic and controlled forms of communication. Unstructured interactions involving more than two individuals entail aspects that do not exist in such controlled dyadic communication. They are highly dynamic processes (Konvalinka and Roepstorff, 2012) where roles and directions become indistinct, and allows moments when some participants are not taking overt communicative actions. In such situations, relative to the processes directly causing overt communicative actions, the roles of implicit social cognitive processes can increase their importance. In this regard, the medial rostral prefrontal cortex to the frontopolar cortex, or Brodmann Area (BA) 10, has been implicated in processes that are deemed essential for successful communication, such as mentalizing; understanding the beliefs, desires, and goals of others; and multi-task coordination (Amodio and Frith, 2006, Gilbert et al., 2006, Stephens et al., 2010). It is thus expected to show enhanced INS during cooperative communication. However, the results on the enhancing effect of cooperative communication on INS in the frontopolar region have been inconclusive among the fNIRS hyperscanning studies that included that region in the measurement (Cheng et al., 2015, Cui et al., 2012, Dommer et al., 2012, Funane et al., 2011, Jiang et al., 2012). Therefore, we intend to verify the hypothesis that interpersonal synchronization in the frontopolar neural activities increases during a cooperative and unstructured verbal communication involving more than two individuals.

For the second objective, previous INS studies using fNIRS captured INS by averaging synchronization measures over a relatively wide-ranging timescale. In addition, the timescale range for averaging INS considerably differed among the studies. In particular, among the studies that used the wavelet transform coherence method to evaluate INS, as in the present study, Cui et al., and later Cheng et al., averaged the coherence estimates over a task-related band period of 3.2–12.8 s or its equivalent of 0.3–0.08 Hz (Cheng et al., 2015, Cui et al., 2012). Osaka et al. also focused on the same band based on the observation that their singing/humming task was performed on that timescale range (Osaka et al., 2015). Furthermore, Holper et al. averaged over a task-related band of 2–4 s (0.25–0.5 Hz) and additionally a heart rate band of 0.4–1 s (1–2.5 Hz) for comparison (Holper et al., 2012). Dommer et al. likewise averaged INS on two bands of low-frequency oscillations at 0.06–0.2 Hz and a heart rate frequency of 0.7–4 Hz (Dommer et al., 2012). However, in more natural and unstructured interactions, especially involving more than two individuals without fixed roles such as sender and receiver, relevant behavioral and cognitive processes may distribute over multiple frequencies and defining a task-related band can become difficult or less appropriate. Meanwhile, Jiang et al. used a slow-frequency band for averaging either 0.01–0.1 Hz (Jiang et al., 2012) or 0.02–0.2 Hz (Jiang et al., 2015) in investigating natural verbal communication. On the other hand, heterogeneous contributions of different sub-scales of low-frequency oscillations (LFO < 0.1 Hz) on functional network formation and psychological functions have been indicated by recent fMRI (Baria et al., 2011, Chang and Glover, 2010, Xue et al., 2014, Zuo et al., 2010) and fNIRS studies (Nakao et al., 2013, Sasai et al., 2011). In addition, recent resting-state fMRI studies indicated the functional contribution of higher-frequency (> 0.1 Hz) fluctuations in blood oxygen level dependent (BOLD) signals (Chen and Glover, 2015, Gohel and Biswal, 2015). Therefore, to investigate the possibility of timescale-dependent modulation of INS by communication, and to avoid the risk of smearing out significant effects by averaging over wide timescales, we examined the effect of communication on INS in finer timescale bins over a wide range. This approach was also expected to be useful in clarifying how the artifact reduction methods, which are introduced below, influence the estimation of INS differently over timescales.

In terms of the third objective, fNIRS hyperscanning of natural, unconstrained communication is expected to be more prone to artifacts than in well-controlled experimental settings, supporting the importance of artifact reduction methods. Previous studies investigated effectiveness of artifact reduction methods for better detection of task-evoked responses (Brigadoi et al., 2014, Cooper et al., 2012, Robertson et al., 2010). However, the effectiveness of those methods for examination of INS has not been sufficiently explored to date. In this study, we focused on the effect of two methods: dual source-detector (S-D) regression to compensate the skin blood-flow component, and discrete wavelet-based motion artifact reduction (WBMR). Both methods have been reported as highly effective for evoked-response studies.

An important point to note is that the fNIRS measurement is highly sensitive to blood-flow changes in the scalp that are unrelated to brain activation and more susceptible to systemic noises and motion-induced artifacts (Haeussinger et al., 2014, Kirilina et al., 2012, Takahashi et al., 2011, Toronov et al., 2001a). Consideration of this issue is particularly important for studying neural synchrony in daily-life communications, which are more dynamic and involve more bodily and autonomic activities compared to well-controlled laboratory experiments. An approach to addressing this issue is to add a measurement of the superficial skin blood-flow signal and to exclude the corresponding component contained in the fNIRS measurement using the dual/multiple S-D regression or other similar methods. Previous studies have shown that this type of combining S-D pairs of different distances is very helpful in removing global artifacts induced by motion, cardiac and respiratory activities, or autonomic nervous system changes, and it thus improves the signal-to-noise ratio (Luu and Chau, 2009, Robertson et al., 2010, Saager and Berger, 2005, Saager et al., 2011, Zhang et al., 2009). Nevertheless, its effectiveness has not been proved for evaluation of INS, although a review suggested that its use is desirable for future studies (Scholkmann et al., 2013). We expected that the dual S-D regression would improve the sensitivity for enhanced INS by facilitating the dissociation of frontopolar neural signals from artifact components.

WBMR method (Molavi and Dumont, 2012) uses discrete wavelet transform to detect and remove outliers, often expressed as spike-like artifacts on a time course. It was shown to be one of the most effective methods for noise reduction and improvement of sensitivity for task-evoked activation (Brigadoi et al., 2014, Cooper et al., 2012); however, its effectiveness has not been systematically investigated for evaluation of INS. Since natural verbal communications can be more prone to head or facial motions, we expected that the WBMR method would contribute to reducing spike artifacts induced by such motions, leading to better detection of INS.

Section snippets

Participants

Twelve groups of four members, totaling 48 healthy, right-handed Japanese university students (20 females and 28 males; age range of 19–24 years; mean 21.9), participated in the experiment. Handedness was evaluated using the Edinburgh Handedness Inventory (Oldfield, 1971). All subjects had normal or corrected-to-normal vision and reported no history of neurological or psychiatric conditions. Except two groups, all participants in each group were acquainted with each other. The constitution of

Characteristics of communication behavior

In summarizing the utterance events during the Communication condition over all groups, sessions, and blocks, the mean ± SD of utterance frequency per group was 0.192 ± 0.057 Hz, and the mean ± SD of utterance frequency per subject was 0.048 ± 0.025 Hz. Figs. 3(a) and (c) show the distribution of group- and subject-wise inter-utterance intervals (IUIs). The median of the group- and subject-wise IUIs were 4 s and 13 s, respectively. The tail of the group-wise IUI distribution corresponds to occasional

Interpersonal prefrontal synchrony in unstructured group verbal communication

The current study investigated the neural synchrony between individuals engaged in natural communication. As expected from its role in social interactions, such as mentalizing, we observed an increase of the frontopolar INS during communication compared to during silent thinking. The results of fNIRS studies to date have been indecisive on the enhancing effect of cooperative communication on INS in the frontopolar region. Some studies showed a significant relationship between communication and

Conclusions

Research of INS using fNIRS hyperscanning is a nascent field that calls for accumulation of findings and establishment of analytic standards. In this study, we conducted an fNIRS hyperscanning study while groups of four young adults were engaged in cooperative verbal communication. Our results (1) confirm that frontopolar INS is enhanced by unstructured communication, as expected from the region's roles in communication; (2) revealed the existence of timescale dependency in the INS modulation

Acknowledgments

We thank the staff of Hitachi Ltd. for the collaborative development of the fNIRS device, as well as the participants and all our colleagues in IDAC, Tohoku University for their support. This study was partially supported by System Development Program for Advanced Measurement and Analysis, Japan Science and Technology Agency (JST; http://www.jst.go.jp/), and also by MEXT/JSPS (http://www.jsps.go.jp/) KAKENHI Grant Numbers 23700137 and 15H01771.

References (86)

  • L. Gagnon et al.

    Short separation channel location impacts the performance of short channel regression in nirs

    NeuroImage

    (2012)
  • L. Gagnon et al.

    Improved recovery of the hemodynamic response in diffuse optical imaging using short optode separations and state-space modeling

    NeuroImage

    (2011)
  • L. Gagnon et al.

    Quantification of the cortical contribution to the NIRS signal over the motor cortex using concurrent NIRS–fMRI measurements

    NeuroImage

    (2012)
  • F.B. Haeussinger et al.

    Reconstructing functional near-infrared spectroscopy (fNIRS) signals impaired by extra-cranial confounds: an easy-to-use filter method

    NeuroImage

    (2014)
  • U. Hasson et al.

    Brain-to-brain coupling: a mechanism for creating and sharing a social world

    Trends Cogn. Sci.

    (2012)
  • B.J. He et al.

    The temporal structures and functional significance of scale-free brain activity

    Neuron

    (2010)
  • L. Holper et al.

    The teaching and the learning brain: a cortical hemodynamic marker of teacher–student interactions in the socratic dialog

    Int. J. Educ. Res.

    (2013)
  • L. Holper et al.

    Between-brain connectivity during imitation measured by fNIRS

    NeuroImage

    (2012)
  • R.W. Homan et al.

    Cerebral location of international 10–20 system electrode placement

    Electroencephalogr. Clin. Neurophysiol.

    (1987)
  • R.M. Hutchison et al.

    Dynamic functional connectivity: promise, issues, and interpretations

    NeuroImage

    (2013)
  • T. Katura et al.

    Quantitative evaluation of interrelations between spontaneous low-frequency oscillations in cerebral hemodynamics and systemic cardiovascular dynamics

    NeuroImage

    (2006)
  • E. Kirilina et al.

    The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy

    NeuroImage

    (2012)
  • T. Koike et al.

    Neural substrates of shared attention as social memory: a hyperscanning functional magnetic resonance imaging study

    NeuroImage

    (2016)
  • T. Koike et al.

    Hyperscanning neuroimaging technique to reveal the “two-in-one” system in social interactions

    Neurosci. Res.

    (2015)
  • I. Konvalinka et al.

    Frontal alpha oscillations distinguish leaders from followers: multivariate decoding of mutually interacting brains

    NeuroImage

    (2014)
  • H. Obrig et al.

    Spontaneous low frequency oscillations of cerebral hemodynamics and metabolism in human adults

    NeuroImage

    (2000)
  • R.C. Oldfield

    The assessment and analysis of handedness: the Edinburgh inventory

    Neuropsychologia

    (1971)
  • R.B. Saager et al.

    Two-detector corrected near infrared spectroscopy (C-NIRS) detects hemodynamic activation responses more robustly than single-detector NIRS

    NeuroImage

    (2011)
  • S. Sasai et al.

    Frequency-specific functional connectivity in the brain during resting state revealed by NIRS

    NeuroImage

    (2011)
  • F. Scholkmann et al.

    A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology

    NeuroImage

    (2014)
  • T. Takahashi et al.

    Influence of skin blood flow on near-infrared spectroscopy signals measured on the forehead during a verbal fluency task

    NeuroImage

    (2011)
  • Q. Zhang et al.

    Adaptive filtering to reduce global interference in non-invasive NIRS measures of brain activation: how well and when does it work?

    NeuroImage

    (2009)
  • H. Zhang et al.

    Functional connectivity as revealed by independent component analysis of resting-state fNIRS measurements

    NeuroImage

    (2010)
  • X.-N. Zuo et al.

    The oscillating brain: complex and reliable

    NeuroImage

    (2010)
  • S. Achard et al.

    A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs

    J. Neurosci.

    (2006)
  • E.A. Allen et al.

    Tracking whole-brain connectivity dynamics in the resting state

    Cereb. Cortex

    (2014)
  • D.M. Amodio et al.

    Meeting of minds: the medial frontal cortex and social cognition

    Nat. Rev. Neurosci.

    (2006)
  • L. Astolfi et al.

    Neuroelectrical hyperscanning measures simultaneous brain activity in humans

    Brain Topogr.

    (2010)
  • A.T. Baria et al.

    Anatomical and functional assemblies of brain bold oscillations

    J. Neurosci.

    (2011)
  • E. Bilek et al.

    Information flow between interacting human brains: Identification, validation, and relationship to social expertise

    Proc. Natl. Acad. Sci. U. S. A.

    (2015)
  • X. Cheng et al.

    Synchronous brain activity during cooperative exchange depends on gender of partner: a fNIRS-based hyperscanning study

    Hum. Brain Mapp.

    (2015)
  • R.J. Cooper et al.

    A systematic comparison of motion artifact correction techniques for functional near-infrared spectroscopy

    Front. Neurosci.

    (2012)
  • F. De Vico Fallani et al.

    Defecting or not defecting: how to “read” human behavior during cooperative games by EEG measurements

    PLoS One

    (2010)
  • Cited by (145)

    View all citing articles on Scopus
    View full text