Interpersonal frontopolar neural synchronization in group communication: An exploration toward fNIRS hyperscanning of natural interactions
Introduction
The functional study of the human brain has recently addressed both conventional function mapping and functional connectivity of regions within an individual brain, and the dynamic relationship of neural activities between multiple brains of interacting individuals. The rapidly growing interest in interpersonal neural synchronization (INS) by measuring socially communicating multiple brains, termed “hyperscanning”, has been reflected in the number of review articles in the field (Babiloni and Astolfi, 2014, Hari and Kujala, 2009, Hasson et al., 2012, Koike et al., 2015, Konvalinka and Roepstorff, 2012, Schilbach et al., 2013). Functional magnetic resonance imaging (fMRI) studies (Bilek et al., 2015, Koike et al., 2016, Schippers et al., 2010, Stephens et al., 2010, Tanabe et al., 2012) and electroencephalography (EEG) studies (Astolfi et al., 2010, De Vico Fallani et al., 2010, Dumas et al., 2010, Kawasaki et al., 2013, Konvalinka et al., 2014, Lindenberger et al., 2009, Sanger et al., 2012) have shown that verbal and nonverbal communication induces interpersonal relationships among neural activities. Remarkably, it has been shown that the level of the interpersonal neural synchronization (INS) correlates with the level of understanding between partners (Stephens et al., 2010), a psychometric measure of social expertise (Bilek et al., 2015), and enhancement of a behavioral index of shared attention (Koike et al., 2016), which underscores the importance of investigating INS for facilitating communication and interactions.
Functional near-infrared spectroscopy (fNIRS) is a promising method for investigating the relationship between interpersonal interactions in natural settings and their neural activities on account of its cost-effectiveness, low constraints on measurements, and relatively high tolerance of head/body motion (Egetemeir et al., 2011, Scholkmann et al., 2013). Indeed, an increasing number of fNIRS hyperscanning studies have been recently published. These studies investigate INS during interactions, both verbal (Holper et al., 2013, Jiang et al., 2012, Jiang et al., 2015), semi-verbal (cooperative singing/humming (Osaka et al., 2015)), and nonverbal types (Cui et al., 2012, Dommer et al., 2012, Funane et al., 2011, Holper et al., 2012). Consistent with other studies using fMRI or EEG, these studies have revealed a general increase of INS during communications or cooperative and engaging interactions. This suggests the possible contribution of INS to successful communication.
In exploiting the strength of fNIRS and applying the hyperscanning method to various forms of unconstrained, real-life communications/interactions, however, both further evidence accumulation and methodological refinements are required. The present study therefore intends to extend fNIRS-based research of neural synchronization during communication in three directions: (1) verifying the enhancement of frontopolar INS by natural and unstructured verbal communication involving more than two individuals; (2) examining timescale dependence of the INS modulation induced by communication; and (3) evaluating the effects of artifact reduction methods in capturing INS using fNIRS.
Regarding the first objective, although some recent studies have begun extending it into natural triadic communication (Jiang et al., 2015) and interactions involving additional individuals (Duan et al., 2015), most previous studies have focused on dyadic and controlled forms of communication. Unstructured interactions involving more than two individuals entail aspects that do not exist in such controlled dyadic communication. They are highly dynamic processes (Konvalinka and Roepstorff, 2012) where roles and directions become indistinct, and allows moments when some participants are not taking overt communicative actions. In such situations, relative to the processes directly causing overt communicative actions, the roles of implicit social cognitive processes can increase their importance. In this regard, the medial rostral prefrontal cortex to the frontopolar cortex, or Brodmann Area (BA) 10, has been implicated in processes that are deemed essential for successful communication, such as mentalizing; understanding the beliefs, desires, and goals of others; and multi-task coordination (Amodio and Frith, 2006, Gilbert et al., 2006, Stephens et al., 2010). It is thus expected to show enhanced INS during cooperative communication. However, the results on the enhancing effect of cooperative communication on INS in the frontopolar region have been inconclusive among the fNIRS hyperscanning studies that included that region in the measurement (Cheng et al., 2015, Cui et al., 2012, Dommer et al., 2012, Funane et al., 2011, Jiang et al., 2012). Therefore, we intend to verify the hypothesis that interpersonal synchronization in the frontopolar neural activities increases during a cooperative and unstructured verbal communication involving more than two individuals.
For the second objective, previous INS studies using fNIRS captured INS by averaging synchronization measures over a relatively wide-ranging timescale. In addition, the timescale range for averaging INS considerably differed among the studies. In particular, among the studies that used the wavelet transform coherence method to evaluate INS, as in the present study, Cui et al., and later Cheng et al., averaged the coherence estimates over a task-related band period of 3.2–12.8 s or its equivalent of 0.3–0.08 Hz (Cheng et al., 2015, Cui et al., 2012). Osaka et al. also focused on the same band based on the observation that their singing/humming task was performed on that timescale range (Osaka et al., 2015). Furthermore, Holper et al. averaged over a task-related band of 2–4 s (0.25–0.5 Hz) and additionally a heart rate band of 0.4–1 s (1–2.5 Hz) for comparison (Holper et al., 2012). Dommer et al. likewise averaged INS on two bands of low-frequency oscillations at 0.06–0.2 Hz and a heart rate frequency of 0.7–4 Hz (Dommer et al., 2012). However, in more natural and unstructured interactions, especially involving more than two individuals without fixed roles such as sender and receiver, relevant behavioral and cognitive processes may distribute over multiple frequencies and defining a task-related band can become difficult or less appropriate. Meanwhile, Jiang et al. used a slow-frequency band for averaging either 0.01–0.1 Hz (Jiang et al., 2012) or 0.02–0.2 Hz (Jiang et al., 2015) in investigating natural verbal communication. On the other hand, heterogeneous contributions of different sub-scales of low-frequency oscillations (LFO < 0.1 Hz) on functional network formation and psychological functions have been indicated by recent fMRI (Baria et al., 2011, Chang and Glover, 2010, Xue et al., 2014, Zuo et al., 2010) and fNIRS studies (Nakao et al., 2013, Sasai et al., 2011). In addition, recent resting-state fMRI studies indicated the functional contribution of higher-frequency (> 0.1 Hz) fluctuations in blood oxygen level dependent (BOLD) signals (Chen and Glover, 2015, Gohel and Biswal, 2015). Therefore, to investigate the possibility of timescale-dependent modulation of INS by communication, and to avoid the risk of smearing out significant effects by averaging over wide timescales, we examined the effect of communication on INS in finer timescale bins over a wide range. This approach was also expected to be useful in clarifying how the artifact reduction methods, which are introduced below, influence the estimation of INS differently over timescales.
In terms of the third objective, fNIRS hyperscanning of natural, unconstrained communication is expected to be more prone to artifacts than in well-controlled experimental settings, supporting the importance of artifact reduction methods. Previous studies investigated effectiveness of artifact reduction methods for better detection of task-evoked responses (Brigadoi et al., 2014, Cooper et al., 2012, Robertson et al., 2010). However, the effectiveness of those methods for examination of INS has not been sufficiently explored to date. In this study, we focused on the effect of two methods: dual source-detector (S-D) regression to compensate the skin blood-flow component, and discrete wavelet-based motion artifact reduction (WBMR). Both methods have been reported as highly effective for evoked-response studies.
An important point to note is that the fNIRS measurement is highly sensitive to blood-flow changes in the scalp that are unrelated to brain activation and more susceptible to systemic noises and motion-induced artifacts (Haeussinger et al., 2014, Kirilina et al., 2012, Takahashi et al., 2011, Toronov et al., 2001a). Consideration of this issue is particularly important for studying neural synchrony in daily-life communications, which are more dynamic and involve more bodily and autonomic activities compared to well-controlled laboratory experiments. An approach to addressing this issue is to add a measurement of the superficial skin blood-flow signal and to exclude the corresponding component contained in the fNIRS measurement using the dual/multiple S-D regression or other similar methods. Previous studies have shown that this type of combining S-D pairs of different distances is very helpful in removing global artifacts induced by motion, cardiac and respiratory activities, or autonomic nervous system changes, and it thus improves the signal-to-noise ratio (Luu and Chau, 2009, Robertson et al., 2010, Saager and Berger, 2005, Saager et al., 2011, Zhang et al., 2009). Nevertheless, its effectiveness has not been proved for evaluation of INS, although a review suggested that its use is desirable for future studies (Scholkmann et al., 2013). We expected that the dual S-D regression would improve the sensitivity for enhanced INS by facilitating the dissociation of frontopolar neural signals from artifact components.
WBMR method (Molavi and Dumont, 2012) uses discrete wavelet transform to detect and remove outliers, often expressed as spike-like artifacts on a time course. It was shown to be one of the most effective methods for noise reduction and improvement of sensitivity for task-evoked activation (Brigadoi et al., 2014, Cooper et al., 2012); however, its effectiveness has not been systematically investigated for evaluation of INS. Since natural verbal communications can be more prone to head or facial motions, we expected that the WBMR method would contribute to reducing spike artifacts induced by such motions, leading to better detection of INS.
Section snippets
Participants
Twelve groups of four members, totaling 48 healthy, right-handed Japanese university students (20 females and 28 males; age range of 19–24 years; mean 21.9), participated in the experiment. Handedness was evaluated using the Edinburgh Handedness Inventory (Oldfield, 1971). All subjects had normal or corrected-to-normal vision and reported no history of neurological or psychiatric conditions. Except two groups, all participants in each group were acquainted with each other. The constitution of
Characteristics of communication behavior
In summarizing the utterance events during the Communication condition over all groups, sessions, and blocks, the mean ± SD of utterance frequency per group was 0.192 ± 0.057 Hz, and the mean ± SD of utterance frequency per subject was 0.048 ± 0.025 Hz. Figs. 3(a) and (c) show the distribution of group- and subject-wise inter-utterance intervals (IUIs). The median of the group- and subject-wise IUIs were 4 s and 13 s, respectively. The tail of the group-wise IUI distribution corresponds to occasional
Interpersonal prefrontal synchrony in unstructured group verbal communication
The current study investigated the neural synchrony between individuals engaged in natural communication. As expected from its role in social interactions, such as mentalizing, we observed an increase of the frontopolar INS during communication compared to during silent thinking. The results of fNIRS studies to date have been indecisive on the enhancing effect of cooperative communication on INS in the frontopolar region. Some studies showed a significant relationship between communication and
Conclusions
Research of INS using fNIRS hyperscanning is a nascent field that calls for accumulation of findings and establishment of analytic standards. In this study, we conducted an fNIRS hyperscanning study while groups of four young adults were engaged in cooperative verbal communication. Our results (1) confirm that frontopolar INS is enhanced by unstructured communication, as expected from the region's roles in communication; (2) revealed the existence of timescale dependency in the INS modulation
Acknowledgments
We thank the staff of Hitachi Ltd. for the collaborative development of the fNIRS device, as well as the participants and all our colleagues in IDAC, Tohoku University for their support. This study was partially supported by System Development Program for Advanced Measurement and Analysis, Japan Science and Technology Agency (JST; http://www.jst.go.jp/), and also by MEXT/JSPS (http://www.jsps.go.jp/) KAKENHI Grant Numbers 23700137 and 15H01771.
References (86)
- et al.
Social neuroscience and hyperscanning techniques: past, present and future
Neurosci. Biobehav. Rev.
(2014) - et al.
Separating respiratory-variation-related fluctuations from neuronal-activity-related fluctuations in fMRI
NeuroImage
(2006) - et al.
The respiration response function: the temporal dynamics of fMRI signal fluctuations related to changes in respiration
NeuroImage
(2008) - et al.
Motion artifacts in functional near-infrared spectroscopy: a comparison of motion correction techniques applied to real cognitive data
NeuroImage
(2014) - et al.
Time-frequency dynamics of resting-state brain connectivity measured with fMRI
NeuroImage
(2010) - et al.
Bold fractional contribution to resting-state functional connectivity above 0.1 hz
NeuroImage
(2015) - et al.
Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics
NeuroImage
(2010) - et al.
NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation
NeuroImage
(2012) - et al.
Between-brain coherence during joint n-back task performance: a two-person functional near-infrared spectroscopy study
Behav. Brain Res.
(2012) - et al.
Quantitative evaluation of deep and shallow tissue layers' contribution to fNIRS signal using multi-distance optodes and independent component analysis
NeuroImage
(2014)