Abstract
We often perceive real-life objects as multisensory cues through space and time. A key challenge for audiovisual integration is to match neural signals that not only originate from different sensory modalities but also that typically reach the observer at slightly different times. In humans, complex, unpredictable audiovisual streams lead to higher levels of perceptual coherence than predictable, rhythmic streams. In addition, perceptual coherence for complex signals seems less affected by increased asynchrony between visual and auditory modalities than for simple signals. Here, we used functional magnetic resonance imaging to determine the human neural correlates of audiovisual signals with different levels of temporal complexity and synchrony. Our study demonstrated that greater perceptual asynchrony and lower signal complexity impaired performance in an audiovisual coherence-matching task. Differences in asynchrony and complexity were also underpinned by a partially different set of brain regions. In particular, our results suggest that, while regions in the dorsolateral prefrontal cortex were modulated by differences in memory load due to stimulus asynchrony, areas traditionally thought to be involved in speech production and recognition, such as the inferior frontal and superior temporal cortex, were modulated by the temporal complexity of the audiovisual signals. Our results, therefore, indicate specific processing roles for different subregions of the fronto-temporal cortex during audiovisual coherence detection.
Significance Statement The brain’s capability to rapidly integrate signals from different sensory sources lies at the very heart of understanding neural information processing. The temporal structure of sensory events is an important sensory property for the identification of perceptual correspondence. The brain processes underlying their analysis, however, remain unclear. This study shows that modulating the temporal complexity and synchrony of audiovisual streams independently affects neural activity in two distinct sets of brain regions. Our study provides novel neuroanatomical information critical for understanding the mechanisms underlying multisensory integration in the human brain.
Footnotes
Authors report no conflict of interest.
This work was supported by the National Health and Medical Research Council (NHMRC; APP1098862) to O.B and CCG.
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
Jump to comment: