Oscillatory mechanisms underlying the enhancement of visual motion perception by multisensory congruency

Highlights

• Auxiliary sounds can enhance visual motion perception.

• Occipital alpha band oscillations are modulated by audio–visual congruency.

• Alpha modulation predicts multisensory perceptual benefits.

• Slow oscillations are critical for multisensory processes and perception.

Abstract

Multisensory interactions shape every day perception and stimuli in one modality can enhance perception in another even when not being directly task relevant. While the underlying neural principles are slowly becoming evident, most work has focused on transient stimuli and little is known about those mechanisms underlying audio–visual motion processing. We studied the facilitation of visual motion perception by auxiliary sounds, i.e. sounds that by themselves do not provide the specific evidence required for the perceptual task at hand. In our experiment human observers became significantly better at detecting visual random dot motion when this was accompanied by auxiliary acoustic motion rather than stationary sounds. EEG measurements revealed that both auditory and visual motion modulated low frequency oscillations over the respective sensory cortices. Using single trial decoding we quantified those oscillatory signatures permitting the discrimination of visual motion similar to the subject's task. This revealed visual motion-related signatures in low (1–4 Hz) and alpha (8–12 Hz) bands that were significantly enhanced during congruent compared to disparate audio–visual conditions. Importantly, the auditory enhancement of these oscillatory signatures was predictive of the perceptual multisensory facilitation. These findings emphasise the importance of slow and alpha rhythms for perception in a multisensory context and suggest that acoustic motion can enhance visual perception by means of attention or priming-related mechanisms that are reflected in rhythmic activity over parieto-occipital regions.

Introduction

Stimuli presented to one modality can enhance our perceptual performance in detecting or discriminating stimuli in another, even without providing information about the task relevant feature. For example, a tone can improve the detection of dim lights (Chen et al., 2011, Lippert et al., 2007, McDonald et al., 2000, Noesselt et al., 2010) or facilitate the search for dynamic visual targets (Van der Burg, Olivers, Bronkhorst, & Theeuwes, 2008). Similar effects occur for other combinations of sensory modalities (Jaekl and Soto-Faraco, 2010, Lovelace et al., 2003, Thorne and Debener, 2008), can be specific to task instructions or stimulus eccentricity (Chen et al., 2011, Gleiss and Kayser, 2013, Jaekl and Soto-Faraco, 2010, Leo et al., 2008), and may relate to mechanisms based on attentional capture or stimulus-triggered changes in sensory gain (Chen et al., 2011, Lippert et al., 2007). While many studies on auxiliary multisensory benefits focus on brief or stationary stimuli, considerably less is known about the interaction of auditory and visual motion.

Previous work has shown that both moving and stationary sounds can affect the precision or quality of a visual motion percept, also when the sound is not of primary relevance for the task at hand. For example, brief sounds can alter bi-stable visual motion precepts (Sekuler, Sekuler, & Lau, 1997) and task-irrelevant acoustic motion can enhance visual motion detection (Kim, Peters, & Shams, 2012). Other work has shown that audio–visual motion features can combine for a general perceptual enhancement (Alais and Burr, 2004, Senkowski et al., 2007, Wuerger et al., 2003) or more specific feature integration (Harrison et al., 2010, Lopez-Moliner and Soto-Faraco, 2007, Meyer et al., 2005). This suggests that auditory and visual motion evidence interact in different ways and experimental contexts (see also (Cappe, Thelen, Romei, Thut, & Murray, 2012; Senkowski et al., 2007)). In search for an underlying neural substrate functional imaging studies localised brain areas mediating the integration of acoustic and visual motion evidence (Alink et al., 2008, Guipponi et al., 2013, Lewis and Noppeney, 2010, Scheef et al., 2009). However, the specific neural mechanisms underlying a sound driven enhancement of visual motion detection remain poorly understood.

To link perceptual multisensory benefits with the neural processing of visual motion we studied a paradigm in which sounds enhance the subject’s accuracy in identifying coherent visual motion without providing direct evidence that would allow solving the perceptual task based on just the acoustic evidence (Kim et al., 2012). Specifically, we used a 2-interval forced choice paradigm in which subjects had to identify which of two intervals contained coherent visual motion. The auditory stimulus was the same during both intervals and by itself did not identify the stimulus interval containing coherent visual motion. However, on half the trials the sound provided congruent motion information that could enhance visual motion processing and hence the perceptual performance.

Our hypothesis about the neural mechanisms underlying the perceptual multisensory benefit in this task was based on recent suggestions for a prominent role of oscillatory brain activity in mediating multisensory interactions (Schroeder and Lakatos, 2009, Schroeder et al., 2008). Experimental support for this has been found in different brain areas and paradigms (Bauer et al., 2009, Fiebelkorn et al., 2011, Gomez-Ramirez et al., 2011, Kayser et al., 2008, Lakatos et al., 2009, Mercier et al., 2013, Thorne et al., 2011). For example, several studies found that brief and transient sounds can shape the amplitude and phase of oscillatory activity over occipital areas (Naue et al., 2011) and that this modulation of low frequency oscillations directly relates to perceptual performance in visual tasks (Romei et al., 2012, Romei et al., 2009, Thut et al., 2012). We hence hypothesised that similar oscillatory mechanisms may underlie an auditory driven enhancement of visual motion perception. We directly tested this hypothesis using methods of single trial decoding to link stimulus related oscillatory signatures with multisensory perceptual benefits.