Proactive Sensing of Periodic and Aperiodic Auditory Patterns

doi:10.1016/j.tics.2018.08.003

Trends in Cognitive Sciences

Volume 22, Issue 10, October 2018, Pages 870-882

https://doi.org/10.1016/j.tics.2018.08.003 Get rights and content

Highlights

We review research on temporal predictions (particularly in auditory contexts), and consider how recent empirical evidence challenges standard oscillatory entrainment models.

Top-down phase resetting mediated by the motor system or higher-order language- or attention-related systems facilitates bottom-up perceptual processing.

A model that considers neuronal oscillations as intrinsic temporal constraints (rather than subserving a specific function) and incorporates top-down phase resetting is suggested as a parsimonious solution for both periodic and aperiodic temporal predictions.

Assuming that temporal predictions prospectively control oscillatory constraints, the proposed perspective provides a new framework for the interpretation of neurophysiological responses to auditory and speech streams.

The ability to predict when something will happen facilitates sensory processing and the ensuing computations. Building on the observation that neural activity entrains to periodic stimulation, leading neurophysiological models imply that temporal predictions rely on oscillatory entrainment. Although they provide a sufficient solution to predict periodic regularities, these models are challenged by a series of findings that question their suitability to account for temporal predictions based on aperiodic regularities. Aiming for a more comprehensive model of how the brain anticipates ‘when’ in auditory contexts, we emphasize the capacity of motor and higher-order top-down systems to prepare sensory processing in a proactive and temporally flexible manner. Focusing on speech processing, we illustrate how this framework leads to new hypotheses.

Section snippets

Temporal Predictions in Auditory Contexts

We extract temporal regularities (see Glossary) from the sensory environment to anticipate upcoming events. Although there are an infinite number of possible temporal patterns or cues that one can exploit for prediction, the periodic (isochronous, often informally referred to as rhythmic) occurrence of a sound is perhaps the first regularity that comes to mind. The human brain is captivated by periodic streams, and human listeners – as well as some other species – seem to be compelled to

Entrainment Aligns Behavior with Periodic Events

Temporal predictions have been studied in paradigms where predictions are set up in a bottom-up, stimulus-driven manner by periodic stimulation. Typically, auditory targets are detected more easily when presented on-time in an isochronous sequence than when presented early or late, or preceded by a temporally random tone sequence [17]. An already classical view, the dynamic attending theory (DAT 2, 18), proposed that attention is directed in time through the entrainment of neuronal oscillations

Top-Down Sources of Temporal Predictions

By enabling the efficient allocation of processing resources, predicting upcoming inputs reduces sensory uncertainty, thereby enhancing the processing of noisy (weak or ambiguous) inputs. To reduce both external and internal noise, the brain can arguably exploit any available source of top-down priors. Consequently, all neural systems that contribute to precise, time-related computations might inform sensory processing in a top-down manner, without being selectively dedicated to this function

A/Periodic Predictions – Speech as a Relevant Case

Temporal processing is typically studied using highly simplified stimuli (e.g., tone sequences). Although useful, this does not allow us to fully capture the essence of predictive processes, namely to enhance the analysis of complex, ecological signals. Speech constitutes an information-rich signal from which temporal regularities at many timescales can be used to facilitate processing. It provides a theoretical challenge and a valuable experimental assay to investigate the substrates and

Concluding Remarks and Future Directions

We propose a parsimonious framework for auditory temporal predictions, including for predictive processing of speech. We suggest a neuronal implementation that simultaneously makes possible bottom-up (stimulus-driven) and top-down (motor-driven, language-driven) phase-reset of low-frequency oscillations in auditory cortex. On the proposed view, neuronal oscillations do not a priori subserve specific functions but instead constitute an intrinsic temporal constraint. Top-down phase-reset signals

Acknowledgments

This work was funded by the Max-Planck-Institute for Empirical Aesthetics (J.R., D.P.) and the BrainCom Project, Horizon 2020 Framework Programme (732032) (L.H.A.). We thank Felix Bernoulli and Sarah Brendecke for graphics support.

Glossary

Active sensing: the proactive contribution of sensorimotor processing to perception. One implementation relies on top-down predictions based on internal copies of movement commands (see also corollary discharge).
Analytic attending mode: in sustained-vigilance settings, high-frequency activity maintains a continuous high-excitability state of the system to support attention.
Climbing neuronal activity (CNA): ramping anticipatory neuronal activity that increases until – and is resolved after – an

References (130)

C.E. Schroeder et al.
Low-frequency neuronal oscillations as instruments of sensory selection
Trends Neurosci.
(2009)
N. Ding
Temporal modulations in speech and music
Neurosci. Biobehav. Rev.
(2017)
A. Keitel
Auditory cortical delta-entrainment interacts with oscillatory power in multiple fronto-parietal networks
NeuroImage
(2017)
B. Morillon
Predictive motor control of sensory dynamics in auditory active sensing
Curr. Opin. Neurobiol.
(2015)
G. Mai
Delta, theta, beta, and gamma brain oscillations index levels of auditory sentence processing
NeuroImage
(2016)
H. Park
Frontal top-down signals increase coupling of auditory low-frequency oscillations to continuous speech in human listeners
Curr. Biol.
(2015)
S. Haegens et al.
Rhythmic facilitation of sensory processing: a critical review
Neurosci. Biobehav. Rev.
(2018)
J.J. Paton et al.
The neural basis of timing: distributed mechanisms for diverse functions
Neuron
(2018)
P. Fries
Rhythms for cognition: communication through coherence
Neuron
(2015)
A.S. Pillai et al.
Symmetry breaking in space-time hierarchies shapes brain dynamics and behavior
Neuron
(2017)

R. VanRullen

Perceptual cycles

Trends Cogn. Sci.

(2016)

A.-L. Giraud

Endogenous cortical rhythms determine cerebral specialization for speech perception and production

Neuron

(2007)

M.A. Goodale et al.

Separate visual pathways for perception and action

Trends Neurosci.

(1992)

R.I. Schubotz

Prediction of external events with our motor system: towards a new framework

Trends Cogn. Sci.

(2007)

J.T. Coull

A frontostriatal circuit for timing the duration of events

H. Merchant et al.

How the motor system both encodes and influences our sense of time

Time Percept. Action

(2016)

N. Cason

Bridging music and speech rhythm: rhythmic priming and audio-motor training affect speech perception

Acta Psychol. (Amst.)

(2015)

C.E. Schroeder

Dynamics of active sensing and perceptual selection

Cogn. Neurosci.

(2010)

M. Saleh

Fast and slow oscillations in human primary motor cortex predict oncoming behaviorally relevant cues

Neuron

(2010)

A. Breska et al.

Taxonomies of timing: where does the cerebellum fit in?

Curr. Opin. Behav. Sci.

(2016)

S. Nozaradan

Specific contributions of basal ganglia and cerebellum to the neural tracking of rhythm

Cortex

(2017)

A.D. Patel et al.

The evolutionary neuroscience of musical beat perception: the action simulation for auditory prediction (ASAP) hypothesis

Front. Syst. Neurosci.

(2014)

M. Aly et al.

Flexible weighting of diverse inputs makes hippocampal function malleable

Neurosci. Lett.

(2018)

M. Bonnefond et al.

Alpha oscillations serve to protect working memory maintenance against anticipated distracters

Curr. Biol.

(2012)

J. Rimmele

The effects of selective attention and speech acoustics on neural speech-tracking in a multi-talker scene

Cortex

(2015)

E.M. Zion Golumbic

Mechanisms underlying selective neuronal tracking of attended speech at a ‘cocktail party’

Neuron

(2013)

J. Costa-Faidella

Selective entrainment of brain oscillations drives auditory perceptual organization

NeuroImage

(2017)

K.B. Doelling

Acoustic landmarks drive delta–theta oscillations to enable speech comprehension by facilitating perceptual parsing

NeuroImage

(2014)

C.C. Canavier

Phase-resetting as a tool of information transmission

SI Brain Rhythms Dyn. Coord.

(2015)

W. Marslen-Wilson et al.

The temporal structure of spoken language understanding

Cognition

(1980)

G.T. Altmann et al.

Incremental interpretation at verbs: restricting the domain of subsequent reference

Cognition

(1999)

S.A. Kotz et al.

Basal ganglia contribution to rule expectancy and temporal predictability in speech

Cortex

(2015)

J.I. Skipper

The hearing ear is always found close to the speaking tongue: Review of the role of the motor system in speech perception

Brain Lang.

(2017)

H. Merchant

Finding the beat: a neural perspective across humans and non-human primates

Philos. Trans. R. Soc. B Biol. Sci.

(2015)

M.R. Jones

Time, our lost dimension: toward a new theory of perception, attention, and memory

Psychol. Rev.

(1976)

T.T. Nguyen

A latent variable model for joint pause prediction and dependency parsing

Polak, R. (in press) Non-isochronous meter is not irregular. A review of theory and evidence. In Proceedings of the...

A. Breska et al.

Neural mechanisms of rhythm-based temporal prediction: Delta phase-locking reflects temporal predictability but not rhythmic entrainment

PLoS Biol.

(2017)

B. Morillon

Temporal prediction in lieu of periodic stimulation

J. Neurosci.

(2016)

L.H. Arnal

Predicting ‘when’ using the motor system’s beta-band oscillations

Front. Hum. Neurosci.

(2012)

B. Morillon et al.

Motor origin of temporal predictions in auditory attention

Proc. Natl. Acad. Sci.

(2017)

H. Park

Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility

eLife

(2016)

J.E. Peelle

Phase-locked responses to speech in human auditory cortex are enhanced during comprehension

Cereb. Cortex

(2013)

M.J. Henry et al.

Low-frequency neural oscillations support dynamic attending in temporal context

Timing Time Percept.

(2014)

E.W. Large et al.

The dynamics of attending: how people track time-varying events

Psychol. Rev.

(1999)

T.E. Cope

Temporal predictions based on a gradual change in tempo

J. Acoust. Soc. Am.

(2012)

A.C. Nobre et al.

Anticipated moments: temporal structure in attention

Nat. Rev. Neurosci.

(2018)

J.T. Coull

Neural substrates of mounting temporal expectation

PLoS Biol.

(2009)

A. Wilsch

Slow-delta phase concentration marks improved temporal expectations based on the passage of time

Psychophysiology

(2015)

J. Obleser

What do we talk about when we talk about rhythm?

PLoS Biol.

(2017)

Cited by (148)

Syllabic rhythm and prior linguistic knowledge interact with individual differences to modulate phonological statistical learning
2024, Cognition
Phonological statistical learning - our ability to extract meaningful regularities from spoken language - is considered critical in the early stages of language acquisition, in particular for helping to identify discrete words in continuous speech. Most phonological statistical learning studies use an experimental task introduced by Saffran et al. (1996), in which the syllables forming the words to be learned are presented continuously and isochronously. This raises the question of the extent to which this purportedly powerful learning mechanism is robust to the kinds of rhythmic variability that characterize natural speech. Here, we tested participants with arhythmic, semi-rhythmic, and isochronous speech during learning. In addition, we investigated how input rhythmicity interacts with two other factors previously shown to modulate learning: prior knowledge (syllable order plausibility with respect to participants' first language) and learners’ speech auditory-motor synchronization ability. We show that words are extracted by all learners even when the speech input is completely arhythmic. Interestingly, high auditory-motor synchronization ability increases statistical learning when the speech input is temporally more predictable but only when prior knowledge can also be used. This suggests an additional mechanism for learning based on predictions not only about when but also about what upcoming speech will be.
Just a phase? Causal probing reveals spurious phasic dependence of sustained attention
2024, NeuroImage
For over a decade, electrophysiological studies have reported correlations between attention / perception and the phase of spontaneous brain oscillations. To date, these findings have been interpreted as evidence that the brain uses neural oscillations to sample and predict upcoming stimuli. Yet, evidence from simulations have shown that analysis artefacts could also lead to spurious pre-stimulus oscillations that appear to predict future brain responses. To address this discrepancy, we conducted an experiment in which visual stimuli were presented in time to specific phases of spontaneous alpha and theta oscillations. This allowed us to causally probe the role of ongoing neural activity in visual processing independent of the stimulus-evoked dynamics. Our findings did not support a causal link between spontaneous alpha / theta rhythms and behaviour. However, spurious correlations between theta phase and behaviour emerged offline using gold-standard time-frequency analyses. These findings are a reminder that care should be taken when inferring causal relationships between neural activity and behaviour using acausal analysis methods.
On natural attunement: Shared rhythms between the brain and the environment
2023, Neuroscience and Biobehavioral Reviews
Rhythms exist both in the embodied brain and the built environment. Becoming attuned to the rhythms of the environment, such as repetitive columns, can greatly affect perception. Here, we explore how the built environment affects human cognition and behavior through the concept of natural attunement, often resulting from the coordination of a person's sensory and motor systems with the rhythmic elements of the environment. We argue that the built environment should not be reduced to mere states, representations, and single variables but instead be considered a bundle of highly related continuous signals with which we can resonate. Resonance and entrainment are dynamic processes observed when intrinsic frequencies of the oscillatory brain are influenced by the oscillations of an external signal. This allows visual rhythmic stimulations of the environment to affect the brain and body through neural entrainment, cross-frequency coupling, and phase resetting. We review how real-world architectural settings can affect neural dynamics, cognitive processes, and behavior in people, suggesting the crucial role of everyday rhythms in the brain-body-environment relationship.
Null effects of temporal prediction on recognition memory but evidence for differential neural activity at encoding. A registered report
2023, Cortex
Previous research has demonstrated that rhythmic presentation of stimuli during encoding boosts subsequent recognition and is associated with distinct neural activity compared with when stimuli are presented in an arrhythmic manner. However, it is unclear whether the effect is driven by automatic entrainment to rhythm or non-rhythmic temporal prediction. This registered report presents an Electroencephalographic (EEG) study aimed at establishing the cognitive and neural mechanisms of the effect of temporal prediction on recognition. In a blocked design, stimulus onset during encoding was systematically manipulated in four conditions prior to recognition testing: rhythmic fixed (RF), rhythmic variable (RV), arrhythmic fixed (AF), and arrhythmic variable (AV). By orthogonally varying rhythm and temporal position we were able to assess their independent contributions to recognition enhancement. Our behavioural results did not replicate previous findings that show a difference in recognition memory based on temporal predictability at encoding. However, event-related potential (ERP) component analysis did show an early (N1) interaction effect of temporal position and rhythm, and later (N2 and Dm) effects driven by temporal position only. Taken together, we observed effects of temporal prediction at encoding, but these differences did not translate to later effects of memory, suggesting that effects of temporal prediction on recognition are less robust than previously thought.
The development of the relationship between auditory and visual neural sensitivity and autonomic arousal from 6 m to 12 m
2023, Developmental Cognitive Neuroscience
The differential sensitivity hypothesis argues that environmental sensitivity has the bivalent effect of predisposing individuals to both the risk-inducing and development-enhancing influences of early social environments. However, the hypothesis requires that this variation in environmental sensitivity be general across domains. In this study, we focused on neural sensitivity and autonomic arousal to test domain generality. Neural sensitivity can be assessed by correlating measures of perceptual sensitivity, as indexed by event-related potentials (ERP) in electrophysiology. The sensitivity of autonomic arousal can be tested via heart rate changes. Domain generality was tested by comparing associations in perceptual sensitivity across auditory and visual domains, and associations between sensitivity in sensory domains and heart rate. We contrasted ERP components in auditory (P3) and visual (P1, N290 and P4) detection-of-difference tasks for N = 68 infants longitudinally at 6 and 12 months of age. Domain generality should produce correlated individual differences in sensitivity across the two modalities, with higher levels of autonomic arousal associating with increased perceptual sensitivity. Having controlled for multiple comparisons, at 6 months of age, the difference in amplitude of the P3 component evoked in response to standard and deviant tones correlated with the difference in amplitude of the P1 N290 and P4 face-sensitive components evoked in response to fearful and neutral faces. However, this correlation was not found at 12 months of age. Similarly, autonomic arousal correlated with neural sensitivity at 6 months but not at 12 months. The results suggest bottom-up neural perceptual sensitivity is domain-general across auditory and visual domains and is related to autonomic arousal at 6 months but not at 12 months of age. We interpret the development of the association of these markers of ES within a neuroconstructivist framework and with respect to the concept of interactive specialisation. By 12 months of age, more experience of visual processing may have led to top-down endogenous attention mechanisms that process visual information in a way that no longer associates with automatic auditory perceptual sensitivity.
Neural entrainment underpins sensorimotor synchronization to dynamic rhythmic stimuli
2023, NeuroImage
Neural entrainment, defined as unidirectional synchronization of neural oscillations to an external rhythmic stimulus, is a topic of major interest in the field of neuroscience. Despite broad scientific consensus on its existence, on its pivotal role in sensory and motor processes, and on its fundamental definition, empirical research struggles in quantifying it with non-invasive electrophysiology. To this date, broadly adopted state-of-the-art methods still fail to capture the dynamic underlying the phenomenon.
Here, we present event-related frequency adjustment (ERFA) as a methodological framework to induce and to measure neural entrainment in human participants, optimized for multivariate EEG datasets. By applying dynamic phase and tempo perturbations to isochronous auditory metronomes during a finger-tapping task, we analyzed adaptive changes in instantaneous frequency of entrained oscillatory components during error correction. Spatial filter design allowed us to untangle, from the multivariate EEG signal, perceptual and sensorimotor oscillatory components attuned to the stimulation frequency.
Both components dynamically adjusted their frequency in response to perturbations, tracking the stimulus dynamics by slowing down and speeding up the oscillation over time. Source separation revealed that sensorimotor processing enhanced the entrained response, supporting the notion that the active engagement of the motor system plays a critical role in processing rhythmic stimuli. In the case of phase shift, motor engagement was a necessary condition to observe any response, whereas sustained tempo changes induced frequency adjustment even in the perceptual oscillatory component. Although the magnitude of the perturbations was controlled across positive and negative direction, we observed a general bias in the frequency adjustments towards positive changes, which points at the effect of intrinsic dynamics constraining neural entrainment.
We conclude that our findings provide compelling evidence for neural entrainment as mechanism underlying overt sensorimotor synchronization, and highlight that our methodology offers a paradigm and a measure for quantifying its oscillatory dynamics by means of non-invasive electrophysiology, rigorously informed by the fundamental definition of entrainment.

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

OpinionSpecial Issue: Time in the BrainProactive Sensing of Periodic and Aperiodic Auditory Patterns

Highlights

Section snippets

Temporal Predictions in Auditory Contexts

Entrainment Aligns Behavior with Periodic Events

Top-Down Sources of Temporal Predictions

A/Periodic Predictions – Speech as a Relevant Case

Concluding Remarks and Future Directions

Acknowledgments

Glossary

Trends Neurosci.

Neurosci. Biobehav. Rev.

NeuroImage

Curr. Opin. Neurobiol.

NeuroImage

Curr. Biol.

Neurosci. Biobehav. Rev.

Neuron

Neuron

Neuron

Trends Cogn. Sci.

Neuron

Trends Neurosci.

Trends Cogn. Sci.

Time Percept. Action

Acta Psychol. (Amst.)

Cogn. Neurosci.

Neuron

Curr. Opin. Behav. Sci.

Cortex

Front. Syst. Neurosci.

Neurosci. Lett.

Curr. Biol.

Cortex

Neuron

NeuroImage

NeuroImage

SI Brain Rhythms Dyn. Coord.

Cognition

Cognition

Cortex

Brain Lang.

Finding the beat: a neural perspective across humans and non-human primates

Philos. Trans. R. Soc. B Biol. Sci.

Time, our lost dimension: toward a new theory of perception, attention, and memory

Psychol. Rev.

A latent variable model for joint pause prediction and dependency parsing

Neural mechanisms of rhythm-based temporal prediction: Delta phase-locking reflects temporal predictability but not rhythmic entrainment

PLoS Biol.

Temporal prediction in lieu of periodic stimulation

J. Neurosci.

Predicting ‘when’ using the motor system’s beta-band oscillations

Front. Hum. Neurosci.

Motor origin of temporal predictions in auditory attention

Proc. Natl. Acad. Sci.

Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility

eLife

Phase-locked responses to speech in human auditory cortex are enhanced during comprehension

Cereb. Cortex

Low-frequency neural oscillations support dynamic attending in temporal context

Timing Time Percept.

The dynamics of attending: how people track time-varying events

Psychol. Rev.

Temporal predictions based on a gradual change in tempo

J. Acoust. Soc. Am.

Anticipated moments: temporal structure in attention

Nat. Rev. Neurosci.

Neural substrates of mounting temporal expectation

PLoS Biol.

Slow-delta phase concentration marks improved temporal expectations based on the passage of time

Psychophysiology

What do we talk about when we talk about rhythm?

PLoS Biol.

Opinion
Special Issue: Time in the Brain
Proactive Sensing of Periodic and Aperiodic Auditory Patterns