Processing of complex stimuli and natural scenes in the auditory cortex

doi:10.1016/j.conb.2004.06.005

Current Opinion in Neurobiology

Volume 14, Issue 4, August 2004, Pages 474-480

https://doi.org/10.1016/j.conb.2004.06.005 Get rights and content

Neuronal responses in auditory cortex show a fascinating mixture of characteristics that span the range from almost perfect copies of physical aspects of the stimuli to extremely complex context-dependent responses. Fast, highly stimulus-specific adaptation and slower plastic mechanisms work together to constantly adjust neuronal response properties to the statistics of the auditory scene. Evidence with converging implications suggests that the neuronal activity in primary auditory cortex represents sounds in terms of auditory objects rather than in terms of invariant acoustic features.

Introduction

Research into signal coding in primary auditory cortex (A1) has enjoyed renewed popularity in recent years. All modern methodologies, including new slice preparations for studying thalamo–cortical interactions [1], intracellular and extracellular single neuron recordings 2., 3.••, 4.••, evoked electrical and magnetic fields (EEG and MEG) 5., 6.•, and functional magnetic resonance imaging (fMRI) 7., 8., are being applied to this system in a variety of model animals, including rodents, bats, cats, primates, and humans.

Despite this accumulation of information, the nature of the representation of complex sounds in A1 remains the subject of heated debate. This is not due to a lack of data, but rather because of the fact that the data are often contradictory. Whereas some studies emphasize a relatively simple cortical representation, other studies show a large degree of complexity in the neuronal responses.

Here, I review evidence that indicates that simplicity and complexity co-exist in A1. Evidence with converging implications suggests that the co-existence of simplicity and complexity in A1 is due to its participation in processes that are often implicitly assigned to higher brain areas. In particular, I review evidence that suggests the involvement of auditory cortex in processes such as the on-line extraction of statistical regularities from the auditory scene and the organization of the auditory scene in terms of auditory objects.

Section snippets

Precise and imprecise temporal coding

One of the complexities in auditory cortex is the interplay among multiple time scales that determine the neural responses. For example, cortical neurons respond to some auditory events with stereotypical response bursts at a fixed latency (‘locking’). The variance of the latency of such bursts might be similar to that of peripheral neurons. However, the same neurons may show sluggish responses to other features of the sounds.

Temporal coding is usually tested using repetitive stimuli, such as

Feature detection or something else?

It seems that depending upon the circumstances, a cortical neuron can choose to be sluggish or precise, linear or non-linear. Thus, the feature sensitivity of a neuron, as determined, for example, by its STRF, cannot be used as an invariant essential characterization of its responses. The multiple time scales at which cortical neurons process sounds provide another argument against a pure role in feature-detection for auditory cortex neurons [25]. Feature detectors are expected to be sensitive

Adaptation and plasticity

The plastic capabilities of auditory cortex have been studied in several preparations on many time scales. Significant changes in electrical and magnetic brain potentials (EEG and MEG) occur during training for the performance of tasks such as the perception of virtual pitch [5] and fine pitch discrimination [37]. Even simple exposure to different auditory environments can substantially change auditory cortical organization and responses: thus, raising rats in an enriched environment increases

Auditory scene analysis in auditory cortex

Several recent studies, using a variety of techniques, suggest a role for auditory cortex in segregation and grouping of sound components. For example, at the brain potential level, Dyson and Alain [50] reported that the amplitude of the mid-latency potentials increased when a harmonic was mistuned, potentially creating two auditory objects instead of one. Furthermore, the enhanced amplitude was correlated with an increased likelihood of reporting two concurrent auditory objects. Krumbholz et al

Speculative synthesis and conclusions

Most of the interesting auditory features might already be extracted from the incoming sounds by the level of the IC, which should therefore be considered as the auditory analog of the primary visual cortex (V1). The role of auditory cortex is to organize these features into auditory objects (Figure 1). To do that, auditory cortex has to use temporal and spectral context at several time scales. The large adaptive and plastic capacity of auditory cortex is used to tune the neural circuits to the

References and recommended reading

Papers of particular interest, published within the annual period of review, have been highlighted as:

• of special interest
•• of outstanding interest

Acknowledgements

Supported by grants from the Israeli Science Foundation (ISF), the German-Israeli Foundation (GIF) and the Volkswagenstiftung.

References (63)

A. Fishbach et al.
Auditory edge detection: a neural model for physiological and psychoacoustical responses to amplitude transients
J Neurophysiol
(2001)
C.K. Machens et al.
Linearity of cortical receptive fields measured with natural sounds
J Neurosci
(2004)
O. Bar-Yosef et al.
Responses of neurons in cat primary auditory cortex to bird chirps: effects of temporal and spectral context
J Neurosci
(2002)
N.M. Weinberger
The nucleus basalis and memory codes: auditory cortical plasticity and the induction of specific, associative behavioral memory
Neurobiol Learn Mem
(2003)
M.W. Spitzer et al.
Transformation of binaural response properties in the ascending auditory pathway: influence of time-varying interaural phase disparity
J Neurophysiol
(1998)
Y.I. Fishman et al.
Neural correlates of auditory stream segregation in primary auditory cortex of the awake monkey
Hear Res
(2001)
A.S. Bregman
Auditory Scene Analysis: The Perceptual Organization of Sound
(1990)
D. Pressnitzer et al.
The lower limit of melodic pitch
J Acoust Soc Am
(2001)
J.P. Rauschecker et al.
Mechanisms and streams for processing of “what” and “where” in auditory cortex
Proc Natl Acad Sci USA
(2000)
S.J. Cruikshank et al.
Auditory thalamocortical synaptic transmission in vitro
J Neurophysiol
(2002)

Cited by (178)

Inference of network connectivity from temporally binned spike trains
2024, Journal of Neuroscience Methods
Processing neural activity to reconstruct network connectivity is a central focus of neuroscience, yet the spatiotemporal requisites of biological nervous systems are challenging for current neuronal sensing modalities. Consequently, methods that leverage limited data to successfully infer synaptic connections, predict activity at single unit resolution, and decipher their effect on whole systems, can uncover critical information about neural processing. Despite the emergence of powerful methods for inferring connectivity, network reconstruction based on temporally subsampled data remains insufficiently unexplored.
We infer synaptic weights by processing firing rates within variable time bins for a heterogeneous feed-forward network of excitatory, inhibitory, and unconnected units. We assess classification and optimize model parameters for postsynaptic spike train reconstruction. We test our method on a physiological network of leaky integrate-and-fire neurons displaying bursting patterns and assess prediction of postsynaptic activity from microelectrode array data.
Results reveal parameters for improved prediction and performance and suggest that lower resolution data and limited access to neurons can be preferred.
Recent computational methods demonstrate highly improved reconstruction of connectivity from networks of parallel spike trains by considering spike lag, time-varying firing rates, and other underlying dynamics. However, these methods insufficiently explore temporal subsampling representative of novel data types.
We provide a framework for reverse engineering neural networks from data with limited temporal quality, describing optimal parameters for each bin size, which can be further improved using non-linear methods and applied to more complicated readouts and connectivity distributions in multiple brain circuits.
Strong energy component is more important than spectral selectivity in modeling responses of midbrain auditory neurons to wide-band environmental sounds
2022, BioSystems
Modeling central auditory neurons in response to complex sounds not only helps understanding neural processing of speech signals but can also provide insights for biomimetics in neuro-engineering. While modeling responses of midbrain auditory neurons to synthetic tones is rather good, modeling those to environmental sounds is less satisfactory. Environmental sounds typically contain a wide range of frequency components, often with strong and transient energy. These stimulus features have not been examined in the conventional approach of auditory modeling centered on spectral selectivity. To this end, we firstly compared responses to an environmental sound of auditory midbrain neurons across 3 subpopulations of neurons with frequency selectivity in the low, middle and high ranges; secondly, we manipulated the sound energy, both in power and in spectrum, and compared across these subpopulations how their modeled responses were affected. The environmental sound was recorded when a rat was drinking from a feeding bottle (called the ‘drinking sound’). The sound spectrum was divided into 20 non-overlapping frequency bands (from 0 to 20 kHz, at 1 kHz width) and presented to an artificial neural model built on a committee machine with parallel spectral inputs to simulate the known tonotopic organization of the auditory system. The model was trained to predict empirical response probability profiles of neurons to the repeated sounds. Results showed that model performance depended more on the strong energy components than on the spectral selectivity. Findings were interpreted to reflect general sensitivity to rapidly changing sound intensities at the auditory midbrain and in the cortex.
Generalizable dimensions of human cortical auditory processing of speech in natural soundscapes: A data-driven ultra high field fMRI approach
2021, NeuroImage
Citation Excerpt :
However, little is currently known about the brain mechanisms that support the processing of speech in realistic acoustic environments. One reason for this is that cortical sound representations are complex, context dependent, and not well understood (Hausfeld et al., 2018; Nelken, 2004), but they appear to be of sufficient complexity to allow for the segregation of different sound sources (Elhilali and Shamma, 2008; Han et al., 2019; Khalighinejad et al., 2019; O’Sullivan et al., 2019). Here we set out to reveal and characterize brain processes in the cortical temporal lobes that support processing of speech in natural soundscapes using unsupervised machine learning methods that exploit the statistical structure of realistic sounds and distributed brain activity.
Speech comprehension in natural soundscapes rests on the ability of the auditory system to extract speech information from a complex acoustic signal with overlapping contributions from many sound sources. Here we reveal the canonical processing of speech in natural soundscapes on multiple scales by using data-driven modeling approaches to characterize sounds to analyze ultra high field fMRI recorded while participants listened to the audio soundtrack of a movie. We show that at the functional level the neuronal processing of speech in natural soundscapes can be surprisingly low dimensional in the human cortex, highlighting the functional efficiency of the auditory system for a seemingly complex task. Particularly, we find that a model comprising three functional dimensions of auditory processing in the temporal lobes is shared across participants’ fMRI activity. We further demonstrate that the three functional dimensions are implemented in anatomically overlapping networks that process different aspects of speech in natural soundscapes. One is most sensitive to complex auditory features present in speech, another to complex auditory features and fast temporal modulations, that are not specific to speech, and one codes mainly sound level. These results were derived with few a-priori assumptions and provide a detailed and computationally reproducible account of the cortical activity in the temporal lobe elicited by the processing of speech in natural soundscapes.
A cortical circuit mechanism for structural knowledge-based flexible sensorimotor decision-making
2021, Neuron
Making flexible decisions based on prior knowledge about causal environmental structures is a hallmark of goal-directed cognition in mammalian brains. Although several association brain regions, including the orbitofrontal cortex (OFC), have been implicated, the precise neuronal circuit mechanisms underlying knowledge-based decision-making remain elusive. Here, we established an inference-based auditory categorization task where mice performed within-session flexible stimulus re-categorization by inferring the changing task rules. We constructed a reinforcement learning model to recapitulate the inference-based flexible behavior and quantify the hidden variables associated with task structural knowledge. Combining two-photon population imaging and projection-specific optogenetics, we found that auditory cortex (ACx) neurons encoded the hidden task rule variable, which requires feedback input from the OFC. Silencing OFC-ACx input specifically disrupted re-categorization behavior. Direct imaging from OFC axons in the ACx revealed task state-related feedback signals, supporting the knowledge-based updating mechanism. Our data reveal a cortical circuit mechanism underlying structural knowledge-based flexible decision-making.
Early cortical processing of pitch height and the role of adaptation and musicality
2021, NeuroImage
Citation Excerpt :
The results of the processing provide the basis for the prediction of how a sound pattern will continue (Winkler et al., 2009). Predictions may refer to different stages of pitch processing, from the note-to-note level (as in the present study) to complex musical structures (Salimpoor et al., 2015); it is argued that neural networks in the brain adapt to the statistical properties of sound sequences to maintain efficient coding (Nelken, 2004; Wark et al., 2007; Yaron et al., 2012; Pérez-González and Malmierca, 2014). For example, an early MEG study by Rupp and Uppenkamp (2005) showed that the amplitude of the N1 wave (Näätänen and Picton, 1987) decreases much more in response to fixed pitch sequences than random sequences.
Pitch is an important perceptual feature; however, it is poorly understood how its cortical correlates are shaped by absolute vs relative fundamental frequency (f₀), and by neural adaptation. In this study, we assessed transient and sustained auditory evoked fields (AEFs) at the onset, progression, and offset of short pitch height sequences, taking into account the listener's musicality. We show that neuromagnetic activity reflects absolute f₀ at pitch onset and offset, and relative f₀ at transitions within pitch sequences; further, sequences with fixed f₀ lead to larger response suppression than sequences with variable f₀ contour, and to enhanced offset activity. Musical listeners exhibit stronger f₀-related AEFs and larger differences between their responses to fixed vs variable sequences, both within sequences and at pitch offset. The results resemble prominent psychoacoustic phenomena in the perception of pitch contours; moreover, they suggest a strong influence of adaptive mechanisms on cortical pitch processing which, in turn, might be modulated by a listener's musical expertise.
Sensory-to-Category Transformation via Dynamic Reorganization of Ensemble Structures in Mouse Auditory Cortex
2019, Neuron
The ability to group physical stimuli into behaviorally relevant categories is fundamental to perception and cognition. Despite a large body of work on stimulus categorization at the behavioral and cognitive levels, little is known about the underlying mechanisms at the neuronal level. Here, combining mouse auditory psychophysical behavior and in vivo two-photon imaging from the auditory cortex, we investigate how sensory-to-category transformation is implemented by cortical neurons during a stimulus categorization task. Distinct from responses during passive listening, many neurons exhibited emergent selectivity to stimuli near the category boundary during task performance, reshaping local tuning maps; other neurons became more selective to category membership of stimuli. At the population level, local cortical ensembles robustly encode category information and predict trial-by-trial decisions during task performance. Our data uncover a task-dependent dynamic reorganization of cortical response patterns serving as a neural mechanism for sensory-to-category transformation during perceptual decision-making.

View all citing articles on Scopus

View full text

Processing of complex stimuli and natural scenes in the auditory cortex

Introduction

Section snippets

Precise and imprecise temporal coding

Feature detection or something else?

Adaptation and plasticity

Auditory scene analysis in auditory cortex

Speculative synthesis and conclusions

References and recommended reading

Acknowledgements

J Neurophysiol

J Neurosci

J Neurosci

Neurobiol Learn Mem

J Neurophysiol

Hear Res

J Acoust Soc Am

Proc Natl Acad Sci USA

Auditory thalamocortical synaptic transmission in vitro

J Neurophysiol

Intracortical pathways determine breadth of subthreshold frequency receptive fields in primary auditory cortex

J Neurophysiol

Balanced inhibition underlies tuning and sharpens spike timing in auditory cortex

Nature

Topography and synaptic shaping of direction selectivity in primary auditory cortex

Nature

Music and learning-induced cortical plasticity

Ann N Y Acad Sci

Morphology of Heschl’s gyrus reflects enhanced activation in the auditory cortex of musicians

Nat Neurosci

Amplitude and frequency-modulated stimuli activate common regions of human auditory cortex

Cereb Cortex

Neural correlates of sensory and decision processes in auditory object identification

Nat Neurosci

Neural processing of amplitude-modulated sounds

Physiol Rev

Temporal and rate representations of time-varying signals in the auditory cortex of awake primates

Nat Neurosci

Information content of auditory cortical responses to time-varying acoustic stimuli

J Neurophysiol

The spectro-temporal receptive field. A functional characteristic of auditory neurons

Biol Cybern

Spectro-temporal response field characterization with dynamic ripples in ferret primary auditory cortex

J Neurophysiol

Spectrotemporal structure of receptive fields in areas AI and AAF of mouse auditory cortex

J Neurophysiol

Optimizing sound features for cortical neurons

Science

Spectrotemporal receptive fields in the lemniscal auditory thalamus and cortex

J Neurophysiol

Neural model for physiological responses to frequency and amplitude transitions uncovers topographical order in the auditory cortex

J Neurophysiol

Response timing constraints on the cortical representation of sound time structure

J Acoust Soc Am

Responses to linear and logarithmic frequency-modulated sweeps in ferret primary auditory cortex

Eur J Neurosci

Sensitivity of neurons in cat primary auditory cortex to tones and frequency-modulated stimuli. I: Effects of variation of stimulus parameters

Hear Res

Dynamics of precise spike timing in primary auditory cortex

J Neurosci

The representation of peripheral neural activity in the middle-latency evoked field of primary auditory cortex in humans(1)

Hear Res