Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm
Highlights
► Novel paradigm combines fMRI, acoustic feature extraction and behavioral psychology. ► Timbre recruits cerebellar cognitive areas, sensory and DMN-related cortical areas. ► Rhythm and tonality recruit limbic regions, cognitive and somatomotor areas.
Introduction
Music is fundamental to humans across all cultures and is capable of eliciting intense emotions (Salimpoor et al., 2011). Uncovering the neural underpinnings of music processing has become a central theme in cognitive neuroscience in the past decade, as evidenced by the constantly increasing corpus of studies on this topic. The intrinsically multidimensional nature of music renders this task challenging. More specifically, music comprises several perceivable features of varying levels of abstraction, such as loudness, pitch (the organization of sounds along a scale from low to high), rhythm (the perceptual organization of sound events in time) and timbre (property that allows to distinguish between different instrument sounds having the same pitch and loudness). Perceiving polyphonic music involves automatic segregation of the musical information in the brain. For instance, when listening to a piece of music played by an orchestra we are able to distinguish one instrument's timbre from that of another, perceive the leading melody by extracting pitch height, and feel the beat (Bregman, 1990, Janata et al., 2002a, Janata et al., 2002b). In this process, domain-specific neural mechanisms for acoustic feature analysis and integration as well as domain-general neural circuits of attention and memory are required. In particular, hierarchical processing within the auditory cortex going from more simple to more complex features (Chevillet et al., 2011, Patterson et al., 2002), and hemispheric specialization (Samson et al., 2011, Zatorre et al., 2002) for spectral vs. temporal acoustic variations, have been identified as putative principles of functional organization of acoustic feature-related processing. Previous neuroimaging studies of music have attempted to identify brain structures involved in the perception of music-related perceptual features, such as pitch (Patterson et al., 2002), sensory dissonance (Blood et al., 1999, Koelsch et al., 2006), rhythm (Chen et al., 2008, Grahn and Rowe, 2009), timbre (Caclin et al., 2006, Halpern et al., 2004), and key (Janata et al., 2002a, Janata et al., 2002b). However, while these studies have successfully identified brain regions participating in processing of individual musical features they have relied on controlled auditory paradigms in which these features have been presented in isolation and manipulated artificially. Although a few studies have investigated brain responses during continuous listening to relatively simple musical stimuli (Janata et al., 2002a, Janata et al., 2002b, Schaefer et al., 2009), it has not been previously studied how the human brain processes, in parallel, the multitude of musical features when participants are listening to a record of real orchestra music during neuroimaging.
In the visual modality, recent evidence suggests that the brain processes visual stimuli presented in a more ecological setting differently than when presented in conventional controlled settings (Hasson et al., 2004). Assuming that this finding is generalizable across sensory modalities, one could expect that the majority of studies in the auditory modality, in which acoustic features were artificially manipulated, may have revealed an incomplete picture of brain function related to musical feature processing. Therefore, studying music listening as a continuous process using naturalistic stimuli could provide more accurate accounts of the processing of musical features in the brain.
We employed a stimulus-wise and task-wise more ecological setting in which participants freely listened to real music without performing any other task, in order to determine the neural mechanisms and structures responsible for musical feature processing under realistic conditions. To tackle the complexity of the problem, we introduced a novel interdisciplinary approach combining neuroimaging with computational acoustic feature extraction and behavioral psychology. As music stimulus we chose the modern tango Adios Nonino by Astor Piazzolla. The participants were scanned with fMRI while listening to this piece. Temporal evolutions of acoustic components representing timbral, tonal and rhythmic features of the stimulus were computationally extracted and validated via a perceptual experiment. Following this we performed correlation analyses of the time series of the individual acoustic components and the time series of the BOLD signal.
In light of previous studies (Samson et al., 2011), we hypothesized that timbral components would activate mainly sensory areas, such as the superior temporal gyrus (STG) and the Heschl's gyrus (HG). Moreover, we expected that the spectrally varying timbral components would activate particularly the caudolateral and anterior superior temporal regions, respectively (see Samson et al., 2011 for an overview). In addition, we predicted hemispheric lateralization, in particular, that right hemispheric regions would show larger areas involved in the processing of these features (Zatorre et al., 2002). Processing of tonality-related features was expected to recruit areas in the brain formerly known to be neural substrates of tonality processing, such as the rostromedial prefrontal cortex (Janata et al., 2002a, Janata et al., 2002b). In addition, as tonality processing draws on long-term knowledge of hierarchical tonality structures (Krumhansl, 1990), we expected the brain areas related to memory processing, such as the hippocampus (see Burianova et al., 2010 for an overview) to be activated. We hypothesized rhythm-related features to recruit, in addition to areas in the auditory cortex, cortical and subcortical areas related to motor processing, such as the premotor and supplementary motor areas, and subcortical structures involved in the processing of time intervals such as the basal ganglia (Harrington et al., 1998, Janata and Grafton, 2003, Rao et al., 2001, Schwartze et al., 2011). Furthermore, as tonal and rhythmic features are known to elicit expectations in listeners (Janata, 2005, Zanto et al., 2006), we hypothesized them to shape activations in the higher-order areas in the brain, such as the supplementary motor areas, which are known to be involved in perceptual tasks having an anticipatory component (Schubotz and von Cramon, 2002).
Section snippets
Participants
Eleven healthy participants (with no neurological, hearing or psychological problems) with formal musical training participated in the study (mean age: 23.2 ± 3.7 SD; 5 females). We chose participants with formal musical training as it has been shown that musicians display stronger neural responses to various musical features in comparison to non-musicians (Pantev et al., 2001, Wong et al., 2007). Five participants were educated in and performed mainly classical music, two musicians were trained
Results
Fig. 2 displays the results of the correlation analysis performed to test the consistency between the participants' fMRI responses. As can be seen, relatively large areas of the brain were found to display significant mean inter-subject correlations with the maximum found in the auditory cortices (r = .64, p < .0001). Following this, we performed correlation analysis between the fMRI data and acoustic components. As described in Perceptual experiment, perceptual validation of the acoustic
Discussion
In the present study, we investigated the neural correlates of timbral, tonal, and rhythmic feature processing of a naturalistic music stimulus. To this end we employed a novel paradigm combining neuroimaging, computational acoustic feature extraction and behavioral psychology. Participants were scanned using fMRI while they freely listened to the musical piece Adios Nonino by Astor Piazzolla. First, inter-subject consistency on a voxel-by-voxel basis was evaluated using mean inter-subject
Conclusions
To sum up, the current study introduced a new paradigm to investigate and predict the neural mechanisms related to the processing of timbral, tonal, and rhythmic features while listening to a naturalistic stimulus. A notable result of this novel naturalistic approach employed is that, in addition to corroborating findings from previous controlled settings, it revealed additional brain areas involved in music feature processing. First, cognitive areas of the cerebellum as well as sensory and
Acknowledgments
The authors wish to thank Geoff Luck for his help. This research was supported by the Academy of Finland (project numbers 7118616, 130412 and 138145) and by the aivoAALTO project of the Aalto University.
References (66)
- et al.
A common functional brain network for autobiographical, episodic, and semantic memory retrieval
Neuroimage
(2010) - et al.
To smooth or not to smooth?
Neuroimage
(2000) - et al.
Neuroanatomical dissociation between bottom-up and top-down processes of visuospatial selective attention
Neuroimage
(2006) - et al.
Behavioral and neural correlates of perceived and imagined musical timbre
Neuropsychologia
(2004) - et al.
Brain correlates of aesthetic judgment of beauty
Neuroimage
(2006) - et al.
Estimation of the probabilities of 3D clusters in functional brain images
Neuroimage
(1998) - et al.
Musical structure is processed in “language” areas of the brain: a possible role for Brodmann area 47 in temporal coherence
Neuroimage
(2003) - et al.
The processing of temporal pitch and melody information in auditory cortex
Neuron
(2002) - et al.
Structural decomposition of EEG signatures of melodic processing
Biol. Psychol.
(2009) - et al.
Predicting perceptual events activates corresponding motor schemes in lateral premotor cortex: an fMRI study
Neuroimage
(2002)
The impact of basal ganglia lesions on sensorimotor synchronization, spontaneous motor tempo, and the detection of tempo changes
Behav. Brain Res.
Investigation of low frequency drift in fMRI signal
Neuroimage
Functional topography in the human cerebellum: a meta-analysis of neuroimaging studies
Neuroimage
Structure and function of auditory cortex: music and speech
Trends Cogn. Sci.
Exploring perceptual and acoustic correlates of polyphonic timbre
Music Percept.
Effect of enculturation on the semantic and acoustic correlates of polyphonic timbre
Music Percept.
Intensely pleasurable responses to music correlates with activity in brain regions implicated in reward and emotion
Proc. Natl. Acad. Sci.
Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions
Nat. Neurosci.
Pitch perception: a physiological and individual act
Subjective appraisal of music
Ann. N. Y. Acad. Sci.
Auditory Scene Analysis: The Perceptual Organization of Sound
Stuttered and fluent speech production: an ale meta-analysis of functional neuroimaging studies
Hum. Brain Mapp.
Separate neural processing of timbre dimensions in auditory sensory memory
J. Cogn. Neurosci.
Moving on time: brain network for auditory–motor synchronization is modulated by rhythm complexity and musical training
J. Cogn. Neurosci.
Functional correlates of the anterolateral processing hierarchy in human auditory cortex
J. Neurosci.
The perception of musical spontaneity in improvised and imitated jazz performances
Front. Psychol.
Emotional processing in anterior cingulate and medial prefrontal cortex
Trends Cogn. Sci.
Statistical Methods for Research Workers
The Global Signal and Observed Anticorrelated Resting State Brain Networks
J. Neurophysiol.
Rhythm and tempo
Feeling the beat: premotor and striatal interactions in musicians and nonmusicians during beat perception
J. Neurosci.
Temporal processing in the basal ganglia
Neuropsychology
Cited by (258)
The neuroscience of music – towards ecological validity
2023, Trends in NeurosciencesNegative correlation between word-level surprisal and intersubject neural synchronization during narrative listening
2022, CortexCitation Excerpt :Thus, in this work, we evaluated if (and to what extent) word-level surprisal can explain the possible (de)synchronization emerging among participants listening to the same narrative. To this purpose, we re-analyzed the data of Russo et al. (2020) and estimated a time-resolved index of neural synchronization, i.e., the intersubject phase synchronization (ISPS) (Glerean et al., 2012) whose variance, so far, has been shown to be reflective of the degree of emotional valence and arousal of a narrative (Nummenmaa et al., 2014; Smirnov et al., 2019), physical pain and empathy (- Xu et al., 2020) and music-related features (Alluri et al., 2012; Sachs et al., 2020), but not (to the best of our knowledge) the level of predictability of a word during narrative listening. In detail, the ISPS of twenty-seven participants listening to a full-spoken narrative, once played forward (FW) and once (for control) played backward (BW) was calculated.
Encoding of melody in the human auditory cortex
2024, Science AdvancesA Qualitative Pilot Study of Adolescents’ Characteristics and Experiences Delivering a Digital, Intergenerational Music Program to Older Adults with Alzheimer’s and Dementia
2024, Journal of Intergenerational Relationships
- 1
Department of Music, University of Jyväskylä, PL 35(M), 40014 Jyväskylä, Finland.
- 2
BECS, Dept. of Biomedical Engineering and Computational Science, P.O.Box 12200, FI-00076 Aalto, Finland.
- 3
Institute of Behavioral Sciences, P.O.B. 9, 00014, University of Helsinki, Finland.