Tracking cognitive fluctuations with multivoxel pattern time course (MVPTC) analysis
Graphical abstract
Introduction
Humans prioritize the processing of goal-relevant sensory information through voluntary shifts of selective attention. Several recent studies have reported that voluntary shifts of attention are associated with transient activity in the medial wall of the superior parietal lobule (mSPL; e.g., Kelley et al., 2008, Liu et al., 2003, Serences et al., 2004, Shomstein and Yantis, 2006, Shulman et al., 2009, Vandenberghe et al., 2001, Yantis et al., 2002). In contrast, sustained activity associated with maintaining attention is found in intraparietal sulcus and prefrontal cortex (e.g., Saygin and Sereno, 2008, Serences and Yantis, 2007, Silver et al., 2005). These findings suggest that mSPL plays a role in reconfiguring or shifting attention rather than in maintaining the current state of attention. Furthermore, this transient mSPL signal is also observed in non-spatial and non-perceptual acts of control, including shifts of categorization rule (Chiu & Yantis, 2009) and shifts of attention between working memory representations (Tamber-Rosenau, Esterman, Chiu, & Yantis, 2011). These studies thus implicate mSPL as a domain-independent hub for cognitive reconfiguration (Chiu and Yantis, 2009, Greenberg et al., 2010).
In addition to mSPL, other cortical and subcortical regions have been associated with cognitive reconfiguration. For example, dorsolateral prefrontal regions are often co-activated with parietal cortex, forming a dorsal frontoparietal attention control network for deployments of attention to goal-relevant sensory information (Corbetta & Shulman, 2002). Furthermore, many studies have demonstrated functional connectivity between dorsal parietal cortex and prefrontal cortex (e.g., Fox, Corbetta, Snyder, Vincent, & Raichle, 2006), and have shown that this connectivity affected behavior during spatial attention tasks (He et al., 2007, Thiebaut de Schotten et al., 2005). In addition to those cortical regions, the basal ganglia (BG) also have been implicated in shifts of spatial attention (e.g., Gitelman et al., 1999, Grande et al., 2006, Shulman et al., 2009), shifts of task set (e.g., Cools et al., 2006, Cools et al., 2004, Leber et al., 2008, Ravizza and Ivry, 2001), and updates in working memory (O’Reilly & Frank, 2006). However, functional connectivity has not been demonstrated between cortical control regions and subcortical structures (e.g., the BG) in humans during shifts of spatial attention. The current study was designed to examine the functional connectivity between mSPL and other cortical and subcortical regions using a novel multivariate technique.
One well-established approach to assessing functional connectivity first computes the time course of the mean blood oxygenation level dependent (BOLD) signal across all voxels within a seed region of interest (ROI). This time course is then entered as a univariate (voxelwise) general linear model (GLM) regressor in order to identify other voxels whose activity covaries with that of the seed region (e.g., Biswal et al., 1995, Friston et al., 1993). Another approach, beta series correlation (Rissman, Gazzaley, & D’Esposito, 2004), uses trial-by-trial beta coefficient values, instead of raw time series, within a seed region to explore correlations across the brain. More sophisticated approaches, such as psychophysiological interaction analysis (e.g., Diekhof et al., 2009, Duann et al., 2009) or dynamic causal modeling (e.g., Friston et al., 1997, Smith et al., 2006, Stephan et al., 2007), further include interaction terms among regressors to explore connectivity. Using the mean BOLD signal (or beta values) within an ROI to explore functional connectivity, however, assumes that all voxels within the ROI are all estimates of a single, common time series (thus justifying taking their average). Furthermore, this method relies on the presence of reliable activations (after correcting for multiple comparisons) in the seed regions of interest.
Here we develop an information-based functional connectivity method. In general, an information-based approach employs multivoxel pattern analysis (MVPA) to identify regions in which information expressed by spatiotemporal patterns of voxel activation reliably reflects distinct cognitive states (e.g., Chiu et al., 2011, Kamitani and Tong, 2005, Kriegeskorte et al., 2006, Norman et al., 2006, Polyn et al., 2005, Serences et al., 2009). Our novel information-based functional connectivity method exploits the multivoxel pattern of activity within a seed region by computing a continuous index of “pattern strength” as it evolves and fluctuates over time. This multivoxel pattern time course (MVPTC) then is used to identify other voxels whose activity covaries with the pattern strength within the seed region. This approach was motivated by recent studies (Esterman et al., 2009, Greenberg et al., 2010, Tamber-Rosenau et al., 2011) showing that spatial patterns of activity within mSPL reliably predicted which of several domains of cognitive control subjects were engaged in (e.g., spatial representations vs. rule representations; and visual vs. working memory representations) at each time point of a experimental run. Thus, multivoxel patterns of activity in mSPL provide a signature of cognitive state that is otherwise masked by averaging the BOLD signal across voxels within the ROI.
In the current study, subjects performed cue-evoked covert shifts of spatial attention in the task. We first identified a region of mSPL that exhibited transient increases in mean activity time-locked to shifts of attention. We then used MVPA (employing a linear support vector machine, or SVM) to model multivoxel activation patterns within mSPL associated with attention shifts and sustained attention, respectively. The resulting classifier was then applied to each time point in the entire mSPL time series, yielding a continuous information-based index (i.e., the classifier decision value) of the degree to which the mSPL pattern reflected shifting vs. sustained attention on a moment-by-moment basis. Finally, the MVPTC obtained from mSPL was entered as a regressor in a whole-brain voxelwise GLM to explore information-based functional connectivity between mSPL and the rest of the brain.
Section snippets
Subjects
Ten right-handed subjects (20–23 yrs old, four males) participated in this study. Two of the subjects failed to maintain fixation during the task and were excluded from further analysis. All subjects had normal or corrected-to-normal vision and had no history of neurological impairment. The Johns Hopkins Medicine Institutional Review Board approved the study protocol. Written informed consents were obtained from all participants.
Stimuli and procedure
Subjects were instructed to maintain fixation on a central
Behavioral results
During the RSVP task in the scanner, mean accuracy was 83.3% (SE = 2.5%). Almost all of the errors were misses (i.e., failure to respond; M = 15.2%, SE = 2.4%) rather than incorrect button-presses (M = 1.5%, SE = 0.5%), indicating that subjects rarely failed to shift attention to the appropriate RSVP stream following a shift cue (i.e., if subjects failed to shift attention to the relevant stream, they would have responded often to targets in the irrelevant stream, resulting in incorrect button-presses).
Neuroimaging results
Discussion
In this study, we replicated the finding of transient activity in mSPL associated with voluntary shifts of spatial attention (Fig. 2A; e.g., Kelley et al., 2008, Yantis et al., 2002). We then implemented a novel multivoxel pattern-based connectivity analysis method that revealed regions of the brain whose activity was correlated with fluctuations in multivoxel patterns within mSPL. These regions included prefrontal cortex and the striatum of the basal ganglia, providing converging evidence for
Acknowledgments
This work was supported by National Institutes of Health Grant R01-DA013165 to S.Y. and postdoctoral National Research Service Awards from the NEI (T32EY07143) and the NIA (AG027668-01) to L.G. We thank Dr. Amy Shelton for helpful discussions and James Gao for research assistance.
References (57)
- et al.
The inferential impact of global signal covariates in functional neuroimaging analyses
Neuroimage
(1998) - et al.
Reversal learning in Parkinson's disease depends on medication status and outcome valence
Neuropsychologia
(2006) - et al.
Psychophysiological and modulatory interactions in neuroimaging
Neuroimage
(1997) - et al.
Support vector machines for temporal classification of block design fMRI data
Neuroimage
(2005) - et al.
A method for the removal of global effects from fMRI time series
Neuroimage
(2004) - et al.
Beyond mind-reading: Multi-voxel pattern analysis of fMRI data
Trends in Cognitive Science
(2006) - et al.
Measuring functional connectivity during distinct stages of a cognitive task
Neuroimage
(2004) - et al.
Task and content modulate amygdala–hippocampal connectivity in emotional retrieval
Neuron
(2006) - et al.
Functional specificity of superior parietal mediation of spatial shifting
Neuroimage
(2001) A hybrid SVM–GLM approach for fMRI data analysis
Neuroimage
(2009)
Empirical analyses of BOLD fMRI statistics. I. Spatially unsmoothed data collected under null-hypothesis conditions
Neuroimage
Functional connectivity in the motor cortex of resting human brain using echo-planar MRI
Magnetic Resonance in Medicine
Linear systems analysis of functional magnetic resonance imaging in human VI
Journal of Neuroscience
The psychophysics toolbox
Spatial Vision
The precuneus: A review of its functional anatomy and behavioural correlates
Brain
Exploring functional connectivity of the human brain using multivariate information analysis
LIBSVM: A library for support vector machines
Decoding task-based attentional modulation during face categorization
Journal of Cognitive Neuroscience
A domain-independent source of cognitive control for task sets: Shifting spatial attention and switching categorization rules
Journal of Neuroscience
Experience sampling during fMRI reveals default network and executive system contributions to mind wandering
Proceedings of the National Academy of Sciences of the United States of America
Differential responses in human striatum and prefrontal cortex to changes in object and rule relevance
Journal of Neuroscience
Control of goal-directed and stimulus-driven attention in the brain
Nature Reviews. Neuroscience
Functional interactions guiding adaptive processing of behavioral significance
Human Brain Mapping
Functional connectivity delineates distinct roles of the inferior frontal cortex and presupplementary motor area in stop signal inhibition
Journal of Neuroscience
Decoding cognitive control in human parietal cortex
Proceedings of the National Academy of Sciences of the United States of America
Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): Use of a cluster-size threshold
Magnetic Resonance in Medicine
Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems
Proceedings of the National Academy of Sciences of the United States of America
The global signal and observed anticorrelated resting state brain networks
Journal of Neurophysiology
Cited by (11)
Testing the distributed representation hypothesis in object recognition in two open datasets
2022, Neuroscience LettersCitation Excerpt :The decision function value of classifiers serves as a good candidate for what the distributed representation information transfer to. Indeed, researchers have used the decision function value of support vector machine (SVM) as a regressor in the GLM to explore where the distributed representation patterns are transformed [4]. However, this study didn’t fully test the distributed representation hypothesis, they only used one classifier, (i.e., the linear SVM), and tested this multi-variate pattern transformation in only one dataset of two mental states (thus two categories of information).
Real-time decoding of covert attention in higher-order visual areas
2018, NeuroImageCitation Excerpt :They have also been shown to demonstrate retinotopy (Cichy et al., 2011a; Halgren et al., 1999; Kim and Biederman, 2011; Kim and Kastner, 2013; Saygin and Sereno, 2008a), as well as modulation by attention (Reddy et al., 2007; Yi et al., 2006). Parietal cortex has been suggested to have a more explicit role in top-down control, including mediating shifts of attention, control of salience maps, and object discrimination (Bressler et al., 2008; Chiu et al., 2012; Esterman et al., 2009; Gmeindl et al., 2016; Koenigs et al., 2009; Yantis et al., 2002). These regions may therefore act as sites of top-down modulation, or serve as ‘binding’ points for multiple sources of information, including object and spatial information.
Decoding individual differences in STEM learning from functional MRI data
2019, Nature CommunicationsStereoscopic processing of crossed and uncrossed disparities in the human visual cortex
2017, BMC NeuroscienceA ticket for your thoughts: Method for predicting content recall and sales using neural similarity of moviegoers
2017, Journal of Consumer Research
- 1
These authors contributed equally to this work.