Modeling correlated noise is necessary to decode uncertainty
Introduction
What sensory stimulus evoked this particular pattern of cortical activity? This question lies at the heart of brain decoding algorithms. Most fMRI decoders will estimate or ‘decode’ a single stimulus value that is, according to some underlying model, most consistent with the observed data. In truth, however, there is rarely just one stimulus that provides a plausible explanation. Rather, different stimuli may all be somewhat consistent with the measured response. The main reason for this imprecision is variability (or noise), which causes even the same stimulus to elicit different activity patterns each time the stimulus is presented. By the same token, variability allows the same response pattern to be evoked by a range of different stimuli. From a noisy response pattern, therefore, the stimulus that elicited the pattern cannot be inferred with perfect precision. Rather, there is some degree of uncertainty in the predictions, and this uncertainty may vary from one decoded activity pattern to the next. Importantly, while uncertainty may stem from imprecise measurements, it can also be of neural origin. Since neural responses themselves are inherently noisy (Dean, 1981, Schiller et al., 1976), cortical activity cannot encode stimulus information with perfect precision, and this imprecision can, moreover, fluctuate over time. Uncertainty, thus, is an important feature of cortical stimulus representations, providing a window on the fidelity of neural stimulus processing from one moment to the next.
How can uncertainty be measured from cortical activity patterns? Mathematically speaking, the presence of uncertainty means that cortical information is most accurately characterized by a probability distribution over all possible stimuli. The wider this distribution, the larger is the range of stimuli that could have evoked the observed pattern of cortical activity, and hence, the higher is the uncertainty. In order to measure uncertainty, therefore, we should estimate probability distributions. But how can this be achieved, and why is this not possible with conventional decoding algorithms? Recall that uncertainty largely stems from noise in the data. This noise turns the causal relationship between stimuli and responses from a deterministic to a stochastic one, described by probabilities rather than fixed outcomes. To estimate probabilities, therefore, a decoding algorithm should capture this noisy, stochastic relationship. Mathematical models that describe the causal link between stimuli and cortical activity are typically known as forward or generative models, and have become a popular tool to describe (and extract information from) fMRI activity. Importantly, however, most models to date assume that noise is simply independent between fMRI voxels (e.g. Kay et al., 2008, Brouwer and Heeger, 2009, Serences et al., 2009, Jehee et al., 2012, Ester et al., 2013). Contrary to this assumption, mounting evidence suggests that variability is instead correlated in cortex (Arcaro et al., 2015, Bair et al., 2001, de Zwart et al., 2008, Henriksson et al., 2015, Parkes et al., 2005, Rosenbaum et al., 2016, Smith and Kohn, 2008, Zohary et al., 1994). It is well known that given incorrect assumptions of independence, decoding algorithms may fail to fully characterize the probability distribution encoded in cortical activity, despite producing reasonable estimates of the most likely presented stimulus (Domingos and Pazzani, 1997, Niculescu-Mizil and Caruana, 2005, Zhang, 2004).
Thus, the ability to measure stimulus distributions from cortical activity patterns hinges on having an appropriate model of the noise correlations in the data. But since the number of these correlations increases quadratically with the number of voxels in an fMRI data set, estimating them individually and without any guiding principles is often statistically impossible. Here, we propose a simpler approach, based on the notion that not all shared noise is equally important. Specifically, as others have argued before (Abbott and Dayan, 1999, Averbeck and Lee, 2006, Moreno-Bote et al., 2014, Smith and Kohn, 2008), correlated noise is most detrimental when it is indistinguishable from the stimulus-driven response. That is, when noise is correlated between similarly-tuned voxels, their joint activation can either indicate the presence of a mutually preferred stimulus, or that of shared noise. A decoder ignorant of the possibility of such correlated noise would tend to conclude that the voxels were activated by their preferred stimulus. Accordingly, we reasoned, this “naïve” decoder would incorrectly assign high probabilities to the stimuli preferred by these voxels, without considering interpretations consistent with shared noise.
To quantify these intuitions, we will first demonstrate in simulations that an accurate characterization of probability distributions is possible if, specifically, those correlations are accounted for that align with similarities in voxel tuning preferences. Correlations that do not have such tuning-dependent structure, on the other hand, may be safely ignored. We then examine noise correlations in fMRI measurements from human visual cortex, and find that these correlations contain the relevant tuning-related structure. Finally, we show that a decoding model that takes these tuning-dependent noise correlations into account provides an accurate window onto trial-by-trial fluctuations in the uncertainty in cortical stimulus representations. These findings exemplify the importance of incorporating noise correlations in forward models of neuroimaging data, and suggest a simple, statistically feasible approach to do so.
Section snippets
Participants
Eighteen healthy adult volunteers (aged 22–31 years, seven female) participated in this study. All had normal or corrected-to-normal vision, and provided written and informed consent prior to participating. The study was approved by the Radboud University Institutional Review Board.
MRI data acquisition
MRI data were acquired using a Siemens 3T Magnetom Trio scanner with an eight-channel occipital receiver coil, located at the Donders Center for Cognitive Neuroimaging. At the start of each session, a high-resolution
Results
This paper examines the relevance of shared noise to the decoding of stimulus information from cortical activity. Specifically, we will contrast two forms of shared noise: noise that is shared between voxels similarly tuned to the decoded stimulus feature, and noise that is correlated but does not depend on voxel tuning preference (i.e., it has arbitrary structure). We first provide a theoretical comparison of these two forms of noise, using simulations, before turning to an investigation of
Discussion
We have shown that accounting for shared noise is important for forward decoding models of fMRI activity – without an explicit account of shared noise in the decoder, it is difficult to go beyond a mere prediction of the most likely stimulus, and assess the degree of uncertainty in the pattern of voxel activity. Specifically, our simulations demonstrate that probability distributions (that indicate uncertainty) become inaccurate when computed using ‘naïve’ decoders that ignore noise
Acknowledgements
This work was supported by ERC Starting Grant 677601 to J.J. We thank Wei Ji Ma, Christian Beckmann and Alberto Llera for helpful discussions, Kendrick Kay and John Serences for comments on an earlier draft of this paper, and Paul Gaalman for MRI support.
References (49)
- et al.
Cortical surface-based analysis. II: inflation, flattening, and a surface-based coordinate system
Neuroimage
(1999) - et al.
Statistically optimal perception and learning: from behavior to neural representations
Trends Cogn. Sci.
(2010) - et al.
Visual representations are dominated by intrinsic fluctuations correlated between areas
Neuroimage
(2015) - et al.
Improved optimization for the robust and accurate linear registration and motion correction of brain images
Neuroimage
(2002) - et al.
The Bayesian brain: the role of uncertainty in neural coding and computation
Trends Neurosci.
(2004) - et al.
Resting-state fMRI confounds and cleanup
Neuroimage
(2013) - et al.
Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion
Neuroimage
(2012) - et al.
Variability in neural activity and behavior
Curr. Opin. Neurobiol.
(2014) - et al.
The radial bias: a different slant on visual orientation sensitivity in human and nonhuman primates
Neuron
(2006) - et al.
Short-term response variability of monkey striate neurons
Brain Res.
(1976)
Estimating the influence of attention on population codes in human visual cortex using voxel-based tuning functions
Neuroimage
The effect of correlated variability on the accuracy of a population code
Neural Comput.
Widespread correlation patterns of fMRI signal across visual cortex reflect eccentricity organization
Elife
Neural correlations, population coding and computation
Nat. Rev. Neurosci.
Effects of noise correlations on information encoding and decoding
J. Neurophysiol.
Correlated firing in macaque visual area MT: time scales and relationship to behavior
J. Neurosci.
The psychophysics toolbox
Spat. Vis.
Cross-orientation suppression in human visual cortex
J. Neurophysiol.
Decoding and reconstructing color from responses in human visual cortex
J. Neurosci.
The variability of discharge of simple cells in the cat striate cortex
Exp. Brain Res.
Reducing correlated noise in fMRI data
Magn. Reson. Med.
Mapping striate and extrastriate visual areas in human cerebral cortex
Proc. Natl. Acad. Sci.
On the optimality of the simple bayesian classifier under zero-one los
Mach. Learn
Retinotopic organization in human visual cortex and the spatial precision of functional MRI
Cereb. Cortex
Cited by (20)
The unreliable influence of multivariate noise normalization on the reliability of neural dissimilarity
2021, NeuroImageCitation Excerpt :In an exploratory analysis we investigated the covariance structure of the run-wise variance-covariance matrices derived from the GLM residuals based on the spatial distance between voxels in an ROI as well as the goodness-of-fit (GoF) of the GLMs for each voxel. Where dij is the pairwise Euclidean distance between voxels i and j. Van Bergen and Jehee (2018) also consider Eq. 11 as a predictor of noise correlations between voxels and vary parameters for the rate and starting value of the decay in order to better fit their data. For the present preliminary analysis both values were set to 1.
New advances in encoding and decoding of brain signals
2018, NeuroImageMotion direction is represented as a bimodal probability distribution in the human visual cortex
2023, Nature CommunicationsDifferent roles of response covariability and its attentional modulation in the sensory cortex and posterior parietal cortex
2023, Proceedings of the National Academy of Sciences of the United States of America