Research ArticleNew Research, Sensory and Motor Systems

The Magnitude of Trial-By-Trial Neural Variability Is Reproducible over Time and across Tasks in Humans

Ayelet Arazi, Gil Gonen-Yaacovi and Ilan Dinstein
eNeuro 11 December 2017, 4 (6) ENEURO.0292-17.2017; DOI: https://doi.org/10.1523/ENEURO.0292-17.2017

Abstract

Numerous studies have shown that neural activity in sensory cortices is remarkably variable over time and across trials even when subjects are presented with an identical repeating stimulus or task. This trial-by-trial neural variability is relatively large in the prestimulus period and considerably smaller (quenched) following stimulus presentation. Previous studies have suggested that the magnitude of neural variability affects behavior such that perceptual performance is better on trials and in individuals where variability quenching is larger. To what degree are neural variability magnitudes of individual subjects flexible or static? Here, we used EEG recordings from adult humans to demonstrate that neural variability magnitudes in visual cortex are remarkably consistent across different tasks and recording sessions. While magnitudes of neural variability differed dramatically across individual subjects, they were surprisingly stable across four tasks with different stimuli, temporal structures, and attentional/cognitive demands as well as across experimental sessions separated by one year. These experiments reveal that, in adults, neural variability magnitudes are mostly solidified individual characteristics that change little with task or time, and are likely to predispose individual subjects to exhibit distinct behavioral capabilities.

  • EEG
  • trial-by-trial variability
  • neural noise
  • variability quenching

Significance Statement

Brain activity varies dramatically from one moment to the next. Recent research has revealed that humans exhibit different magnitudes of trial-by-trial neural variability, which explain differences in their perceptual performance. How stable are neural variability magnitudes over time and across tasks? Here, subjects performed four different experiments in two experimental sessions separated by one year. The results revealed that neural variability magnitudes were remarkably consistent over time and across tasks, suggesting that the magnitude of neural variability is a solidified neural characteristic that may predispose individual subjects to exhibit different behavioral capabilities.

Introduction

Neural activity in the mammalian brain is notoriously variable/noisy over time (Vreeswijk and Sompolinsky, 1996; Faisal et al., 2008). One way of studying neural variability is by quantifying trial-by-trial variability in sensory cortices across trials containing an identical stimulus. Such studies often distinguish between variability that is apparent before stimulus presentation and variability that is apparent after stimulus presentation (Arieli et al., 1996; Churchland et al., 2010; Goris et al., 2014). Recent research has shown that prestimulus neural variability is considerably larger than poststimulus variability, thereby demonstrating that sensory stimulation reduces (“quenches”) ongoing neural variability (Churchland et al., 2010). Such variability quenching was reported consistently across studies examining a variety of cortical areas and arousal states, while using different types of stimuli, and when measuring neural activity with electrophysiology in animals (Monier et al., 2003; Finn et al., 2007; Mitchell et al., 2007; Churchland et al., 2010, 2011; Hussar and Pasternak, 2010) or neuroimaging in humans (He, 2013; He and Zempel, 2013; Schurger et al., 2015). An interesting exception is that during perceptual decision-making, neural variability sometimes increases before the decision in made (Churchland et al., 2011).

Several lines of evidence suggest that neural variability affects behavioral performance. First, larger variability quenching in sensory cortices is associated with better perceptual performance, whether examined across trials (Schurger et al., 2015) or across individual subjects (Arazi et al., 2017). Second, actively allocating attention to a visual stimulus improves behavioral performance not only by gain modulation, but also by reducing the trial-by-trial response variability of single neurons in visual cortex (Mitchell et al., 2007) and the shared/correlated variability across the local neural population (Cohen and Maunsell, 2009; Mitchell et al., 2009). Third, increasing dopamine and norepinephrine levels increases the magnitude of neural variability in animals (Aston-Jones and Cohen, 2005; Sakata et al., 2008) and generates behavior that is more exploratory (Aston-Jones and Cohen, 2005). Fourth, it has been suggested that larger ongoing cortical variability is associated with better cognitive performance (McIntosh et al., 2008; Garrett et al., 2011, 2013).

While neural variability is under the flexible control of attention and neuromodulation to a certain extent, many of the mechanisms that generate and govern neural variability are likely to be a product of individual genetics and early development. For example, mechanisms that govern the reproducibility of neural activity by maintaining stable excitation-inhibition balances (Turrigiano, 2011) and reliable synaptic transmission (Ribrault et al., 2011), are the product of individual genetics and environmental exposure during early critical periods (Hensch, 2005; Takesian and Hensch, 2013). Since individual subjects have different genetics and experience different environments, one may expect intrinsic neural variability magnitudes to differ across individuals and potentially predispose them to different behavioral capabilities.

With this in mind, we hypothesized that neural variability magnitudes of individual adult subjects are, to a large extent, a solidified neural characteristic. A prediction of this hypothesis is that individual subjects should exhibit distinct magnitudes of neural variability that would be reproducible across experiments with different stimuli, tasks, and over time. To test this, we measured neural variability in visual cortex with EEG while subjects performed four tasks that differed in their structure, stimulus, attentional demands, and cognitive requirements. The same subjects performed all four experiments in two experimental sessions separated by a year. We then quantified the neural variability magnitudes of individual subjects in each of the experiments and experimental sessions to determine their consistency across experiments and over time.

Materials and Methods

Subjects

Twenty-four subjects (eight males, mean age during the first session = 23.7 years, SD = 1.4) took part in this study. All subjects had normal or corrected-to-normal vision. The study was approved by the Ben-Gurion University Internal Review Board. Subjects provided written informed consent during both experimental sessions and were either paid for their participation or received research credit.

Experimental design

All subjects completed four experiments in each of the two experimental sessions. The gap in time between the first and the second session was 12.3 months on average (SD = 1.1). The study was performed in a dark and sound proof room. The stimuli were presented using MATLAB (Mathworks, Inc.) and Psychtoolbox (Brainard, 1997).

Checkerboard experiment

The visual stimulus consisted of a checkerboard annulus with an inner radius of 0.6° visual angle and an outer radius of 3.7° visual angle. The experiment contained 600 trials: 400 trials with the stimulus and 200 trials where the stimulus was omitted. The stimulus was presented for 50 ms and followed by a randomized intertrial interval lasting 750-1200 ms. The experiment also included an orthogonal color-detection task at fixation, which was intended to divert attention away from the checkerboard stimulus. Subjects were instructed to press a key whenever the black fixation cross changed its color to gray. The experiment contained 80 random color changes, which lasted 30 ms, and subjects had 1 s to respond. Correct and incorrect responses were indicated by changing the fixation cross to green or red, respectively.

Choice reaction time (CRT) experiment

A black triangle or a circle was presented at the center of the screen for 300 ms on each trial. Subjects were instructed to press the right or left arrow keys, respectively, as quickly as possible using their right index finger. Each trial was followed by an intertrial interval of 1200 ms. A total of 200 trials were presented, 100 trials with each of the two stimuli.

Go-no-go experiment

Stimuli and structure were identical to those described in the CRT experiment, except that participants were instructed to press the spacebar as quickly as possible with their right index finger whenever they saw a circle (“go” trial) and not when the triangle was presented (“no go” trial). A total of 300 trials were presented and 80% of the trials contained the go stimulus.

2-Back experiment

Stimuli were composed of four Chinese letters, presented at the center of the screen and participants were asked to press the “J” key whenever the current letter matched the one that was presented two trials before. Each letter was presented for 500 ms and followed by an inter trial interval of 500 ms. A total of 300 trials were presented with third of them containing a 2-back repeat.

EEG and eye tracking recordings

EEG data were recorded using a 64-channel BioSemi system (Biosemi Inc.), connected to a standard EEG cap according to the international 10-20 system. Data were referenced to the vertex electrodes. Electrooculography (EOG) was recorded using two electrodes at the outer canthi of the left and right eyes and one electrode placed below the right eye. In the Checkerboard experiment, the position of the right eye was recorded using an eye-tracker (EyeLink 1000, SR Research) at a sampling rate of 1000 Hz.

EEG preprocessing

Data were analyzed using Matlab and the EEGLAB toolbox (Delorme and Makeig, 2004). Continuous EEG data were down sampled to 512 Hz, filtered using a 1- to 40-Hz band pass filter, and rereferenced to the bilateral mastoid electrodes. EEG epochs were extracted using a time window of 700 ms (200-ms prestimulus to 500-ms poststimulus) and baseline correction was not performed so as not to alter trial-by-trial variability in the prestimulus interval. In the Checkerboard experiment, only trials where stimulus was presented were extracted, in the CRT experiment trials with both stimuli (circle or triangle) were extracted, in the go-no-go experiment only the go trials were extracted and in the 2-back experiment trials with the four different stimuli (Chinese letters) were extracted.

Epochs containing absolute amplitudes that exceeded 70 µV or where the power exceeded 25 db in the 20- to 40-Hz frequency range were identified as containing eye blinks or muscle artifacts, respectively, and were removed from further analysis. In the Checkerboard experiment identification of eye blinks was confirmed by eye tracking; trials containing horizontal or vertical eye movements that exceeded 1.5 SD of the mean were identified as trials where fixation was not maintained (i.e., trials containing saccades) and excluded from EEG analyses. Mean number of trials across subjects and sessions after trials rejection in the four experiments was 249 trials in the Checkerboard experiment (SD = 50), 146 trials in the CRT experiment (SD = 39), 161 trials in the go-no-go experiment (SD = 53), and 254 trials in the 2-back experiment (SD = 46). The mean number of trials did not differ between the first and second experimental sessions.

EEG data analysis

Trial-by-trial variability was computed for each time point in the extracted epochs (−200 to 500 ms) for each of the 64 electrodes, in each subject separately. Trials from the first and second experimental sessions were analyzed separately. Absolute level of trial-by-trial variability in the prestimulus interval was computed within a time window of −200- and 0-ms prestimulus, the variance was computed across trials for each time point and then averaged across the time points within the window. Absolute level of trial-by-trial variability in the poststimulus interval was computed in the same manner within a time window of 150- to 400-ms poststimulus.

Relative trial-by-trial variability was computed by converting the variability time courses to percentage change units relative to the mean trial-by-trial variability in the prestimulus period (−200 to 0 ms). We then estimated variability quenching for each subject in each task and session by computing the difference in variability between the prestimulus period (−200 to 0 ms) and poststimulus period (150–400 ms). We focused our analyses on the four occipital electrodes (O1, O2, PO7, and PO8) with the strongest visual responses.

To ensure that changes in variability were not driven by changes in the mean EEG activity, we also performed a control analysis, where we computed the coefficient of variation (CV) by dividing the magnitude of variability by the area under the curve of the mean ERP response (i.e., the ERP amplitude). This was computed separately for the pre (−200 to 0 ms) or post (150–400 ms) stimulus intervals. We then computed CV quenching as the relative change in the CV between the pre- and poststimulus periods (in units of percentage change). To examine the temporal dynamics of the CV, we used a sliding window with a width of 50 ms and overlap of 5 ms.

Behavioral data analysis

Mean accuracy, mean reaction time (RT), and RT variability (across trials) was computed for each subject and each session, in CRT, go-no-go, and 2-back tasks as well as the color-detection task in the Checkerboard experiment. The first 10 trials in each experiment and trials with RT below 200 ms were excluded from the analysis. Trials with incorrect responses were excluded from the RT analyses.

Statistical analysis

We assessed relationships across measures using Pearson’s correlations. The statistical significance of the correlation coefficients was assessed with a randomization test where we shuffled the labels of the subjects before computing the correlation coefficient. This procedure was performed 10,000 times while shuffling the labels across subjects randomly each time to generate a null distribution for each pair of EEG/behavioral measures. For the true correlation coefficient to be considered significant it had to exceed the 97.5th percentile or lower than the 2.5th percentile of this null distribution (i.e., equivalent to a p = 0.05 in a two-tailed t test). Comparisons across experiments/tasks were performed using a one-way ANOVA with task as the only factor, followed by post hoc Tukey’s tests when the initial result indicated significant differences. When performing correlations between neural variability and behavioral measures for each of the 64 electrodes, we used the false discovery rate (FDR) correction (Benjamini and Hochberg, 1995) to control for the multiple comparisons problem.

Electrode offset variability

The Biosemi EEG system utilizes active electrodes, which do not yield a measure of impedance. Instead, fluctuations in electrode offset (i.e., slow changes in the voltage potential over time) are considered the best indication for the quality of EEG recording (Kappenman and Luck, 2010). We, therefore, computed the electrode offset variability across trials for each subject during each of the experiments in each experimental session. We computed the offset value for each trial, the variability across trials in each of the four examined electrodes, and finally the mean across electrodes in each experiment. We then correlated offset variability with the EEG variability measures to check if differences in the quality of EEG recordings across individuals could explain our results.

Gaze variability

Gaze position was measured during the Checkerboard experiment only. We computed the distance from the fixation cross at each time point from stimulus onset to 500-ms poststimulus, then computed the standard deviation across trials for each time point, and finally averaged across all time points (0–500 ms) to generate a single measure of gaze variability per subject. We correlated gaze variability across the first and second sessions to determine if individual subjects exhibited reproducible gaze variability across sessions. Three subjects were excluded from this analysis due to difficulties in the calibration process of the eye tracker in one of sessions.

Results

Twenty-four subjects completed two experimental sessions separated by one year. Each session included four experiments that differed in their structure, stimulus, attentional demands, and cognitive loads. In the first experiment, subjects passively observed a checkerboard annulus on each trial, while their task was to identify and report infrequent color changes of the fixation cross. This enabled us to measure neural variability magnitudes to an unattended stimulus (i.e., the checkerboard). In the second experiment, subjects performed a CRT task where they responded with one button to a circle stimulus and with another button to a triangle stimulus. This enabled us to measure neural variability magnitudes to an attended stimulus during a very easy task. In the third experiment, subjects performed a go-no-go task where they responded only to the circles (go trials) and not to the triangles (no-go trials). This enabled us to measure neural variability magnitudes to an attended stimulus during a somewhat harder task that required response inhibition. In the final experiment, subjects performed a 2-back task where they were presented with alternating Chinese letters and instructed to respond whenever the current letter matched the letter that was presented two trials before. This enabled us to measure neural variability magnitudes to an attended stimulus during a difficult working memory task.

Differences in the attentional and cognitive demands of the four tasks were clearly evident in the behavioral performance of the subjects (Fig. 1). One-way ANOVA analyses demonstrated that there were clear differences in the accuracy rates (Formula ), mean RTs (Formula ), and RT variability (Formula) across the four tasks. Post hoc Tukey’s tests revealed that there were significant differences across all pairs of tasks (Formula for all behavioral measures), except for the CRT and go-no-go tasks. Specifically, accuracy rates were higher, mean RTs were lower, and RT variability was lower in the CRT and go-no-go tasks as compared with the color-detection task in the Checkerboard experiment and the 2-back task. In addition, accuracy rates were significantly higher, mean RTs were lower, and RT variability was lower in the color-detection task as compared with the 2-back task. This demonstrates that the CRT and go-no-go tasks were easier than the color-detection and 2-back tasks and that the 2-back task was harder than the color-detection task.

Figure 1.
Figure 1.

Behavioral performance measures. Mean across subjects and sessions for accuracy (A), RT (B), and RT variability (C) in each of the four tasks. Error bars: SEM across subjects. Asterisks: significant differences across experiments (post hoc Tukey’s tests, Formula ). One asterisk: significant differences between CB experiment and choice reaction time (CRT) or GNG experiments. Two asterisks: significant differences between 2B experiment and all other experiments. CB, Checkerboard; GNG, go-no-go; 2B, 2-back.

Note that in the Checkerboard experiment, the relatively difficult color-detection task diverted the subjects’ attention away from the checkerboard stimulus, thereby allowing us to quantify trial-by-trial neural variability to an unattended stimulus. In contrast, the 2-back task required that subjects attend the stimulus, thereby allowing us to quantify trial-by-trial neural variability to a strongly attended stimulus.

Neural variability quenching

We examined trial-by-trial neural variability as a function of time before and after stimulus presentation in each of the four experiments (Fig. 2). Trial-by-trial variability was reduced (i.e., quenched) following stimulus presentation in all experiments and in both recording sessions performed a year apart. Variability quenching was sustained from 150 to 400 ms after stimulus presentation and most evident in occipital electrodes (O1, O2, PO7, and PO8). We quantified variability quenching as the relative change (in units of percentage change) between prestimulus (−200 to 0 ms) and poststimulus (150–400 ms) periods, while focusing our analyses on the four electrodes noted above. Note that we filtered our EEG data using a bandpass filter (1–40 Hz; see Materials and Methods) as commonly performed in many EEG experiments. This means that the quantified trial-by-trial variability was generated by EEG activity in this frequency range and mostly dominated by lower frequencies.

Figure 2.
Figure 2.

Temporal and spatial dynamics of trial-by-trial neural variability. Each time course represents the changes in relative trial-by-trial variability (percentage-change units relative to the prestimulus period, mean across the four selected electrodes) during the first (black) or second (gray) experimental session, which were separated by one year. Each panel displays the results of a different experiment. Gray background: 150- to 400-ms poststimulus period with sustained variability quenching that was selected for further analyses. Insets, Topographic maps of variability quenching magnitudes during the 150- to 400-ms window, demonstrating that quenching was strongest in occipital electrodes across all four experiments.

Neural variability is a stable neural characteristic

We quantified three measures of trial-by-trial variability for each subject. Absolute trial-by-trial variability was quantified in the prestimulus (−200 to 0 ms) and poststimulus (150–400 ms) periods for each subject, in each of the four experiments, and each of the experimental sessions (see Materials and Methods). Variability quenching was quantified as the difference between variability magnitudes in the pre- and poststimulus periods. All three measures of variability were strongly and significantly correlated across the two EEG recording sessions in each of the four experiments (Formula ; Fig. 3). This demonstrates that the neural variability magnitudes of individual subjects barely changed over a one-year period.

Figure 3.
Figure 3.

Individual neural variability magnitudes were consistent across experimental sessions separated by one year. Scatter plots present the magnitudes of variability quenching (A), prestimulus variability (B), and poststimulus variability (C) in individual subjects during the first and second experimental sessions for each of the four experiments. The unity line is drawn for reference in each panel. Each point represents a single subject. Asterisks: significant correlation as assessed by a randomization test (Formula ). Pearson’s correlation coefficients and p values are noted in each panel.

Individual variability magnitudes were also strongly correlated across experiments despite their distinct stimuli, temporal structures, and attentional/cognitive demands. Given the strong correlations across sessions (Fig. 3), we averaged each of the variability measures across the two sessions. We then compared individual variability magnitudes across experiment pairs. This analysis revealed strong, positive, and significant correlations across all pairs of experiments when examining variability quenching (Formula ; Fig. 4), prestimulus variability (Formula ; Table 1), or poststimulus variability (Formula ; Table 1) magnitudes.

Figure 4.
Figure 4.

Individual variability quenching magnitudes were consistent across experiments. Scatter plots demonstrate the relationship between variability quenching magnitudes in each pair of experiments. Each dot represents a single subject. The linear fit is drawn for reference in each panel. Asterisks: significant correlation as assessed by a randomization test (Formula ). Pearson’s correlation coefficients and p values are noted in each panel.

View this table:
Table 1.

Individual magnitudes of prestimulus (top row) and poststimulus (bottom row) neural variability were strongly correlated across experiments

Neural variability and behavioral performance

We examined the relationship between individual measures of neural variability and behavioral performance by computing the correlations between each of the three behavioral measures (accuracy rate, RT, and RT variability) and each of the three variability measures (prestimulus variability, poststimulus variability, and variability quenching). No significant correlations were found when we separately examined data from the first (accuracy: Formula ; RT: Formula RT variability: Formula ) or second (accuracy: Formula ; RT: Formula RT variability: Formula ) experimental sessions, nor when we computed the mean behavioral/variability measures across sessions (Table 2).

View this table:
Table 2.

Relationship between measures of neural variability and behavioral measures

In an additional analysis we examined these relationships separately for each electrode and present the results as a spatial correlation map (Fig. 5). The spatial correlation maps differed across experiments, suggesting a lack of robust relationship between neural variability and behavioral measures in any of the electrodes. Furthermore, there were no significant correlations in any of the electrodes following FDR correction.

Figure 5.
Figure 5.

Scalp maps representing the correlation between measures of neural variability (quenching, prestimulus, or poststimulus) and behavioral measures: accuracy (A) or RT (B). Color bar: magnitude of Pearson’s correlation coefficients.

Signal strength and trial-by-trial variability

Previous studies have demonstrated that trial-by-trial variability is associated with the mean strength of the neural response (Churchland et al., 2010). To demonstrate that the findings described above are independent of between-subject differences in the mean EEG response amplitudes, we also performed the analysis using the CV (trial-by-trial variability divided by the mean EEG response; see Materials and Methods). As with trial-by-trial variability (Fig. 2), the CV also exhibited a strong reduction following stimulus presentation, within the same time-window, across all four experiments (Fig. 6).

Figure 6.
Figure 6.

Temporal dynamics of the CV in percentage change units relative to prestimulus period. Each panel presents results from a single experiment in the first (black) and second (gray) experimental sessions. Gray background: time window (150-400 ms) of sustained variability quenching that was selected for the previous analyses.

Most importantly, the CV magnitudes of individual subjects exhibited significant positive correlations across sessions (quenching: Formula , prestimulus: Formula , poststimulus: Formula and across most of the tasks (quenching: Formula , prestimulus: Formula , poststimulus: Formula ). Exceptions included the following: Checkerboard and CRT (prestimulus: Formula ; poststimulus: Formula ), Checkerboard and go-no-go (poststimulus: Formula ), and Checkerboard and 2-back (prestimulus: Formula ). Note that even in these cases, correlations were always positive and most were close to significant. Individual magnitudes of CV were, therefore, entirely consistent over time and mostly consistent across tasks.

Alternative sources of trial-by-trial variability

We examined whether non-neural sources of variability, such as gaze variability or the quality of EEG recordings could explain our results regarding consistency across tasks or sessions. We used eye tracking data from the Checkerboard experiment to demonstrate that individual magnitudes of neural variability quenching were not significantly correlated with gaze position variability in either the first (Formula , uncorrected) or second (Formula , uncorrected) recording session. Furthermore, rerunning our analysis after regressing out individual measures of gaze position variability revealed equivalent results to those described above (Figs. 3, 4). Specifically, variability magnitudes remained correlated over time (quenching: Formula ; prestimulus: Formula ; poststimulus: Formula ). This reassured us that the consistent magnitudes of neural variability described above were not associated with the ability of the subjects to maintain fixation.

To demonstrate that our results were not due to individual differences in the quality of the EEG recordings, we computed the electrode offset variability (see Materials and Methods). Electrode offset variability, was not significantly correlated with the magnitude of variability quenching in any of the experiments performed in either the first or second sessionFormula , uncorrected). Furthermore, rerunning our analysis after regressing out individual magnitudes of electrode offset variability revealed equivalent results to those described above (Figs. 3, 4). Specifically, variability magnitudes remained correlated over time (quenching: Formula prestimulus: Formula , poststimulus: Formula ) and across tasks (quenching: Formula , prestimulus: Formula , poststimulus: Formula ).

Discussion

Our results demonstrate that neural variability magnitudes differ across adult subjects in a consistent and reproducible manner over long periods of time and across tasks with different stimuli, structures, and attentional/cognitive demands. This was true for neural variability magnitudes in either pre- or poststimulus periods and for variability quenching magnitudes (Figs. 3, 4; Table 1). These results suggest that neural variability magnitudes in sensory cortices are mostly stable individual characteristics that are modulated to a relatively small degree by task demands. While these reliable individual differences in neural variability magnitudes were not associated with cognitive performance measures in the current study (Fig. 5), they have previously been associated with perceptual performance measures in several studies using different tasks and neuroimaging techniques (Schurger et al., 2010, 2015; Xue et al., 2010; Arazi et al., 2017). These stable neural variability magnitudes may, therefore, predispose individual subjects to exhibit distinct behavioral capabilities.

Flexibility of neural variability

To what degree is neural variability under flexible behavioral control? Previous electrophysiology studies have reported that visual attention reduces trial-by-trial response variability while improving the accuracy of behavioral performance (Mitchell et al., 2007, 2009; Cohen and Maunsell, 2009) . In contrast, raising the levels of dopamine and/or norepinephrine increases the magnitude of ongoing neural variability (Aston-Jones and Cohen, 2005; Sakata et al., 2008; Garrett et al., 2015). This may generate behavioral states associated with exploration, where the subject behaves in a more variable manner that enables learning through trial-and-error (Aston-Jones and Cohen, 2005).

While attention and neuromodulation are invaluable mechanisms for flexibly changing the magnitude of neural variability and consequent behaviors, individual differences in neural variability magnitudes are also governed by many other neurophysiological mechanisms including the noisy response characteristics of peripheral sensors (Schneeweis and Schnapf, 1999), the stochastic nature of synaptic transmission (Ribrault et al., 2011), and the dynamic changes caused by neural adaptation (Clifford et al., 2007) and synaptic plasticity (Feldman, 2009). Additional neural variability is generated by adjustments of the excitation/inhibition balance (Turrigiano, 2011) and continuous interaction and competition across large neural populations (Kelly et al., 2008). These mechanisms are likely to be the product of genetic and environmental factors that govern the development and solidification of neural circuits during critical periods (Hensch, 2005).

We believe that our results highlight the role of these later mechanisms in creating and maintaining stable individual differences in neural variability. While attention and neuromodulation are critical for adaptive behavior, we speculate that their ability to alter the neural variability magnitude of an adult individual is relatively small and limited by these static intrinsic mechanisms. Future research could further quantify the flexibility of neural variability in individual subjects and determine if some individuals are more flexible than others. Furthermore, studying measures of neural variability in young children would be particularly interesting for understanding how neural variability changes during development. Analogous behavioral research in humans (MacDonald et al., 2006) and birds (Ölveczky et al., 2011) has already shown that behavioral variability diminishes during early development and stabilizes in adulthood.

The behavioral significance of neural variability

There is ongoing debate regarding the potential behavioral significance of different measures of neural variability. On the one hand, several studies have demonstrated that smaller trial-by-trial neural variability is associated with better perceptual performance and memory encoding. For example, fMRI studies have reported that trial-by-trial variability is smaller on trials where a threshold-level stimulus is detected (Schurger et al., 2010) and on trials where a stimulus is later remembered (Xue et al., 2010) . Similarly, MEG and EEG studies have reported that neural variability quenching in sensory cortices is larger on trials where a threshold-level stimulus is detected (Schurger et al., 2015) and in individuals with lower (better) contrast discrimination thresholds (Arazi et al., 2017). Faster RTs were apparent in trials with smaller trial-by-trial variability in firing rates of macaque V4 neurons (Steinmetz and Moore, 2010), smaller trial-by-trial fMRI variability (He, 2013) and in trials where ECOG activity that was more similar to the mean response across trials (He and Zempel, 2013). Furthermore, excessive neural variability in sensory cortices has been reported in different disorders, including autism (Milne, 2011; Dinstein et al., 2012), ADHD (Gonen-Yaacovi et al., 2016), and schizophrenia (Yang et al., 2014), while electrophysiology studies have reported that neural responses are more variable in elderly animals (Turner et al., 2005; Yang et al., 2009) and humans (Anderson et al., 2012) who exhibit cognitive decline. These results are in line with signal detection theory principles, which suggest that intrinsic variability/noise reduces the detection and discrimination abilities of a perceptual system (Green and Swets, 1966).

Other studies, however, have reported that younger individuals exhibit larger fMRI time-course variability than elderly individuals (Garrett et al., 2010) and that this coincides with faster and more consistent responses when performing cognitive tasks such as perceptual matching, attentional cueing, and delayed match to sample (Garrett et al., 2013). It has been proposed that such increased ongoing variability may be beneficial for cognitive performance, because it allows for higher neural complexity and enables a neural network to flexibly switch between different states (McIntosh et al., 2008). Note that certain tasks, such as perceptual decision-making, may generate an increase in neural variability before the decision is made (Churchland et al., 2011). In our study, we note that neural variability increased following stimulus presentation in the CRT task in fronto-central electrodes (Fig. 2). The specificity of this finding to the CRT task and its potential cognitive significance remain to be determined.

In the current study, we did not find any significant relationships between measures of cognitive performance and measures of neural variability despite the use of several different tasks with distinct stimuli, structures, and attentional/cognitive demands. Note that this disappointing result was apparent across two independent experimental sessions. This suggests that the relationship between neural variability and cognitive performance is not as strong and clear as previously suggested (Garrett et al., 2013), in contrast to the relationship with perceptual performance, which has been reproduced by several labs using different neuroimaging and analysis techniques (Schurger et al., 2010, 2015; Xue et al., 2010; Arazi et al., 2017). Regardless of the precise relationship between neural variability magnitudes and behavioral performance, the main contribution of this study is in demonstrating that neural variability magnitudes differ across individuals in a reliable manner. These individual differences in neural variability are likely to constrain individual behavioral performance at some level, whether only with respect to perceptual performance or also in other behavioral domains.

Signal strength and trial-by-trial variability

Previous electrophysiology studies have demonstrated that trial-by-trial variability scales with the mean amplitude of the examined neural responses (Shadlen and Newsome, 1994). To examine trial-by-trial variability independently of the mean response most electrophysiology studies, therefore, use the Fano factor or the CV, which normalize trial-by-trial variability by the mean response (Churchland et al., 2010; Goris et al., 2014). Animal studies, however, have rarely examined the behavioral impact of neural variability magnitudes.

In contrast, human neuroimaging studies that have examined the relationship between response variability and behavior using fMRI and EEG have rarely reported CV (Garrett et al., 2013, 2015; He and Zempel, 2013; Gonen-Yaacovi et al., 2016; Arazi et al., 2017). Nevertheless, to relate our findings to both literatures, we conducted our analyses once using trial-by-trial variability measures and again using the CV measure. We found almost equivalent results in both cases, which revealed that large between-subject differences in variability magnitudes are consistent across experimental sessions and tasks even when normalizing trial-by-trial variability by the mean EEG response. Consistent differences in neural variability magnitudes across subjects are, therefore, likely to reflect differences in underlying physiologic mechanisms that are specific to the variability of neural activity rather than the strength of neural activity.

Measurement noise

Measures of trial-by-trial neural variability may be biased by subject-specific measurement noise of non-neural origin. We examined two potential sources of non-neural variability in our study: eye-gaze variability (indicative of the stability of fixation across trials) and trial-by-trial variability in electrode offset (indicative of the stability of the EEG recording). We did not find any significant correlation between electrode-offset variability or gaze-position variability and neuronal measures of variability. Furthermore, regressing out individual magnitudes of electrode offset variability or gaze position variability did not alter the results. These additional analyses demonstrate that the individual magnitudes of trial-by-trial variability were not associated with these potential sources non-neural measurement noise. With that said, additional studies examining the consistency of individual neural variability magnitudes across different neuroimaging techniques (e.g., fMRI and EEG) would be necessary for demonstrating the potential robustness of these findings across techniques with different types/sources of measurement noise.

Conclusions and Future Directions

While neural variability is to some degree under flexible control of attention and neuromodulation, our results demonstrate that individual differences in neural variability magnitudes are remarkably consistent across distinct tasks and over long periods of time. We, therefore, propose that neural variability magnitudes represent mostly stable between-subject differences in fundamental neural characteristics that were forged by genetics and environmental exposures during early development. These differences are likely to predispose individual subjects to exhibit distinct behavioral capabilities. Revealing how neural variability magnitudes develop during childhood and how they may be manipulated in adulthood are likely to be of great interest for further basic and clinical research (Dinstein et al., 2015).

Footnotes

  • The authors declare no competing financial interests.

  • This work was supported by the Israel Science Foundation Grant 961/14 (to I.D.) and by an Ministry of Immigrant Absorption fellowship (G.G.-Y.).

  • Received August 21, 2017.
  • Revision received November 20, 2017.
  • Accepted November 20, 2017.

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.

References

  1. Anderson S, Parbery-Clark A, White-Schwoch T, Kraus N (2012) Aging affects neural precision of speech encoding. J Neurosci 32:14156–14164. doi:10.1523/JNEUROSCI.2176-12.2012 pmid:23055485
  2. Arazi A, Censor N, Dinstein I (2017) Neural variability quenching predicts individual perceptual abilities. J Neurosci 37:97–109. doi:10.1523/JNEUROSCI.1671-16.2016 pmid:28053033
  3. Arieli A, Sterkin A, Grinvald A, Aertsen A (1996) Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273:1868–1871. pmid:8791593
  4. Aston-Jones G, Cohen JD (2005) An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 28:403–450. doi:10.1146/annurev.neuro.28.061604.135709 pmid:16022602
  5. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57:289–300.
  6. Brainard D (1997) The psychophysics toolbox. Spat Vis 10:433–436. pmid:9176952
  7. Churchland AK, Kiani R, Chaudhuri R, Wang XJ, Pouget A, Shadlen MN (2011) Variance as a signature of neural computations during decision making. Neuron 69:818–831. doi:10.1016/j.neuron.2010.12.037 pmid:21338889
  8. Churchland MM, Yu BM, Cunningham JP, Sugrue LP, Cohen MR, Corrado GS, Newsome WT, Clark AM, Hosseini P, Scott BB, Bradley DC, Smith MA, Kohn A, Movshon JA, Armstrong KM, Moore T, Chang SW, Snyder LH, Lisberger SG, Priebe NJ, et al. (2010) Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat Neurosci 13:369–378. doi:10.1038/nn.2501 pmid:20173745
  9. Clifford CWG, Webster MA, Stanley GB, Stocker AA, Kohn A, Sharpee TO, Schwartz O (2007) Visual adaptation: neural, psychological and computational aspects. Vision Res 47:3125–3131. doi:10.1016/j.visres.2007.08.023
  10. Cohen MR, Maunsell JHR (2009) Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci 12:1594–1600. doi:10.1038/nn.2439
  11. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9–21. doi:10.1016/j.jneumeth.2003.10.009 pmid:15102499
  12. Dinstein I, Heeger DJ, Lorenzi L, Minshew NJ, Malach R, Behrmann M (2012) Unreliable evoked responses in autism. Neuron 75:981–991. doi:10.1016/j.neuron.2012.07.026 pmid:22998867
  13. Dinstein I, Heeger DJ, Behrmann M (2015) Neural variability: friend or foe? Trends Cogn Sci 19:322–328. doi:10.1016/j.tics.2015.04.005 pmid:25979849
  14. Faisal AA, Selen LPJ, Wolpert DM (2008) Noise in the nervous system. Nat Rev Neurosci 9:292–303. doi:10.1038/nrn2258 pmid:18319728
  15. Feldman DE (2009) Synaptic mechanisms for plasticity in neocortex. Annu Rev Neurosci 32:33–55. doi:10.1146/annurev.neuro.051508.135516 pmid:19400721
  16. Finn IM, Priebe NJ, Ferster D (2007) The emergence of contrast-invariant orientation tuning in simple cells of cat visual cortex. Neuron 54:137–152. doi:10.1016/j.neuron.2007.02.029 pmid:17408583
  17. Garrett DD, Kovacevic N, McIntosh AR, Grady CL (2010) Blood oxygen level-dependent signal variability is more than just noise. J Neurosci 30:4914–4921. doi:10.1523/JNEUROSCI.5166-09.2010 pmid:20371811
  18. Garrett DD, Kovacevic N, McIntosh AR, Grady CL (2011) The importance of being variable. J Neurosci 31:4496–4503. doi:10.1523/JNEUROSCI.5641-10.2011 pmid:21430150
  19. Garrett DD, Kovacevic N, McIntosh AR, Grady CL (2013) The modulation of BOLD variability between cognitive states varies by age and processing speed. Cereb Cortex 23:684–693. doi:10.1093/cercor/bhs055 pmid:22419679
  20. Garrett DD, Nagel IE, Preuschhof C, Burzynska AZ, Marchner J, Wiegert S, Jungehülsing GJ, Nyberg L, Villringer A, Li S-C, Heekeren HR, Bäckman L, Lindenberger U (2015) Amphetamine modulates brain signal variability and working memory in younger and older adults. Proc Natl Acad Sci USA 112:7593–7598. doi:10.1073/pnas.1504090112 pmid:26034283
  21. Gonen-Yaacovi G, Arazi A, Shahar N, Karmon A, Haar S, Meiran N, Dinstein I (2016) Increased ongoing neural variability in ADHD. Cortex 81:50–63. doi:10.1016/j.cortex.2016.04.010 pmid:27179150
  22. Goris RLT, Movshon JA, Simoncelli EP (2014) Partitioning neuronal variability. Nat Neurosci 17:858–865. doi:10.1038/nn.3711 pmid:24777419
  23. Green DM, Swets JA (1966) Signal detection theory and psychophysics. New York, NY: John Wiley.
  24. He BJ (2013) Spontaneous and task-evoked brain activity negatively interact. J Neurosci 33:4672–4682. doi:10.1523/JNEUROSCI.2922-12.2013 pmid:23486941
  25. He BJ, Zempel JM (2013) Average is optimal: an inverted-U relationship between trial-to-trial brain activity and behavioral performance. PLoS Comput Biol 9:e1003348. doi:10.1371/journal.pcbi.1003348 pmid:24244146
  26. Hensch TK (2005) Critical period plasticity in local cortical circuits. Nat Rev Neurosci 6:877–888. doi:10.1038/nrn1787 pmid:16261181
  27. Hussar C, Pasternak T (2010) Trial-to-trial variability of the prefrontal neurons reveals the nature of their engagement in a motion discrimination task. Proc Natl Acad Sci USA 107:21842–21847. doi:10.1073/pnas.1009956107
  28. Kappenman ES, Luck SJ (2010) The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology 47:888–904. doi:10.1111/j.1469-8986.2010.01009.x pmid:20374541
  29. Kelly AMC, Uddin LQ, Biswal BB, Castellanos FX, Milham MP (2008) Competition between functional brain networks mediates behavioral variability. Neuroimage 39:527–537. doi:10.1016/j.neuroimage.2007.08.008
  30. MacDonald SWS, Nyberg L, Bäckman L (2006) Intra-individual variability in behavior: links to brain structure, neurotransmission and neuronal activity. Trends Neurosci 29:474–480. doi:10.1016/j.tins.2006.06.011
  31. McIntosh AR, Kovacevic N, Itier RJ (2008) Increased brain signal variability accompanies lower behavioral variability in development. PLoS Comput Biol 4:e1000106. doi:10.1371/journal.pcbi.1000106 pmid:18604265
  32. Milne E (2011) Increased intra-participant variability in children with autistic spectrum disorders: evidence from single-trial analysis of evoked EEG. Front Psychol 2:51. doi:10.3389/fpsyg.2011.00051
  33. Mitchell JF, Sundberg KA, Reynolds JH (2007) Differential attention-dependent response modulation across cell classes in macaque visual area V4. Neuron 55:131–141. doi:10.1016/j.neuron.2007.06.018 pmid:17610822
  34. Mitchell JF, Sundberg KA, Reynolds JH (2009) Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63:879–888. doi:10.1016/j.neuron.2009.09.013 pmid:19778515
  35. Monier C, Chavane F, Baudot P, Graham LJ, Frégnac Y (2003) Orientation and direction selectivity of synaptic inputs in visual cortical neurons. Neuron 37:663–680. pmid:12597863
  36. Ölveczky BP, Otchy TM, Goldberg JH, Aronov D, Fee MS (2011) Changes in the neural control of a complex motor sequence during learning. J Neurophysiol 106:386–397. doi:10.1152/jn.00018.2011 pmid:21543758
  37. Ribrault C, Sekimoto K, Triller A (2011) From the stochasticity of molecular processes to the variability of synaptic transmission. Nat Rev Neurosci 12:375–387. doi:10.1038/nrn3025 pmid:21685931
  38. Sakata JT, Hampton CM, Brainard MS (2008) Social modulation of sequence and syllable variability in adult birdsong. J Neurophysiol 99:1700–1711. doi:10.1152/jn.01296.2007 pmid:18216221
  39. Schneeweis DM, Schnapf JL (1999) The photovoltage of macaque cone photoreceptors: adaptation, noise, and kinetics. J Neurosci 19:1203–1216.
  40. Schurger A, Pereira F, Treisman A, Cohen JD (2010) Reproducibility distinguishes conscious from nonconscious neural representations. Science 327:97–99. doi:10.1126/science.1180029 pmid:19965385
  41. Schurger A, Sarigiannidis I, Naccache L, Sitt JD, Dehaene S (2015) Cortical activity is more stable when sensory stimuli are consciously perceived. Proc Natl Acad Sci USA 112:E2083–E2092. doi:10.1073/pnas.1418730112 pmid:25847997
  42. Shadlen MN, Newsome WT (1994) Noise, neural codes and cortical organization. Curr Opin Neurobiol 4:569–579. pmid:7812147
  43. Steinmetz NA, Moore T (2010) Changes in the response rate and response variability of area V4 neurons during the preparation of saccadic eye movements. J Neurophysiol 103:1171–1178. doi:10.1152/jn.00689.2009 pmid:20018834
  44. Takesian AE, Hensch TK (2013) Changing brains. Applying brain plasticity to advance and recover human ability. Amsterdam: Elsevier.
  45. Turner JG, Hughes LF, Caspary DM (2005) Affects of aging on receptive fields in rat primary auditory cortex layer V neurons. J Neurophysiol 94:2738–2747. doi:10.1152/jn.00362.2005 pmid:16000522
  46. Turrigiano G (2011) Too many cooks? Intrinsic and synaptic homeostatic mechanisms in cortical circuit refinement. Annu Rev Neurosci 34:89–103. doi:10.1146/annurev-neuro-060909-153238 pmid:21438687
  47. Vreeswijk CV, Sompolinsky H (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274:1724–1726. doi:10.1126/science.274.5293.1724
  48. Xue G, Dong Q, Chen C, Lu Z, Mumford JA, Poldrack RA (2010) Greater neural pattern similarity across repetitions is associated with better memory. Science 330:97–101. doi:10.1126/science.1193125
  49. Yang GJ, Murray JD, Repovs G, Cole MW, Savic A, Glasser MF, Pittenger C, Krystal JH, Wang X-J, Pearlson GD, Glahn DC, Anticevic A (2014) Altered global brain signal in schizophrenia. Proc Natl Acad Sci USA 111:7438–7443. doi:10.1073/pnas.1405289111 pmid:24799682
  50. Yang Y, Liang Z, Li G, Wang Y, Zhou Y (2009) Aging affects response variability of V1 and MT neurons in rhesus monkeys. Brain Res 1274:21–27. doi:10.1016/j.brainres.2009.04.015 pmid:19376092

Synthesis

Reviewing Editor: Daniela Schiller, Mount Sinai School of Medicine

Decisions are customarily a result of the Reviewing Editor and the peer reviewers coming together and discussing their recommendations until a consensus is reached. When revisions are invited, a fact-based synthesis statement explaining their decision and outlining what is needed to prepare a revision will be listed below. The following reviewer(s) agreed to reveal their identity: Biyu He, Timo van Kerkoerle

There are large differences between individuals on different neural markers like the center-frequency of different rhythms. It is an important open question whether those differences are correlated with any behavioral measure. The current manuscript nicely shows individual differences on the neural variability across a number of tasks and shows that these differences are stable across different recording sessions. Moreover, they show these differences do not explain behavioral differences across individuals. Although this null finding can be explained by the fact that the tasks are not that demanding in terms of visual detection, it is still important to report as this study provides unique data in the field of neural variability.

Major comments:

1) The difference in neural variability across tasks is problematic in a couple of ways. First of all, previous research indicated that the attended location has a reduced neural variability relative to the unattended location (Mitchell et al. 2007, 2009; Cohen & Maunsell 2009). This is something else than whether attentional load would decrease neural variability. Also, attentional load sometimes gives opposite results in visual areas as in more anterior areas, with enhanced attentional effects for easier tasks in V1 (Roelfsema & Spekreijse 2001 Neuron). Furthermore, most of the tasks are not compatible in terms of timing and the stimuli used. An alternative interpretation of the data could for example be that the pre-stimulus variability for the CB task is lowest because the stimulus generates the strongest response and the inter-trial interval is shorter than the CRT and GNG task, while the 2B task has a lower variability than the CRT and GNG task because of the shorter inter-trial interval. Please show ERP maps to be able to judge the general differences across tasks.

2) Although the null finding of the relationship with performance is interesting, it remains somewhat under-explored. A downside for the tasks used in the current study is that they are visually not very demanding, so variability in early visual areas might not be expected to contribute much to performance. To show that the task and measures used are sensitive enough to pick up a difference, it would be relevant to see the correlations so across trials within subjects, as this is what has been found in previous studies. Furthermore, it could be expected that an inverted U-shape is found instead of a linear trend, as for example in He & Zempel 2013. Finally, other measures might still show a difference, in particular the variability or quenching in more anterior regions. Furthermore, alpha power could be interesting to look at as well, as this has been shown to give a U-shaped curve with performance (Snyder, Morais, Willis & Smith, Nature Neuroscience 2015).

Specific comments to improve quality and readability of the manuscript:

1) The abstract is somewhat confusing. As neural variability is often performed using spiking activity, it seems to indicate that the study was done using single unit recordings in humans. Please specify it was done using EEG.

2) P. 3 “variability that is apparent after stimulus presentation (i.e., stimulus-evoked variability)” - the term “stimulus-evoked variability” should not be used. The stimulus does not evoke variability, it may evoke a response, or it may modulate ongoing activity. The more accurate term here is post-stimulus trial-to-trial variability.

3) P. 3. “actively allocating attention to a visual stimulus improves behavioral performance primarily by reducing the trial-by-trial response variability” - This sentence, and in particular the word “primarily” is too strong. Although reduced variability has been reported during attended condition, attention may exert its influence through gain modulation, a longer-established mechanism.

4) Line 39-41, and line 424: humans are not animals?

5) The authors band-pass filter EEG signal at 1-40 Hz, and compute trial-to-trial variability on this relatively broadband signal. Their analysis is thus dominated by the slowest frequencies, the slow cortical potentials, due to the 1/f distribution of the broadband power. This approach is very similar to that adopted in He & Zempel 2013, and it should be made clear that this is the component of the EEG signal assessed. The authors could say that they assess broadband, arrhythmic activity, and the results are likely dominated by the lowest frequencies. Otherwise, it would be very confusing about which EEG component is contributing to the finding (which frequencies? Oscillations vs. arrhythmic activity?).

6) P. 8 “Absolute level of trial-by-trial variability in the pre-stimulus interval was

computed as the mean variance within a time window of -200ms and 0ms pre

stimulus. Absolute level of trial-by-trial variability in the post-stimulus interval was

computed as the mean variance within a time window of 150-400ms post-stimulus.”

This is not clear. Was the EEG signal averaged across time within the window first, and trial-to-trial variability calculated next? Or was variance calculated across time?

7) Fig. 2. It is clear that the CRT task elicits increased variability over frontocentral regions, and this should be discussed in the right context in Introduction / Discussion. In particular, task demand doesn't always decrease across-trial variability (e.g., see Churchland...Shadlen Neuron 2011).

8) Figure 2:

- What does it mean that the variability is positive for some electrodes in the CRT task?

- The traces are the averages of the 4 electrodes?

- Colorbar is not easy to interpret. Maybe change to a colorbar with two colors and grey in the middle, centered around 0?

- Checkerboard is not well spelled.

9) Fig. 3. The units should be labeled for each axis.

10) P. 21 the paragraph starting with “most importantly” is difficult to read. The authors should slow down and unpack this section a bit. For example, in the first sentence, was CV correlated between the two sessions across subjects?

11) P. 24. “While these reliable individual differences in neural variability magnitudes were not associated with cognitive performance measures in the current study, we suggest that they are associated with perceptual performance measures as demonstrated in previous studies with multiple different tasks and neuroimaging techniques (Schurger et al., 2010, 2015; Xue et al., 2010; Arazi et al., 2017).”

This sentence seems arbitrary and ad hoc.

12) Discussion, section “flexibility of neural variability”. This section has multiple repetitions with Introduction and should be condensed.

13) Discussion, section “behavioral significance of neural variability” Some relevant references are missing:

Steinmetz & More J Neurophys 2010 (this is especially relevant to the authors' claim later that animal studies have not examined behavioral impact); He, J Neurosci 2013 (Fig. 7); He & Zempel, PLoS Comput Biol 2013;

Author Response

Dear editors and reviewers,

We would like to thank you for taking the time to consider our paper and for the positive and constructive remarks. Below is a point by point response to each of the points raised in the reviews.

Major comments:

1. The difference in neural variability across tasks is problematic in a couple of ways. First of all, previous research indicated that the attended location has a reduced neural variability relative to the unattended location (Mitchell et al. 2007, 2009; Cohen & Maunsell 2009). This is something else than whether attentional load would decrease neural variability. Also, attentional load sometimes gives opposite results in visual areas as in more anterior areas, with enhanced attentional effects for easier tasks in V1 (Roelfsema & Spekreijse 2001 Neuron). Furthermore, most of the tasks are not compatible in terms of timing and the stimuli used. An alternative interpretation of the data could for example be that the pre-stimulus variability for the CB task is lowest because the stimulus generates the strongest response and the inter-trial interval is shorter than the CRT and GNG task, while the 2B task has a lower variability than the CRT and GNG task because of the shorter inter-trial interval. Please show ERP maps to be able to judge the general differences across tasks.

We agree that it is problematic to compare neural variability magnitudes across tasks that contain different stimuli, temporal structures, and attentional/cognitive loads. Our intention was to demonstrate that all of these factors have very little effect on neural variability as measured with EEG (i.e., there were no significant differences across tasks in any of the neural variability measures). But to do this properly it is indeed necessary to disentangle the different potential generators/modulators of neural variability and examine their effect one by one. With this in mind, we are currently performing a follow-up study where we are carefully manipulating attention across trials with identical stimuli to determine the effects of attention on the neural variability magnitudes of individual subjects (will be presented in a poster at SFN).

Given the complexity of this point, we have decided to drop the section comparing neural variability across tasks and focus on the between-subject differences and their consistency across sessions and tasks.

2. Although the null finding of the relationship with performance is interesting, it remains somewhat under-explored. A downside for the tasks used in the current study is that they are visually not very demanding, so variability in early visual areas might not be expected to contribute much to performance. To show that the task and measures used are sensitive enough to pick up a difference, it would be relevant to see the correlations so across trials within subjects, as this is what has been found in previous studies. Furthermore, it could be expected that an inverted U-shape is found instead of a linear trend, as for example in He & Zempel 2013. Finally, other measures might still show a difference, in particular the variability or quenching in more anterior regions. Furthermore, alpha power could be interesting to look at as well, as this has been shown to give a U-shaped curve with performance (Snyder, Morais, Willis & Smith, Nature Neuroscience 2015).

We were indeed surprised by the lack of a relationship between neural variability measures and behavior. In a previous study utilizing a demanding perceptual task (contrast discrimination), a clear and reproducible linear relationship was found between perceptual performance and variability quenching (Arazi, J. Neurosci. 2017).

As suggested, we have explored this issue further and added a new figure which presents the correlation values between behavior (accuracy and RT) and each of the neural variability measures per electrode (Figure 5). This figure demonstrates that there were no significant relationships between behavior and neural variability that were reproducible across tasks. Furthermore, none of the correlations were significant after FDR correction for multiple comparisons.

We speculate that these null findings are indeed due to the choice of cognitive tasks. While null results may be disappointing, we believe that they are important and worthy of publication, particularly when considered in contrast to previous studies that have utilized alternative tasks and reported strong relationships with behavior.

We would prefer not to expand our analyses to other measures such as single trial amplitudes or alpha power, which are outside the scope of this study (the study focuses on between-subject differences). We agree that there are many additional measures that would be interesting to examine, but believe that the paper already contains many interesting and valuable analyses.

Specific comments to improve quality and readability of the manuscript:

1) The abstract is somewhat confusing. As neural variability is often performed using spiking activity, it seems to indicate that the study was done using single unit recordings in humans. Please specify it was done using EEG.

We have clarified that the study involved EEG recordings from adult humans.

2) P. 3 “variability that is apparent after stimulus presentation (i.e., stimulus-evoked variability)” - the term “stimulus-evoked variability” should not be used. The stimulus does not evoke variability, it may evoke a response, or it may modulate ongoing activity. The more accurate term here is post-stimulus trial-to-trial variability.

We have switched to post-stimulus throughout the paper.

3) P. 3. “actively allocating attention to a visual stimulus improves behavioral performance primarily by reducing the trial-by-trial response variability” - This sentence, and in particular the word “primarily” is too strong. Although reduced variability has been reported during attended condition, attention may exert its influence through gain modulation, a longer-established mechanism.

We have deleted the word primarily in the relevant places in the introduction and discussion and note that attention generates an increase in response gain.

4) Line 39-41, and line 424: humans are not animals?

Indeed! But will this distinction not help the readers sort out the relevant studies/citations? We are happy to eliminate this distinction if requested…

5) The authors band-pass filter EEG signal at 1-40 Hz, and compute trial-to-trial variability on this relatively broadband signal. Their analysis is thus dominated by the slowest frequencies, the slow cortical potentials, due to the 1/f distribution of the broadband power. This approach is very similar to that adopted in He & Zempel 2013, and it should be made clear that this is the component of the EEG signal assessed. The authors could say that they assess broadband, arrhythmic activity, and the results are likely dominated by the lowest frequencies. Otherwise, it would be very confusing about which EEG component is contributing to the finding (which frequencies? Oscillations vs. arrhythmic activity?).

We have added the following sentence to the results section where we describe neural variability for the first time (page 12). ”Note that we filtered our EEG data using a band-pass filter (1-40 Hz, see Methods) as commonly performed in many EEG experiments. This means that the quantified trial-by-trial variability was generated by EEG oscillations in this frequency range and mostly dominated by lower frequencies.”

6) P. 8 “Absolute level of trial-by-trial variability in the pre-stimulus interval was

computed as the mean variance within a time window of -200ms and 0ms pre

stimulus. Absolute level of trial-by-trial variability in the post-stimulus interval was

computed as the mean variance within a time window of 150-400ms post-stimulus.”

This is not clear. Was the EEG signal averaged across time within the window first, and trial-to-trial variability calculated next? Or was variance calculated across time?

We have clarified this paragraph: ”Absolute level of trial-by-trial variability in the pre-stimulus interval was computed within a time window of -200ms and 0ms pre-stimulus – the variance was computed across trials for each time-point and then averaged across the time-points within the window. Absolute level of trial-by-trial variability in the post-stimulus interval was computed in the same manner within a time window of 150-400ms post-stimulus.”

7) Fig. 2. It is clear that the CRT task elicits increased variability over frontocentral regions, and this should be discussed in the right context in Introduction / Discussion. In particular, task demand doesn't always decrease across-trial variability (e.g., see Churchland...Shadlen Neuron 2011).

We now note the Churchland paper in the introduction (p. 3) and discuss the increase in variability in the CRT task in the discussion (p. 19)

8) Figure 2:

- What does it mean that the variability is positive for some electrodes in the CRT task?

It means that trial-by-trial variability increased in the post-stimulus window (150 to 400ms) relative to the pre-stimulus window (-200 to 0ms).

- The traces are the averages of the 4 electrodes?

Yes, we have added an explanation of this in the figure legend.

- Colorbar is not easy to interpret. Maybe change to a colorbar with two colors and grey in the middle, centered around 0?

As requested we have switched to a colorbar with two colors and white in the middle.

- Checkerboard is not well spelled.

Corrected

9) Fig. 3. The units should be labeled for each axis.

Units have been added.

10) P. 21 the paragraph starting with “most importantly” is difficult to read. The authors should slow down and unpack this section a bit. For example, in the first sentence, was CV correlated between the two sessions across subjects?

Yes, we have re-written this paragraph to clarify it.

11) P. 24. “While these reliable individual differences in neural variability magnitudes were not associated with cognitive performance measures in the current study, we suggest that they are associated with perceptual performance measures as demonstrated in previous studies with multiple different tasks and neuroimaging techniques (Schurger et al., 2010, 2015; Xue et al., 2010; Arazi et al., 2017).” - This sentence seems arbitrary and ad hoc.

We are not sure what is meant by arbitrary and ad hoc. The intention of the sentence is to summarize our findings and contrast them with previous findings. We have modified the statement to clarify it: “While these reliable individual differences in neural variability magnitudes were not associated with cognitive performance measures in the current study (Figure 5), they have previously been associated with perceptual performance measures in several studies using different tasks and neuroimaging techniques (Schurger et al., 2010, 2015; Xue et al., 2010; Arazi et al., 2017). These stable neural variability magnitudes may, therefore, predispose individual subjects to exhibit distinct behavioral capabilities.”

12) Discussion, section “flexibility of neural variability”. This section has multiple repetitions with Introduction and should be condensed.

We have shortened the noted section in the discussion and minimized repetitions with the introduction.

13) Discussion, section “behavioral significance of neural variability” Some relevant references are missing:

Steinmetz & More J Neurophys 2010 (this is especially relevant to the authors' claim later that animal studies have not examined behavioral impact); He, J Neurosci 2013 (Fig. 7); He & Zempel, PLoS Comput Biol 2013;

Thank you, these studies have been added to the discussion.