Abstract
Conscious reportability of visual input is associated with a bimodal neural response in the primary visual cortex (V1): an early-latency response coupled to stimulus features and a late-latency response coupled to stimulus report or detection. This late wave of activity, central to major theories of consciousness, is thought to be driven by the prefrontal cortex (PFC), responsible for “igniting” it. Here we analyzed two electrophysiological studies in mice performing different stimulus detection tasks and characterized neural activity profiles in three key cortical regions: V1, posterior parietal cortex (PPC), and PFC. We then developed a minimal network model, constrained by known connectivity between these regions, reproducing the spatiotemporal propagation of visual- and report-related activity. Remarkably, while PFC was indeed necessary to generate report-related activity in V1, this occurred only through the mediation of PPC. PPC, and not PFC, had the final veto in enabling the report-related late wave of V1 activity.
Significance Statement
A long-standing objective in the study of consciousness is to characterize the neural signatures of perceived compared with unperceived sensory stimuli. In particular, the role of different cortical areas in generating consciousness is an active debate. Here we combined electrophysiological and modeling studies to investigate the mechanistic role of individual connections between brain regions in the generation of the activity patterns observed during conscious report. We found that, while frontal areas initiate report-related activity, the parietal cortex acts as a gate to determine whether such activity will propagate back to the visual cortex. This division of labor between prefrontal and parietal cortices is unexplained by current theoretical models of consciousness and will impact mechanistic neural models of conscious report.
Introduction
A long-standing objective in the investigation of the neural mechanisms of consciousness is to characterize the signatures of perceived versus nonperceived stimuli at the level of neurons and their interactions (Aru et al., 2012; Koch et al., 2016). A major step forward regarding this has been the discovery of a bimodal neural response in the primary visual cortex: an early-latency response coupled to stimulus presentation and a late-latency response that is only observed when agents report a stimulus (Supèr et al., 2001; Del Cul et al., 2007; van Vugt et al., 2018). Despite ongoing debates on the functional significance of this late-latency activity (Koch et al., 2016; Sergent et al., 2021; Hatamimajoumerd et al., 2022), there is agreement that it strongly correlates with explicit report across several species, e.g., primates (Supèr et al., 2001; van Vugt et al., 2018), ferrets (Yin et al., 2020), and mice (Allen et al., 2017; Steinmetz et al., 2019; Oude Lohuis et al., 2022c).
The origin of the late, report-related activity has been pinpointed to frontal areas. In several species, correlates of sensory detection behavior first originate in the prefrontal areas and only later appear in the association and sensory areas (Del Cul et al., 2007; van Vugt et al., 2018; Steinmetz et al., 2019; Yin et al., 2020). A recent study even demonstrated that activity in a secondary motor area (in mice considered to be part of the prefrontal cortex; Le Merre et al., 2021) is necessary for this late activity to emerge in mice (Allen et al., 2017). However, it is currently not understood how late, report-related activity reaches sensory regions and whether, besides originating in the prefrontal regions, it is also shaped by other cortical regions, and if so how. For instance, it is debated whether prefrontal regions directly trigger late, report-related activity in primary sensory cortices or whether this is (also) mechanistically driven by intermediate regions, such as association areas in the parietal and temporal lobes (Fahrenfort et al., 2008; Quiroga et al., 2008; Fisch et al., 2009; van Vugt et al., 2018). Addressing this question is important to better characterize how patterns of cortical activity that have been linked to conscious report are generated and propagate through the cortical regions and is consequential for arbitrating between major theories of consciousness (Seth and Bayne, 2022; COGITATE Consortium et al., 2023).
Nevertheless, it is currently unfeasible to fully dissect the circuit-level architecture underlying the origin and propagation of neural activity, and although options (chiefly optogenetics) are available to modulate individual cortical areas (Oude Lohuis et al., 2022a,b,c), this approach is unsuitable to causally manipulate individual connections between regions. Indeed, projection-specific optogenetic inactivation is only moderately effective on synaptic terminals or has relatively low temporal dynamics (Rost et al., 2022). The alternative approach of silencing the activity of feedback-projecting neurons, while achieving high efficacy and fast temporal specificity, inevitably modifies the activity of source cortical regions as well (Tervo et al., 2016; Huh et al., 2018). To overcome these limitations, we developed a minimal model of neural dynamics (Chaudhuri et al., 2015; Joglekar et al., 2018), testing the contribution of individual feedback pathways in report-related activity. Compared with previous human studies following a similar approach for studying report-related activity (Dehaene et al., 2003; Castro et al., 2020; Alilović et al., 2023; Canales-Johnson et al., 2023), we leveraged the recent availability of functional and structural data in mice (Harris et al., 2019; Steinmetz et al., 2019; Oude Lohuis et al., 2022c) to develop a computational model with anatomically faithful connectivity strengths between regions and capable of reproducing patterns of spiking activity that we observed in mice performing perceptual tasks. Both the model and neural analyses focused on mouse primary visual cortex (V1), posterior parietal cortex (PPC), and prefrontal cortex (PFC) to test to what extent PFC is necessary to generate report-related activity in V1 and what the role of association areas (e.g., PPC) is.
Materials and Methods
Experimental design
Collection and analysis of in vivo recordings
The model we developed (see below, Model description) was qualitatively fitted on data previously collected in vivo in two different sets of experiments that have been reported in, respectively, Oude Lohuis et al. (2022b,c, 2024) and Steinmetz et al. (2019). The experimental procedures performed to collect the data are only summarized here and are described more extensively in Oude Lohuis et al. (2022b,c, 2024) and Steinmetz et al. (2019). For the latter dataset, all raw data and metadata are publicly available in a published repository (see below for further details). However, this is not the case for the former dataset, as not all recordings have yet been made publicly available, and data is still only available from the authors upon reasonable request. For this reason, we will here mainly focus on describing experimental procedures related to the former set of experiments (Oude Lohuis et al., 2022b,c, 2024), so that all relevant details about yet-to-be-published experimental recordings are available.
Experimental subjects
All animal experiments followed the relevant national and institutional regulations. Experimental procedures were approved by the Dutch National Commission for Animal Experiments and by the Animal Welfare Body of the University of Amsterdam. The data presented here was collected from 17 male mice, obtained from two transgenic mouse lines: PVcre [B6;129P2-Pvalbtm1(cre)Arbr/J, RRID: IMSR_JAX:008069] and F1 offspring of this PVcre line and Ai9-TdTomato cre reporter mice [Gt(ROSA)26Sortm9(CAG-tdTomato)Hze RRID: ISMR_JAX 007909]. Mice were group-housed under a reversed day–night schedule (lights on at 20:00 and off at 8:00) and all experimental procedures were done in the dark period. Temperature was kept between 19.5 and 23.5°C, and humidity between 45 and 65%. During behavioral training (starting when mice were ∼8 week old), mice were kept under a water restriction regime. Their minimum weight was kept above 85% of their average weight between postnatal day (P) 60 and P90. Mice were normally trained 5 d/week and generally obtained all their daily liquids in the form of rewards during task performance. A supplement was delivered when the amount of liquid obtained during the task was below a minimum of 0.025 ml/g body weight per day. The same amount was provided during weekends. Mice received ad libitum food.
Surgical procedures
At the start of experimental procedures, mice were implanted with a headbar to allow head fixation in the experimental setup. Approximately 3 weeks before electrophysiological recordings, a subset of mice received an injection of an adeno-associated virus mediating the Cre-dependent expression of ChR2 in parvalbumin-positive interneurons; the injection was performed, in separate sets of mice, in either V1 or PPC. Data collected during optogenetic interventions was not utilized for the analyses presented in this study. The day before the start of extracellular recordings, small craniotomies (∼200 μm in diameter) were made over the cortical areas of interest using a dental drill. The cortical regions [V1, PPC, and anterior cingulate cortex (ACC) for this study] were identified either via stereotactic coordinates or via intrinsic optical signal imaging (Fig. 1A). Details about all surgical procedures can be found in Oude Lohuis et al. (2022b,c, 2024).
Figure 1-1
(A) Same as Fig. 1C, but including neuronal responses during false alarms (grey). Neuronal responses during error trials (licks toward the wrong side) are not shown in view of their rare occurrence. For visualization purposes, hit and miss trials are only shown for max visual change trials. (B) Same as Fig. 1E, but including neuronal responses during false alarms (grey) and during errors (black), the latter indicating responses towards the lowest-contrast visual stimulus. For visualization purposes, hit and miss trials are only shown for max visual change trials. Download Figure 1-1, TIF file.
Behavioral task and sensory stimuli
Mice were trained, over the course of several weeks, to perform an audio-visual change detection task (Fig. 1A).Visual stimuli were drifting square-wave gratings (temporal frequency, 1.5 Hz; spatial frequency, 0.08 cycles per degree; contrast, 70%; gamma-corrected), presented over the full screen (18.5 inch monitor, 60 Hz refresh rate). Gratings were continuously presented at a distance of ∼21 cm from the eyes. In a subset of trials (visual change trials), the orientation of the drifting grating was instantaneously changed. The degree of orientation change determined the visual saliency, which was set, based on the properties of the psychometric curve of individual mice, to a value corresponding to a threshold or max change (detection threshold and 90°, respectively). Mice were trained to respond to a visual change by licking to one reward port (left or right, counterbalanced across mice) and received 5–8 μl of liquid reward (infant formula) upon a correct response. Visual stimuli were the subject of analysis in the current manuscript, and a detailed description can be found in Oude Lohuis et al. (2022b,c). Correct responses to an auditory changes corresponded to licks toward the port not rewarded for visual stimuli (counterbalanced across mice). Importantly, similar neuronal responses were obtained across the measured areas irrespective of the side to which the mice had to lick upon a visual change, as well as independently of whether mice were trained to only report visual but not auditory changes. A more in-depth account can be found in Oude Lohuis et al. (2022b,c, 2024).
Multi-area recordings: acquisition and pre-processing
Extracellular recordings were performed simultaneously in 2 or 3 cortical areas (V1, PPC, ACC, and A1 were targeted in different experimental sessions). Recordings were performed on a maximum of 4 consecutive days. Several types of NeuroNexus silicon probes were used (A1 × 32-Poly2–10mm-50s-177, A2 × 16-10mm-100-500-177, A4 × 8-5mm-100-200-177, A1 × 64-Poly2-6mm-23s-160). Neurophysiological signals were preamplified, bandpass filtered (0.1 Hz–9 kHz), and acquired at 32 kHz (a bandpass filter was set between 0.1 Hz and 9 kHz) with a Digital Lynx SX 128 channel system, via the acquisition software Cheetah 5.0 (Neuralynx). Spike detection and sorting were performed using the Klusta (version 3.0.16) and Phy (version 1.0.9) software packages. For more details about acquisition and preprocessing, refer to Oude Lohuis et al. (2022b,c, 2024).
Histology
At the end of experiments, mice were perfused in 4% PFA in PBS and their brains were recovered for histological reconstruction meant to verify the correct placement of silicon probes in V1, PPC, and ACC.
Data availability
All the data used for the analyses presented in Figure 1 will be shared by the corresponding authors upon request.
This paper also analyzes existing, publicly available data from a study previously published by an independent lab (Steinmetz et al., 2019). This is available at https://doi.org/10.6084/m9.figshare.11274968.
Model description
We modeled a network of three regions, namely, V1 (primary visual cortex), PPC (posterior parietal cortex), and PFC (prefrontal cortex). Each region comprises one excitatory and one inhibitory population (Fig. 2, schematic), and the activity of each population is described by a neural mass model (Ermentrout and Cowan, 1980; Ermentrout and Terman, 2010; Bressloff, 2014; Chaudhuri et al., 2015; Joglekar et al., 2018). The model describes the evolution of the average population firing rates. Such models are macroscopic in nature; that is, they describe population activity, as opposed to single-neuron activity. Populations are connected through weighted links, which represent anatomical connectivities. Neural mass Wilson–Cowan models, such as the ones described below, are an established framework to investigate large-scale neuronal dynamics (Ermentrout and Cowan, 1980; Ermentrout and Terman, 2010; Bressloff, 2014; Chaudhuri et al., 2015; Joglekar et al., 2018).
The ith cortical area in the network evolves according to the following equations:
The local couplings are denoted by
The nonlinear function
The described equations are numerically integrated using the function ode23s in Matlab, which is based on a modified Rosenbrock formula of order 2 (Shampine and Reichelt, 1997).
Numerical parameter values
Connection strengths between areas (matrix W; Table 1) were taken from recent experimental data (Knox et al., 2019). All other model parameters (Tables 2–4) were manually calibrated to enable the excitatory nodes to reproduce patterns of activity comparable with those observed in vivo and reported earlier. As the model describes population activity, its variable
Noise in the initial conditions
We investigated the network behavior when the nominal setup is perturbed by noise, by sampling the initial conditions from a random, uniform distribution: ui(t = 0) ∈ [0, δ].
The analysis of network behavior under noise is obtained by with 50 realizations for each network setup. We fixed δ = 0.05 spikes/s, as this value was one order of magnitude smaller than the typical scales of the excitatory firing rates, and we additionally studied the effects of varying δ.
Instantaneous inactivation and connectivity morphing parameter
We performed two experiments to examine the robustness of the network behavior with respect to changes in the coupling between areas and also to infer which nodes are most relevant for the formation of the late activity bump. First, we introduce a connectivity morphing parameter which amplifies (α > 1), dampens, or suppresses (α < 1) one or more synaptic connections
In a second experiment, we consider instantaneous inactivation of certain synaptic connection: at specific time T*, we set one or more entries of the connectivity matrix
Quantification and statistical analysis
In vivo recordings: data analysis
All data analysis was performed in Matlab 2021b (MathWorks).
In vivo recordings: sensory-evoked and task-related responses
For each single neuron identified through the spike sorting procedure (V1, 594 neurons; PPC, 529 neurons; ACC, 629 neurons), we computed the average peristimulus time histogram (PSTH) aligned to the onset of visual changes, separately for hit and miss trials, as well as for small and large visual changes. PSTHs were computed with a 10 ms time bin and smoothed with a Gaussian window (standard deviation, 25 ms). Each PSTH was baseline corrected; i.e., we subtracted the average activity computed in the [−500, −10] ms window with respect to stimulus onset.
In vivo recordings: sensory-evoked and task-related responses—previously released dataset
The dataset used for the Steinmetz et al. (2019) study was downloaded from https://figshare.com/articles/dataset/Dataset_from_Steinmetz_et_al_2019/9598406 and analyzed using the same approach described above. Trials were pooled together based on whether a hit or miss was observed and separately for visual contrasts of 25, 50 and 100%.
In vivo recordings: statistical analyses
Differences between sensory-evoked responses were assessed using a permutation-based approach. For each pair of conditions to be tested (e.g., hit trials to high vs low saliency stimuli), we used the corresponding single-neuron PSTHs to compute the difference between average responses (across neurons) separately for each time bin. We then randomly swapped the trial identify of each PSTH, separately for each neuron, and computed the corresponding response difference. This was repeated 1,000 times. We then ranked, separately per time bin, the actual response difference between two conditions compared with the values obtained through random permutations. If the actual response difference was higher than 95% of the values obtained through random permutations, a difference was considered to be significant, and the corresponding p value was computed as the fraction of randomly obtained values which was higher than the actual difference. All p values were then corrected for the false discovery rate (Bonferroni’s correction). To compute if an area encoded differences between high and low saliency stimuli, we further specified that this difference had to be present for both hit and miss trials, to prevent any interaction effect. Similarly, any hit/miss difference had to be present for both low and high saliency stimuli.
Model: neural activity measure
We defined an integral measure which counts the cumulative number of spikes in V1, from time
Code accessibility
The code/software described in the paper is freely available online at https://github.com/simoneciceri/IAS_project. The code is available as Extended Data.
Extended Data 1
The file code.zip includes all code used to generate the figures and perform the analyses presented in this manuscript. The folder "./ephys analysis/" contains code and instructions concerning experimental data and Figure 1. The files "Figure2CDE.m", "Figure3AB.m" and "Figure3C.m" simulate the model in the nominal setup, and generate the corresponding figures. The file "Morphing.m" allows changing the connectivity strengths with respect to the nominal setup, and plotting the resulting trajectories. The folder "./Figure4/" contains instructions, code and data concerning Figure 4. Download Extended Data 1, ZIP file.
Results
Detection of visual stimuli is coupled to large-scale activity patterns in the dorsal cortex
We first aimed to replicate and expand earlier reports that visual detection in mice correlates with a bimodal response pattern in V1 (Oude Lohuis et al., 2022c) and with the emergence of report-related activity across multiple cortical regions (Allen et al., 2017; Pho et al., 2018; van Vugt et al., 2018; Steinmetz et al., 2019; Yin et al., 2020; Zatka-Haas et al., 2021). To this aim, we first analyzed neuronal activity collected in head-fixed mice performing an audio-visual change detection task (Oude Lohuis et al., 2022b,c, 2024; Fig. 1A). Mice were trained to report the change in the orientation of the presented visual stimulus, by performing—for instance—a left lick, and a change in the pitch of the presented auditory stimulus by performing a right lick (with contingencies counterbalanced across mice; see Materials and Methods for details). In this report we only focus on the processing of the visual stimuli. Multi-area laminar probe recordings were performed in the primary visual cortex (V1), posterior parietal cortex (PPC), and anterior cingulate cortex (ACC; Fig. 1B). We computed stimulus-evoked spiking responses across the three areas as a function of the saliency of the visual stimulus (threshold or max change) and based on whether a stimulus was detected (hit trial) or not (miss trial). Trials from the max change condition will be referred to as “high saliency” and trials from the threshold condition as “low saliency” from here on. Of relevance, previous studies indicated that neuronal responses in both PPC and V1 did not show major deviations based on whether licking responses to full-field visual stimuli had to be done toward a detector positioned toward the left or right side of a mouse's snout (Oude Lohuis et al., 2022b,c, 2024).
In V1, we observed a bimodal pattern of activity: an early-onset wave of sensory-evoked activity, lasting until ∼200 ms after stimulus onset, followed by a late-onset wave which was mainly encoding whether a trial was a hit or miss (cf. Oude Lohuis et al., 2022c; Fig. 1C). Early sensory-evoked activity did not differ between hit and miss trials, but firing rates were positively correlated with the saliency of visual stimuli (Fig. 1C). Instead, late activity encoded both whether a trial was a hit or miss, as well as whether the sensory input was strong or weak (Fig. 1C; cf. Oude Lohuis et al., 2022c). Activity in PPC and ACC mainly encoded differences between hit and miss trials, although a generalized increase in firing rates could be observed as a consequence of the presentation of sensory stimuli (Fig. 1C; cf. Oude Lohuis et al., 2022b).
To verify that these results were not specific to our experimental protocol, we also analyzed recordings from another previously published experiment (Steinmetz et al., 2019). In this paradigm, mice had to identify which of two visual stimuli presented in the right and left hemifield had the highest contrast and rotate a wheel to move the highest-contrast sensory stimulus toward the center of the screen (Fig. 1D). We computed sensory-evoked responses as a function of both stimulus contrast (difference between the contrast of the two presented Gabor patches) and hit/miss responses, for trials in which the highest-contrast stimulus was shown in the hemifield of view contralateral to the recorded hemisphere. We analyzed neuronal responses in areas corresponding to those analyzed in the previous set of experiments [V1; VISa and VISam (data not shown), which are two secondary visual cortices spatially overlapping with PPC; ACC], as well as in a region broadly defined as the supplementary motor cortex (MOs), where report-related activity has been shown to originate (Allen et al., 2017; Fig. 1B). Results were in line with those that we observed in the first dataset. V1 showed a bimodal pattern of activity, with an early sensory-evoked response (that, however, in contrast with our dataset, did not encode stimulus saliency) followed by a report-related bump in activity (Fig. 1E). Activity in higher-order regions followed the early response displayed in V1 and was only report related. These results suggest that the spatiotemporal progression of visual- and report-related activity is mostly independent from the details of the task being performed. Report-related activity, however, could reflect several factors besides stimulus detection, such as motor correlates and reward expectation (Stringer et al., 2019; Avitan and Stringer, 2022; Bimbard et al., 2023; Pennartz et al., 2023; Oude Lohuis et al., 2024). To better understand what the late component of V1 activity reflects, we also plotted neuronal responses during false alarms—when a response is performed in the absence of a visual stimulus—and error trials, when animals perform an incorrect response (Extended Data Fig. 1-1). False alarms in general did not correlate with any clear neuronal activity, for what pertains the data we collected (Extended Data Fig. 1-1A; note that we did not observe a sufficiently high number of error trials). For the data presented in the Steinmetz et al. (2019) manuscript, both false alarms and errors correlated with an increase in firing rates in frontal and association areas (MOs, ACC, and PPC; Extended Data Fig. 1-1B) but strikingly did not display a clear late wave of activity in V1. Thus, nonsensory activity has been previously observed in V1 and has been shown to reflect several factors including corollary discharges related to spontaneous and task-related movement. However, for what specifically pertains the late wave of activity that emerges in hit trials, this predicts the upcoming report and might also correlate with other factors such as reward but does not purely reflect motor actions. Nevertheless, an in-depth assessment of this aspect is beyond the scope of this study.
Earlier findings (Allen et al., 2017; van Vugt et al., 2018; Yin et al., 2020) indicated that report-related activity showed an earliest peak in the prefrontal regions, followed by PPC and V1. Our results suggest a similar picture for what pertains higher-order regions, with prefrontal areas (ACC, MOs; Le Merre et al., 2021) showing earlier indications of hit/miss differences compared with PPC. The relative timing of the appearance of report-related activity in V1 is, however, less clear (compare Fig. 1C,E) but is overall very close to that observed in the prefrontal regions (Allen et al., 2017; van Vugt et al., 2018; Yin et al., 2020). Thus, while the prefrontal regions remain the most likely candidate for the origin of report-related activity—as supported by the causal experiments performed by Allen et al. (2017)—the mechanistic pathway via which this form of activity reaches other cortical areas remains unclear.
A minimal network model reproduces the spatiotemporal propagation of visual- and report-related activity
In order to understand the possible network-level mechanisms underlying the spatiotemporal propagation of visual- and report-related activity across the cortical areas from which we analyzed neuronal activity, we developed a minimal mean-field computational model of the cortical network that: (1) uses available connectomic data (Ermentrout and Cowan, 1980; Ermentrout and Terman, 2010; Bressloff, 2014; Chaudhuri et al., 2015; Joglekar et al., 2018) and (2) is calibrated using in vivo recordings. For this reason, the model only includes the three cortical areas from which we analyzed in vivo neuronal recordings: V1, PPC, and PFC (Fig. 2A). The activity in each area is modeled with a firing rate neural mass model comprising one excitatory and one inhibitory population. Firing rate models of this type are a well-tested tool to describe macroscopic neuronal dynamics, as they average single-neuron spike rates (Ermentrout and Cowan, 1980; Ermentrout and Terman, 2010; Bressloff, 2014; Chaudhuri et al., 2015; Joglekar et al., 2018). Within each mass, the synaptic dynamic has a tunable dispersion time, and oscillatory dynamics are possible because of the coupling between the excitatory and inhibitory population (Coombes and Wedgwood, 2023). We also adopted a classical nonlinear sigmoidal firing rate for each neuronal population (see Materials and Methods for a complete description), which is standard for neural mass models in the literature (Ermentrout and Cowan, 1980; Ermentrout and Terman, 2010; Bressloff, 2014; Chaudhuri et al., 2015; Joglekar et al., 2018).
The mean field model comprises a total of six neuronal populations, two in each cortical area, which feature local as well as long-range connections. More precisely, the excitatory–inhibitory pair in each cortical area is fully connected and should be taken together as a model of a cortical area, where the excitatory node has been fitted to experimentally collected neuronal activity and therefore represents the output of an area (in terms of “firing” activity) and all other variables represent hidden state variables. In addition, there are long-range excitatory connections to and from each cortical area. Crucially, connection strengths between areas were taken from recent experimental data (Knox et al., 2019; Fig. 2B, highlighted entries). In particular, we employed values of directed connection density between V1, secondary visual areas A and AM (which are considered the mouse homologue of PPC; Driscoll et al., 2017; Pinto et al., 2019; Arlt et al., 2022; Oude Lohuis et al., 2022b), and MOs (which is considered as a component of PFC; Le Merre et al., 2021) and is thought to be the key cortical area mainly in view of generating report-related activity (Allen et al., 2017; Steinmetz et al., 2019). All other model parameters were calibrated (see Materials and Methods) to enable the excitatory nodes to reproduce patterns of activity comparable with those observed in vivo, as reported in earlier sections. During the tuning procedure all parameters in the model (characteristic rise/decay times, activation of the nonlinear firing rate functions, and local excitatory-inhibitory coupling strengths) were calibrated, while the interareal connections were kept fixed, because we had direct access to experimental data on these parameters. In this way we could test to what extent the generation and propagation of report-related activity in the three cortical areas on which we focus is shaped by corticocortical connectivity. Importantly, as our model comprises neural masses that jointly mimic the activity of whole cortical areas, all parameters except the activity of excitatory nodes do not represent measurable variables, but rather hidden state variables or input parameters that do not aim to model specific single-neuron parameters. For instance, while input currents are measured in picoampere, they reflect input to a whole cortical area and not to single neurons.
We modeled a visual stimulus via an applied transient step current, with varying intensity, on the excitatory population of V1 (Fig. 2A; Fig. 2C–E, Iapp time traces), and monitored the elicited cortical firing rate response in excitatory and inhibitory populations of V1, PPC, and PFC (whose time traces are also seen in Fig. 2C–E). To calibrate the model, we applied a visual input lasting for 500 ms to the excitatory V1 node and replicated the following experimental results. First, when subjected to a sufficiently strong stimulus, V1 activity displayed an early-onset response peaking ∼100–200 ms (before the termination of the visual stimulus) that then dropped to lower values (Fig. 2C). This reproduces the adaptation to stimuli typically observed in the visual cortex (Fig. 1; Supèr et al., 2001; Steinmetz et al., 2019; Kirchberger et al., 2021; Oude Lohuis et al., 2022c). Second, high-amplitude visual stimuli evoked a bimodal V1 response, that is, an early-onset peak of activity followed by a later peak (Fig. 2E). This second peak was absent if the stimulus had a low-amplitude (low salience stimulus) and accounts for the report-related activity observed in vivo (Fig. 1, compare hit trials). While the model itself does not include an actuator stage to perform an actual report, we consider the emergence of the late-activity bump to represent an instance of stimulus detection by the prefrontal/premotor cortical regions. This mimics the spatiotemporal time course of sensory detection, as can be observed from the neural recordings shown in Figure 1 and from the related publications (Steinmetz et al., 2019; Oude Lohuis et al., 2022c).
Following this tuning procedure, we observed that, when the network was subjected to stimuli of various intensity for 500 ms, it displayed a strongly nonlinear response for the late peak of report-related activity: for smaller inputs
This is in line with experimental findings showing that, when subject to a sufficiently large stimulus, large late-latency activity arises with high probability, but not with certainty (van Vugt et al., 2018). The probabilistic nature of this response is analyzed in detail in later sections. Before addressing this aspect, we observed the dynamics in each neuronal population and noted that the occurrence of a late-latency activity bump appears to be feedback-induced. The external stimulus activated V1 which, in turn, following a feedforward chain, activated PPC and PFC. The activity in the latter areas reached a peak (Fig. 2D,E, vertical time marker) before decaying owing to local (intrapopulation) inhibition. The time marker aligned remarkably well with the small late-latency activity in V1, signaling the onset of a feedback mechanism (from PFC and PPC back to V1; Fig. 2A, schematic). In realizations in which the late activity bump occurred, it was again PFC and PPC that displayed a peak preceding the late, report-related activity in V1, in line with a feedback mechanism. This claim will be further substantiated in the following sections.
Thus the model, using a set of nominal parameters, was able to qualitatively reproduce the types of activity we observed in vivo. While the model was specifically tuned to reproduce V1 activity, we also obtained comparable patterns of activity in PPC and PFC, indicating that the model could be used to study the mechanisms underlying the propagation of activity across cortical areas during sensory motor transformations. In particular, we focused on studying the role of feedback connections in the genesis of the late activity bump, i.e., of report-related activity.
We highlight that the time courses of the activity of excitatory nodes are strongly determined by the inhibitory ones: in Figure 2D,E, it is visible that inhibitory nodes in each cortical area activate after the corresponding excitatory node, and this determines the rise-and-fall behavior in the latter. It is known that neurotransmitter release in excitatory and inhibitory populations are affected by timescale separation between signals (Rodrigues et al., 2016). Our model achieves the delayed inhibitory activation using timescale separation between excitatory and inhibitory rising times (see discrepancies in the parameters
The likelihood of late activity bumps is influenced by variations in internal state
As we have seen above, when the network is in the nominal setup and the visual stimulus is sufficiently high, a late activity bump occurs with a given probability, upon perturbing the initial state of the system. We investigated systematically this scenario by running 100 simulations during which the initial state of the excitatory V1 population was picked randomly and uniformly between 0 and 0.05 spikes/s, thereby imposing a small variance in the internal state (Fig. 3A) that is in line with experimental observations linking cortical state fluctuations to perception (Supèr et al., 2003; McGinley et al., 2015a,b; Speed et al., 2019; Samaha et al., 2020). We observed that early V1 responses elicited by either small
We therefore introduced a cumulative (integral) spiking measure S, with the view of determining the likelihood of late-latency activity in V1. For each of the V1 traces seen in Figure 3A, we counted the average cumulative number of spikes occurring after the early activity bump in V1 (Materials and Methods). More precisely, we disregarded the trajectory before the reference time
Based on the fraction of trajectories whose S value falls in each of these three bands, we estimated the probability of having only the early activity bump as P1b, both the early and late activity bumps as P2b and overshoot as Pov. For example, from the histograms in Figure 3B with stimulus
Late-latency activity relies on network feedback
Armed with a quantitative index to inspect the likelihood of late-latency activity, we investigated how this likelihood changes upon variations in the network topology. As we shall see below, this analysis revealed that the late-latency activity is feedback induced. We performed two different experiments which transform the connectivity matrix. In the first one, we perturbed the connectivity matrix at the initial time and kept the matrix constant thereafter. In the second experiment, we dynamically perturbed the matrix to explore the effects of abrupt changes to the topology of the network.
With the view of imposing changes in the network connectivity, we introduced a morphing parameter
α (see Materials and Methods). When
We first used the morphing parameter to vary the strength of selected networks links from the starting time onward, beginning with the feedback link from PFC to PPC (Fig. 4A), signposted with a red arrow in the network schematic (mathematically, the PFC→PPC connection was scaled by factor
α). We repeated the experiment of Figure 3A,B for various values of the morphing parameter (
α between 0 and 1.5) and recorded the probability of a single early bump
The overall information we gathered from the heatmap in Figure 4A can be summarized as follows. First, the network produced a late activity bump robustly with respect to changes in the PFC→PPC feedback link: light yellow areas (late-bump probabilities close to 100%) were found in a variety of network configurations (for various values of
α). Second, there are regimes, labeled “ov” and “1b,” where the feedback bump was absent, but either an overshoot (ov) or an early bump only (1b) were found with probability >99%, respectively. Third, the network could produce a late activity bump even when the feedback link PFC→PPC was weakened with respect to the nominal condition, provided that the strength of the impinging stimulus was increased; this can be deduced from the yellow area in Figure 4A “curving upwards” toward higher values of
A markedly different behavior was observed when we perturbed the feedback link PPC→V1 (Fig. 4B). From Figure 4B, it can be seen that the likelihood of a late activity bump was strongly affected by changes in this feedback link. Small deviations from the nominal value of the link caused the late activity bump to disappear quickly. While the network could tolerate a weaker PFC→PPC link (Fig. 4A), even the slightest weakening of the PPC→V1 link caused a complete suppression of the feedback bump. This data revealed that the experimentally derived anatomical connectivity used in the nominal conditions
On the contrary, the network's activity was only minimally affected by changes to the feedback link PFC→V1. Figure 4C shows that feedback bumps could be produced with high probability even when this link was absent
For the experiments in Figure 4A–C, we perturbed one link at a time, but the morphing parameter can also be varied on multiple links simultaneously. In Figure 4D, for instance, we strengthened or weakened all the feedback pathways at once. These manipulations showed that late-latency bumps cannot exist without (or with too much) feedback. Therefore, it is the interplay between the various feedback pathways that generates the late-latency bump. This was further confirmed by the results displayed in Figure 4E,F, showing that a network in which the PFC or PPC nodes were progressively isolated (transforming the architecture into a two-node network) did not display robust late activity bumps.
The previous experiment modulated the strength of feedback connections over the whole simulation period. However, we expect that, if the network receives a shock in the form of instantaneous removal of certain links during sensory processing (rather than from the initial time), this will also have an impact on late-latency activity. We investigated this scenario in a further experiment: we selected initial states which, in nominal conditions, would lead to late-latency activity; we then ran this network up to a chosen time T*, at which we instantaneously set selected links to zero (see Materials and Methods). In Figure 5A.1, the PFC→V1 link is removed for different values of T* in the range 50–450 ms, when applying a value of Imax = 1.8 pA. We found that inactivating the link at any time resulted in a quick decay in V1 activity, preventing the feedback bump to occur. In Figure 5B.1, analogous results are found when PFC is instantaneously isolated from the rest of the network. We found similar results when we removed network links from the initial time (Fig. 4).
However, as we discussed earlier, the amplitude of the external stimulus to V1 (i.e., Imax) influences the genesis of the feedback bump. We thus repeated the previous experiments using a larger value of Imax = 3 pA in order to establish whether sufficiently large inputs can compensate for instantaneous inactivation of the network links. In all cases, except for the PFC→V1 link, we found that large stimuli cannot overcome instantaneous inactivation of a set of links, hence impairing the generation of a feedback bump. In Figure 5B.2, representative of most cases, it can be seen that a second bump is not formed, irrespective of the value T*. On the contrary, a feedback bump is still visible when inactivating the PFC→V1 link with a T* between 150 and 300 ms, showing that feedback activity can be restored by sufficiently large external drive in spite of the inactivation of this specific link.
In summary, the analyses performed using the morphing parameter shed light on the fact that the late activity bump is dependent on feedback connections. Upon external stimulation, V1 is activated and a feedforward pathway excites PPC and PFC. From these two areas, a late-latency activity bump appears in V1, mainly owing to an indirect pathway from PFC to PPC and then to V1. It turns out that direct feedback from PFC to V1 is instead not crucial for feedback bumps to be observed in V1, because the impact of reducing this link can be easily compensated by increasing stimulus strength.
Discussion
In this study we set out to elucidate how report-related late activity patterns observed in the cortex may mechanistically emerge as a function of the anatomically constrained connectivity between sensory, association, and prefrontal cortices. We presented evidence of two crucial stages in the processing of visual information during perceptual decision-making in mice, replicating and extending previous work in both humans and animals (Del Cul et al., 2007; Dehaene and Changeux, 2011; Allen et al., 2017; van Vugt et al., 2018; Steinmetz et al., 2019; Oude Lohuis et al., 2022c; Alilović et al., 2023). Neural recordings collected from two independent labs using two different tasks revealed that V1 firing rate at 100 ms after stimulus change was uniquely modulated by the saliency of the stimulus and not by the decision made by the mouse (Oude Lohuis et al., 2022c). In contrast, a second, later wave of V1 activity reflected a combination of stimulus saliency and decision outcome. This late wave of activity was stronger for hits than for misses and coincided with increased report-related activity in both posterior parietal and frontal cortex. Previous studies have shown that this late wave of activity is—in sensory cortices—orthogonal to sensory-evoked, early activity (Zatka-Haas et al., 2021). Moreover, the onset of this late component determines for how long V1 is necessary to detect visual stimuli: inactivating V1 after its onset does not impair detection (Oude Lohuis et al., 2002c). Some hypotheses have been made about a possible involvement in learning processes or in performing self-motion correction of visual information (Avitan and Stringer, 2022; Pennartz et al., 2023), but the function of the large increase in V1 spiking activity that predicts the upcoming detection of a visual stimulus remains poorly understood.
Here we set out to model the interactions between three key cortical regions that are thought to underlie the origin and propagation of report-related activity, to understand the mechanisms giving rise to the observed neural dynamics. Crucially, while designing the model, connection strengths between V1, PPC, and PFC were taken from recently revealed anatomical connectivity profiles between these regions. Thus, connectivity between cortical regions was imposed as a fixed constraint onto neural activity. Previous studies on the origin of report-related activity either incorporated a limited set of cortical areas without constraining interarea connectivity to realistic values (Del Cul et al., 2007; van Vugt et al., 2018) or included many cortical regions but only globally varied interareal coupling strength (Dehaene et al., 2003; Dehaene and Changeux, 2005; Castro et al., 2020). Our approach allowed to mechanistically test the role of all combinations of single and multiple feedback connections with realistic strengths, something not feasible in whole-brain models. Importantly, thanks to the close link between experiments and our model, the time-specific inactivation experiment also provides predictions that, in contrast with more complex modeling approaches, are directly testable. Specifically, by optogenetically inactivating (i.e., isolating) PPC or PFC starting from different time points following stimulus onset [see Resulaj et al. (2018); Kirchberger et al. (2021) and Oude Lohuis et al. (2022c) for a similar approach], one could test the model validity in an existing experimental setup. Nevertheless, we cannot exclude that including additional areas known to play a role in top-down modulation of sensory areas (Wimmer et al., 2015; Zhang et al., 2016; Halassa and Kastner, 2017) might have partially modified the results we obtained. However, the fact that model behavior echoed the neural data observed in vivo indicates that the model's architecture and the free model parameters were enabling physiologically plausible sensory-evoked activity for a given, anatomically based interareal connectivity matrix. Nevertheless, it is important to briefly discuss the possible role of different areas. First, the anterior cingulate cortex (ACC) sends stronger feedback projections to V1 compared with other subdivisions of PFC such as MOs (Zhang et al., 2016; Le Merre et al., 2021). However, even though ACC has been shown to significantly modulate V1 activity (Zhang et al., 2014; Fiser et al., 2016), report-related activity has consistently been found to first appear in other PFC subdivisions compatible with MOs (Allen et al., 2017; Steinmetz et al., 2019; Yin et al., 2020; Takahashi et al., 2021; Inagaki et al., 2022). Second, while we focused on PPC, other temporal association areas have been implicated in visual perception (Conway, 2018). However, at least in rodents, medial association areas such as PPC, which have been hypothesized as the rodent homolog of the dorsal stream (Wang et al., 2012; Glickfeld et al., 2014), are more strongly connected to prefrontal areas compared with temporal association cortices (Harris et al., 2019; Knox et al., 2019). Furthermore, to what extent individual areas in the mouse association cortex provide unique contributions to sensory processing or form instead a redundant network is a matter of active debate (Glickfeld et al., 2014; Jin and Glickfeld, 2020; Keller et al., 2020; Kirchberger et al., 2021; Oude Lohuis et al., 2022b,c). Therefore, we believe that considering PPC as a single network node is an important first step to better understand the role of the parietal and temporal cortex in the generation of report-related activity, to be followed up by a more detailed characterization. Finally, although we cannot exclude a role of thalamic nuclei in mediating long-range top-down modulation, the thalamus seems to mainly play a modulatory role and not directly be involved in information transfer (Wimmer et al., 2015; Halassa and Kastner, 2017).
There were at least three notable observations in our model behavior. First, increasing the input strength to V1 led to both stronger early activity waves (∼100 ms) as well as an increased likelihood of a second late wave of activity (∼400 ms), as observed in our data and in previous studies (Supèr et al., 2001; Del Cul et al., 2007). Stronger V1 input also directly led to increased late activity in PPC and PFC, temporally preceding the late wave of V1.
Second, the initial prestimulus condition of each neural node strongly determined the likelihood of the late wave to occur. In fact, we observed a regime of model parameters in which there was a clear nonlinear threshold for “igniting” this late wave, in line with the Global Workspace model (Dehaene et al., 2003; Dehaene and Changeux, 2005, 2011; Mashour et al., 2020). This is in agreement with recent observations that variations in behavioral and cortical state, associated with ongoing fluctuations in prestimulus neural activity, partly driven by neuromodulatory sources, strongly determine the likelihood that a stimulus will be perceived and reported and that late report-related neural activity in cortex is observed (McGinley et al., 2015a,b; Speed et al., 2019; Waschke et al., 2019; McCormick et al., 2020; Nuiten et al., 2023; Beerendonk et al., 2024).
Third, not only did we show that late activity in V1 was driven by feedback from higher-order regions but also, more specifically, that although PFC was necessary to generate report-related activity in V1, it exerted its final influence on V1 likely only indirectly, through PPC. While PFC activity was necessary for the buildup and initiation of the late, report-related bump in V1 activity, it was PPC that seemed to have the final veto, determining its characteristics. Removing the feedback connection from PFC to V1 had very limited influence, and the occurrence of a late wave could still be observed when input strength was increased. Thus, an interplay between frontal and parietal regions is required, and most efficient, for eliciting late feedback activity in V1. This observation is important because, in the Global Neuronal Workspace model, the frontal cortex has always been considered the site responsible for igniting the network and hence threshold setting (Dehaene, 2014; but see Sergent et al., 2021). It has been hypothesized that once the threshold for conscious report in PFC is crossed, PFC sends information to other brain regions, including the parietal cortex. We provide tentative evidence for a possible different division of labor between the frontal and parietal regions, in which the frontal cortex acts as a fast accumulator of evidence and parietal cortex as a much slower one (Kim and Shadlen, 1999; Shadlen and Newsome, 2001; but see Pinto et al., 2022). In this scenario, PFC starts to quickly feed stimulus evidence to the parietal cortex, but activity in PFC quickly reaches a ceiling level. When activity in PFC reaches such a level, this is not per se sufficient to elicit report-related activity. In contrast, the parietal cortex keeps on accumulating evidence from both the sensory and prefrontal regions over time and once PPC activity crosses a certain threshold, this triggers report-related feedback in V1. This result is a direct consequence of the known anatomically constrained connectivity between areas (Harris et al., 2019; Knox et al., 2019) and specifically from the fact that PPC has a much stronger feedback connection to V1 compared with PFC. Thereby, our study provides a novel mechanistic hypothesis on the cortical pathway via which report-related activity, a hallmark of conscious access, is generated in the frontoparietal cortex and then reaches sensory areas. Future experimental work will be required to validate our results in vivo.
Footnotes
The authors declare no competing financial interests.
We thank Eric Dijkema for providing support in accessing the data used for Figure 1 and the Carandini/Harris lab for providing free access to the raw data used in Steinmetz et al. (2019). We also thank the Institute for Advanced Studies of the University of Amsterdam for supporting a workshop that spearheaded this study. This work was supported by Netherlands Organization for Scientific Research (NWO Crossover project INTENSE to U.O. and C.M.A.P.) and the European Research Council (ERC Starting grant, 715605 project CONSCIOUSNESS to S.V.G.). This project was also made possible through the support of a grant from Templeton World Charity Foundation, Inc. (funder DOI 501100011730) through grants TWCF0646 (to U.O. and C.M.A.P.) and TWCF-2022-30261 (to U.O.). The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of Templeton World Charity Foundation, Inc.
↵*D.A., S.G., and U.O. contributed equally to this work.
M.N.O.L.’s present address: Champalimaud Research, Champalimaud Foundation, Lisbon 1400-038, Portugal
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.