Information Theoretic Approaches to Deciphering the Neural Code with Functional Fluorescence Imaging

The SMGM information metrics

Here, we review the derivation of the SMGM information metrics and the underlying assumptions. For illustrative purposes throughout this manuscript, we use the example of spatial encoding in which the firing pattern of neurons carry information about the animal’s location along a linear track; however, the derivations, equations and conclusions generalize to encoded variables over other domains and dimensionalities.

Consider a random variable X representing the positions an animal might take, with x being its value measured at one time sample. The positions are subdivided into N spatial bins, such that x can take on the values {1,2,...,N} . For our analyses, N=60 . Consider a random variable Y representing the number of APs a neuron might fire, where y is the count measured within a time sample. y can take on the values of {0,1,...,+∞} . X and Y are both discrete. If X and Y both obey the assumption that each time sample is independent (i.e., they are stationary), then the MI (I, in bits per sample) between X and Y is expressed as follows: I(X;Y)=∑i=1N∑y=0+∞pX,Y(xi,y)log2pX,Y(xi,y)pX(xi)pY(y), (1)

where pX(xi) is the (marginal) probability that the animal is in the ith spatial bin during a time sample, pY(y) is the probability that the neuron fires y APs in the time sample, and pX,Y(xi,y) is the joint probability that the neuron fires y APs and is in the ith bin. Recall that pX,Y(xi,y)=pY|X(y|xi)pX(xi) , where pY|X(y|xi) is the conditional probability that the neuron fires y APs given that the animal is in the ith spatial bin. We can thus rewrite Equation 1 as follows: I(X;Y)=∑i=1N∑y=0+∞pY|X(y|xi)pX(xi)log2pY|X(y|xi)pY(y). (2)With the further assumption that the firing of the neuron follows Poisson statistics, we can then estimate the MI as follows: let the AP rate (AP/s or Hz) in a single bin be λi , and the average across the session be λ¯ . For an arbitrarily small time window Δt , the probability that an AP occurs in that window is Pr(Y=1|x=i)=λiΔt , with the probability that an AP occurs regardless of position as Pr(Y=1)=λ¯Δt . We can thus rewrite Equation 2 as: I(X;Y)=∑i=1NλiΔtpX(xi)log2λiλ¯. (3)By integrating over 1 s (∫01I(X;Y)dΔt ), we obtain the first key SMGM metric for spatial information as measured by AP firing, which is in units of bits per second: IsÊ=∑i=1NλipX(xi)log2λiλ¯. (4)For notation, we will use a carrot (̂) to indicate an information value that is measured from experiment, the superscript (E in this case) to show the source of the data, and a subscript to show the units/formula used (bits per second in this case). Thus, IsÊ is the information measured via electrophysiology in bits per second. This metric is linearly dependent on the average firing rate of the neuron, and this dependence is often removed through normalization by the average firing rate to obtain the second key metric of spatial information as measured by AP firing, which is in units of bits/(s/Hz), or more commonly, bits per AP: IAPÊ=1λ¯∑i=1NλipX(xi)log2λiλ¯. (5)Therefore, these two key metrics of spatial information are defined completely by quantities that can be experimentally measured: the mean firing rate (λ¯) from the AP counts over the duration of the recording, the AP firing rate in the ith bin from the average rate map (λi) , and the probability that the animal is in the ith spatial bin from the normalized occupancy map (pX(xi) ). The quantity of and noise in these measurements affects the quality of the metric: in particular, undersampling because of low firing rates or low trial counts induces a substantial positive bias (Treves and Panzeri, 1995).

In the derivation of these metrics, there are two key assumptions that are violated by functional fluorescence recordings. First, the recordings do not follow Poisson statistics: instead of discrete counts of APs ( y), the functional fluorescence traces consists of a continuous relative change in fluorescence (ΔF/F ), and instead of a firing rate map (λi) measured in Hz, average intensity maps in units of ΔF/F are generated. The stationarity assumption is also violated: because of the slow decay, a time sample of the fluorescence traces depend on the previous samples. The violation of these assumptions by functional fluorescence recording will affect the precision and induce biases in the SMGM information metrics. Since these effects have not previously been addressed or quantified, we measured these biases here using a simulation study.

Building a ground truth library of 10,000 neurons with known values of information

To create a neuron with a known, ground truth information value, it was necessary to generate a continuous (i.e., infinitesimally small bins) rate map (λ(x)) matching the desired information. To do this, we first normalized the track length to 1 and assumed the animal’s occupancy map to be spatially uniform (pX(xi)=1N) . We then created an exponentiated cubic spline with five randomly positioned nodes (Fig. 1A) to build a starting continuous map of the normalized instantaneous firing rate, λλ¯(x) , with the integral normalized to 1. We calculated the ground truth amount of information in bits per AP as follows: IAPE=∫01λλ¯(x)log2(λλ¯(x))dx. (6)The locations of the five nodes were then systematically varied (see Materials and Methods) to minimize the squared error between the value calculated in Equation 6 and a target amount of information (Fig. 1A,B), in the end resulting in a mean error of 5.1 × 10⁻⁹ bits/AP and a mean absolute error of 1.5 × 10⁻⁷ bits/AP. The rate map at this convergence point was used for further analysis. This procedure was repeated to generate 10,000 mock neurons with a range of (known and ground truth) information values. Note that the value in Equation 5 cannot be higher than when all the APs arrive in one spatial bin; the rate in that bin is Nλ¯ . If we assume uniform occupancy (pX(xi)=1N) , then the maximum measurable information is log2N , in our case, 5.9 bits/AP with N=60 bins. Thus, the information values considered here range between 0 and 6 bits/AP (Fig. 1C). We chose a mean firing rate (λ¯) for the neurons between 0.1 and 30 Hz, a range observed for a variety of different cortical and hippocampal neurons during behavior (Shafi et al., 2007; DeWeese et al., 2008; O’Connor et al., 2010; Roxin et al., 2011; Buzsáki and Mizuseki, 2014). From Equations 4, 5, the ground truth information in bits per second is IsE=λ¯IAPE . IsE for these choices resulted in ground truth information values between IsE=0 and IsE=24 bits per second (Fig. 1C). Example low (IAPE=0.04 bits/AP) and mid (IAPE=2 bits/AP) rate maps are shown in Figure 1D.

These rate maps provided a basis for generating mock AP firing data (and functional fluorescence data, see below). Under real experimental conditions, recording duration and bin sizes are finite and animal occupancy maps (pX(xi) ) are not spatially uniform. These experimental limitations add error to the estimate of a neuron’s ground truth information value. Therefore, to accurately re-create these limitations in our simulation study, we used real behavior datasets from head-restrained mice running along a 3-m virtual linear track for water rewards (acquired as in Sheffield and Dombeck, 2015; Sheffield et al., 2017). Unless otherwise indicated, all values reported will be the mean±standarddeviation . We selected at random from a library of 574 behavior sessions from mice navigating along familiar tracks and concatenated and truncated these sessions to create behavior sessions uniformly sampled up to 60 min in duration (average 30.2±17.1 min), resulting in an average 132±71.2 laps per session and an average running speed of 19.3 + 3.87 cm/s (Fig. 1E1). This behavior, the average firing rate (λ¯) , and the normalized rate map (λλ¯(x) ) from the mock neurons were used to create an instantaneous firing rate trace (Fig. 1E2), sampled at 1 kHz, from which AP times were generated assuming Poisson firing statistics (Fig. 1E3). An example mock of spiking in response to behavior for low (0.04 bits/AP) and mid (2 bits/AP) information neurons can be seen in Figure 1F–H. From these spiking responses, we then generated mock fluorescence traces by convolving the raster with a double-exponential kernel matching the rise and fall times for GCaMP6f (Chen et al., 2013; Fig. 1E4) and adding random Gaussian noise to model shot noise. Mock fluorescence traces for the two example neurons in Figure 1F–H can be seen in Figure 1I,J. The mock AP and fluorescence traces were used to create session mean spatial maps, of binned firing rate (λi in Hz) and change in fluorescence (fi in ΔF/F ), for information analyses (Fig. 1K). By repeating this process, we built a large dataset of spiking and fluorescence traces, generated from our library of mock neurons with known amounts of information and using real animal spatial behavior. With tens of thousands of these mock neuron recordings, we could then assess the effects of many simulation parameters on the information values determined from the metrics including firing rate, session duration, fluorescence kernel shape, and ground truth information value.

Quantification of the accuracy and precision of the SMGM bits per second metric using functional fluorescence recordings

We first applied the SMGM bits per second metric (IsÊ ) to our mock AP recording traces to verify that they can recover our ground-truth information values given finite recording durations and bin sizes, and non-uniform animal occupancy maps (pX(xi) ). Figure 2A shows three mock neurons with ground truth information values of IsE=3 , 15, and 23 bits per second. When the SMGM bits per second metric (IsÊ ) was applied to the AP traces from these example neurons, the information was well recovered, with IsÊ=2.8 , 15, and 24 bits per second, respectively. The results from these examples also held across the full 10,000 mock neuron library (Figs. 2B–D), as a linear fit (y-intercept=0.093±0.040 , intercept p = 4.6 × 10⁻⁶ bits per second and slope=0.97±0.0030 , slope p ≪ 0.01) explained nearly all the variance (R² = 0.97), the average error was 0.22±1.25 bits per second (1.0±0.69% error) and the absolute error was 0.64±1.05 bits per second (8.4±0.69% error). There is a substantial positive bias for the lowest firing rates and smallest number of trials (Extended Data Fig. 2-1A,B) which has been previously well characterized (Treves and Panzeri, 1995), with average errors exceeding +10% for <6 min of recording, mean rate under 0.6 Hz, and under 11 trials. Thus, the SMGM bits per second metric (IsÊ ) recovers the ground-truth information well using AP recordings, with the only error coming from finite recording time and variable animal behavior.

We next discuss the changes to the SMGM bits per second metric (IsÊ ) commonly used for application to functional fluorescence traces (Hainmueller and Bartos, 2018; Heys and Dombeck, 2018), and explore the implications of these changes. Most simply, the mean firing rate (λ¯) and the mean firing rate in a spatial bin (λi ) are replaced by the mean change in fluorescence (f¯) and the mean change in fluorescence in a bin (fi) . Making these substitutions in Equation 4 results in the information as measured by functional fluorescence: IsF̂=∑i=1NfipX(xi)log2fif¯. (7)The fluorescence map fi differs from the firing rate map λi in two ways. First, the fluorescence map is approximated by the firing rate map scaled by a factor c, dependent on the height and width of the kernel and measured in units of ΔFFHz ; and second, it is smoothed by the kernel (Fig. 1E4). If we discount the latter for a moment and focus on the scaling, f≈cλ , we can see that substituting λ with cλ in Equation 4 results in IsF̂=cIsÊ . The units for IsF̂ are no longer in bits per second, as it has previously been reported (Hainmueller and Bartos, 2018), but are instead in units of bitsΔFFsecHz or bitsΔFFAP , which are difficult to interpret (see below, Guidelines for application of information metrics to functional fluorescence imaging data). The effect of smoothing is difficult to analytically since it both alters c by changing the average intensity and distorts the firing rate map. Therefore, to fully quantify the impact of convolving an AP recording with a functional fluorescence kernel on recovering ground truth information, we used our mock fluorescence traces.

We applied a GCaMP6f modeled kernel to the 10,000 mock AP traces to generate 10,000 mock fluorescence calcium traces. Figure 2A shows the fluorescence traces generated from three mock neurons with ground truth information values of IsE=3 , 15, and 23 bits per second. The effects of the convolution can be seen in the differences in scaling and shape between the fluorescence maps fi and the firing rate maps λi . When the fluorescence metric (IsF̂ ) was applied to the fluorescence traces from these example neurons, the information recovered was IsF̂=0.13 , 0.47, and 1.1 bitsΔFFAP , respectively, indicating significant deviation from the ground truth information values assuming the units are comparable. The results from these examples also held across the full 10,000 mock neuron library (Fig. 2E–G), as there was a clear scaling of the ground truth information and a consistent underestimation with a mean error of −11.1±6.7 AU (−96.0±1.3% error). The best-fit line of the measured information (IsF̂ ) versus the ground truth information (IsE) had an intercept near 0 (0.0029±0.0016 bitsΔFFAP , p = 0.07). The slope of this fit was 0.039±12e-4 ΔFFHz (p ≪ 0.01), which provides a measure of the scaling factor ( c). This error was not corrected for with denser sampling: it remained consistent even at high firing rates and many trials (Extended Data Fig. 2-1C,D). In addition to this scaling effect caused by c, smoothing of the rate map could induce nonlinearity in the relationship between and IsE . To test for such an effect, we fit the measured information in Figure 2E with a saturating exponential and compared the fits using a likelihood ratio test: the exponential did not significantly improve the fit (χ12=0.093 , p = 0.76), which indicates that smoothing by the kernel does not induce significant nonlinearities. c is dependent on the height and width (the integral) of the kernel and was measured here as 0.039 ± 12e-4 ΔFFHz . The consistent, negative bias observed in estimating information with IsF̂ (Fig. 2E) would be easy to correct for assuming the c factor, and therefore the kernel, were similar across all measured neurons. This point is considered further below in the Guidelines for application of information metrics to functional fluorescence imaging data. We conclude that ground truth information, as measured by the fluorescence SMGM bits per second metric (IsF̂ ), is transformed into different units and is linearly scaled by a factor c dependent on the height and width of the kernel.

The amplitude (height) of the change in fluorescence can vary across indicators and conditions. The height of the kernel, given a constant kernel width, should linearly scale c and the error in estimating information with IsF̂ . To explicitly test this prediction, we simulated an additional 5000 fluorescence traces with kernels of varying height (0–3 ΔF/F ; Fig. 2I–K), but that maintain the same shape and width (from the GCaMP6f kernel), and then measured the percent error in estimating information with IsF̂ . As observed above for the GCaMP6f example (Fig. 2E–G), the percent error in estimating information with IsF̂ shows little dependence on ground truth information (Fig. 2I–K). However, as a function of the height of the kernel, the percent error (averaged over all ground truth information values) in estimating information with IsF̂ is fit well with an increasing linear function (intercept = −99.8 ± 0.42%, intercept p ≪ 0.01, slope 20.7 ± 0.14%/ΔFF , slope p ≪ 0.01, R2=0.80 ; Fig. 2I,J). Over the wide array of available functional fluorescent indicators in use today (Fig. 2H), this leads to differences in error because of differences in transient height of the indicator used alone. For the indicators shown in Figure 2H, there is an average height of 0.603 ± 0.10 SD ΔF/F : the error spans from −95.8% for the kernel height reported for gCamp6f (0.19 ΔF/F ) to −88.2% for gCamp7f (0.56 ΔF/F ). It should be noted that fluorescence (ΔF/F ) is always reported here as a fractional change, not as a percentage (% ΔF/F ); if a kernel height of 19% ΔF/F is used, the units would again change. Thus, as expected, the percent error in estimating information with the SMGM bits per second estimator (IsF̂ ) scales linearly with the height of the kernel.

The width of the kernel can vary widely across fluorescent indicators (Fig. 2H), with “faster” indicators boasting shorter rise and fall times. The combined effect of a longer rise and fall time is to smooth and delay the AP train; in other words, it acts as a causal low-pass filter. The cutoff period of this low pass filter provides a measurement of the effective width of the kernel (see Materials and Methods). The effect of such differences in kernel shape on the error in estimating information with IsF̂ is difficult to measure analytically. We therefore simulated an additional 5000 fluorescence traces with kernels of different kernel widths (but constant height of the GCaMP6f kernel), resulting in a range of kernel durations (rise times: 1 ms to 1 s, fall times longer than the rise time up to 2 s), and then we measured the percent error in estimating information with IsF̂ . Similarly, as observed above for the GCaMP6f and varying kernel height examples (Fig. 2E–K), the percent error in estimating information with IsF̂ shows little dependence on ground truth information (Fig. 2N). Interestingly, the percent error (averaged over all ground truth information values) in estimating information with IsF̂ shows a complex nonlinear response as a function of the width of the kernel (Fig. 2L,M). The error increases up to a kernel width of ∼3 s, at which point it saturates at approximately −85% error. This arises from an interaction between changing the average value of the original AP trace and flattening the average fluorescence map (fi ). Over the wide array of available functional fluorescent indicators in use today, this leads to differences in error because of differences in width of the indicator used alone. For example, an average error of −97.1 ± 0.63% were observed for iGluSnfR, the shortest indicator considered here at 0.52 s. For gCamp6s, the slowest indicator examined (2.54 s), the average was −89.6 ± 4.6%. To estimate the percent errors for these five indicators considering differences in both height and duration, we used these five kernels to generate mock fluorescence traces from the 10,000 neurons in Figure 2B–G. The resulting distributions, estimated c values, and mean and absolute errors can be seen in Extended Data Figure 2-2. In summary, we conclude that information, as measured by the fluorescence SMGM bits per second metric (IsF̂ ), is transformed into different units and is linearly scaled by a factor ( c) dependent on the height and width of the kernel, with c linearly dependent on height and nonlinearly dependent on width. The error induced by these transformations changes substantially over the range of kernel values of the different functional indicators widely used today, and therefore these are important factors to consider when designing and interpreting functional imaging experiments (for further discussion, see below, Guidelines for application of information metrics to functional fluorescence imaging data).

Quantification of the accuracy and precision of the SMGM bits per AP metric using functional fluorescence recordings

The SMGM metric is commonly normalized by the mean rate to obtain a measurement in units of bits per AP. We thus applied the SMGM bits per AP metric (IAPÊ ) to our mock AP recording traces to verify that they can recover our ground-truth information values. Figure 3A shows three mock neurons with ground truth information values IAPE=0.05 , 1.8, and 4.2 bits/AP. When the SMGM bits per AP metric (IAPÊ ) was applied to the AP traces from these example neurons, the information was well recovered, with IAPÊ=0.06 , 1.8, and 4.2 bits/AP, respectively. The results from these examples also held across the full 10,000 mock neuron library (Fig. 3B–D), as a linear fit (y-intercept=0.087±0.029 , intercept p = 2.8e-184 bits per second and slope = 0.93 ± 0.0010, slope p ≪ 0.01) explained nearly all the variance (R² = 0.99), the average error was −0.071 ± 0.23 bits/AP (3.2 ± 5.9% error) and the absolute error was 0.13 ± 0.21 bits per second (8.1 ± 9% error). However, the data were better fit with a saturating exponential (χ12=1.6e3 , p ≪ 0.01) converging to 5.8 bits/AP as it approached the limit because of the finite bin count. There is a substantial positive bias for the lowest firing rates and smallest number of trials (Extended Data Fig. 2-1E,F) which has been previously well characterized (Treves and Panzeri, 1995). Thus, the SMGM bits per-AP metric (IAPÊ ) recovers the ground-truth information well using AP recordings (except at the largest ground truth information values), with the primary error coming from finite recording time and variable animal behavior.

We next discuss the changes needed to apply the SMGM bits per AP metric (IAPÊ ) to functional fluorescence traces and explore the implications of these changes. Most simply, the mean firing rate (λ¯) and the mean firing rate in a spatial bin (λi ) are replaced by the mean change in fluorescence (f¯) and the mean change in fluorescence in a bin (fi) . Making these substitutions in Equation 5 results in the information as measured by functional fluorescence: IAPF̂=1f¯∑i=1NfipX(xi)log2fif¯. (8)

As discussed above, the fluorescence map (fi ) can be approximated as a scaled version of the rate, that is, f=cλ and f¯=cλ¯ . Thus, under this approximation, the c factors in Equation 8 cancel, leading to IAPF̂ equivalent to IAPÊ , with the same units of bits/AP. This, of course, ignores the fact that the kernel smooths the rate map, leading to a bias in the metric that is difficult to quantify analytically.

We then applied the fluorescence SMGM bits per AP metric (IAPF̂ ) to our 10,000 mock GCaMP6f traces. Figure 3A shows the fluorescence traces generated from three mock neurons with ground truth information of IAPE=0.05 , 1.8, and 4.2 bits/AP. When the fluorescence metric (IAPF̂ ) was applied to the fluorescence traces in these examples, the information recovered was 0.04, 1.8, and 3.5 bits/AP, indicating some deviations, especially for the highest information neuron. These results held for the 10,000 mock neuron library (Fig. 3E–G). At low information values, there was little bias, but at higher information values the information recovered was substantially lower than the ground truth information. The mean resulting error was −0.38 ± 0.58 bits/AP (−9.7 ± 27.8%) and absolute error of 0.39 (12.9 ± 26.4%). This error was better fit with a saturating exponential than a linear fit (χ12=1.6e3 , p ≪ 0.01), with the average error <5% up to ground truth information of 1.8 bits/AP and <10% up to 3.0 bits/AP. At ground truth information values higher than 3 bits/AP, the average error was −1.06 ± 0.595 (−22.5 ± 9.44%) and absolute error was 1.07 ± 0.589 bits/AP (22.6 ± 9.21%). This error persisted even with denser sampling: it remained consistent even at high firing rates and many trials (Extended Data Fig. 2-1E,F). Thus, the indicator induces relatively little error at lower information values (<3 bits/AP), but the smoothing effect of the kernel induces a nonlinear, negative bias to the estimator, particularly at ground truth information values over 3 bits/AP.

Although the height of the kernel can vary between different functional fluorescence indicators (Fig. 2H), these height variations linearly scale the fluorescence map. Thus, since IAPF̂ involves normalization by the mean change in fluorescence (f¯) , IAPF̂ should not depend on kernel height. To explicitly test this prediction, we used the 5000 fluorescence traces described in the previous section (Quantification of the accuracy and precision of the SMGM bits per second metric using functional fluorescence recordings), with kernels of varying height (0–3 ΔF/F), but that maintain the same shape and width (from the GCaMP6f kernel). Then, we measured the percent error in estimating information with IAPF̂ (Fig. 3H–J). Unlike for the SMGM bits per second metric, the percent error (averaged over all ground truth information values) in estimating information with IAPF̂ shows little or no dependence on the height of the kernel (p = 0.43), but a nonlinear dependence on ground truth information as in Figure 3E–G, with no significant difference in the parameters of the saturating exponential fit (χ22=1.67 , p = 0.43). Thus, as expected, the percent error in estimating information with the SMGM bits per AP metric (IAPF̂ ) does not vary with the height of the kernel.

With little effect of kernel height on IAPF̂ , the width of the kernel likely drives biases in the metric. We thus used the 5000 fluorescence traces generated from a range of different kernel durations (rise times: 1 ms to 1 s, fall times longer than the rise time up to 2 s, but constant height of the GCaMP6f kernel) from the previous section (Quantification of the accuracy and precision of the SMGM bits per second metric using functional fluorescence recordings), and then we measured the percent error in estimating information with IAPF̂ (Fig. 3K–M). Similarly, as observed above for GCaMP6f and the varying kernel height examples (Fig. 3E–J), the percent error in estimating information with IsF̂ shows a nonlinear dependence on ground truth information (Fig. 3E–G). The percent error (averaged over all ground truth information values) showed a nonlinear response as a function of the width of the kernel (Fig. 3K,L), with a steep increase in error for kernel widths >∼1 s. Even for kernel widths <∼1 s, the percent error was strongly dependent on the ground truth information value, with steep increases in error for values more than ∼2.5–3 bits/AP (Fig. 3M). Thus, as the kernels gets wider, there is more negative bias at lower and lower information measured. The resulting errors are thus larger for wider kernel indicators, for example, with a kernel width the same as gCaMP6s (2.54s), the error exceeds −17% even at low (<0.25 bits/AP) information, with average errors of −0.86 ± 1.0 bits/AP (−31 ± 19% error) and absolute errors of 0.87 ± 1.0 bits/AP (32.6 ± 16% error). In contrast, with a kernel width the same as iGluSnfR (0.52 s), the average error exceeded 5% at 3 bits/AP and 10% at 3.7 bits/AP with a mean error of −0.57±1.00 (−8.0 ± 14%) bits/AP and absolute error of 0.41 ± 0.56 bits/AP (11 ± 11%). To estimate the percent errors for the five indicators shown in Figure 2H, taking into account differences in both height and duration, we used the five kernels to generate mock fluorescence traces from the 10k neurons in Figure 3B–G. The resulting distributions, mean and absolute errors, and error thresholds can be seen in Extended Data Figure 3-1.

Since the known information values in our library of 10,000 mock neurons were determined using the SMGM metric, which includes the assumption that neuron firing follows an inhomogeneous Poisson process, we next investigated whether the biases observed between AP and fluorescence metrics (ISÊ vs ISF̂ and IAPÊ vs IAPF̂ ; Figs. 2, 3) in our mock neuron datasets were also observed in real neuron recordings (i.e., real spiking that could deviate from Poisson firing).We therefore measured information in a real spiking dataset from hippocampal neurons in rats running on a behavioral track (Chen et al., 2016; Grosmark and Buzsáki, 2016; Grossmark et al., 2016). We generated mock fluorescence traces as we did with simulated AP trains from our mock neurons, compared the information measured from APs versus fluorescence (ISÊ vs ISF̂ and IAPÊ vs IAPF̂ ) in the real neuron recordings and found that the biases were largely consistent with the simulated mock neuron datasets (Extended Data Fig. 5-1).

In summary, we conclude that ground truth information, as measured by the fluorescence SMGM bits per AP metric (IAPF̂ ), retains the units and insensitivity to height scaling of the electrophysiological metric (IAPÊ ), but is nonlinearly biased by the smoothing of the fluorescence map dictated by the width of the kernel. The estimation errors strongly depended on both the width of the kernel and the information value being measured. Since these parameters change substantially over the different functional indicators and different neuron types and behaviors that are commonly used today, they are important factors to consider when designing and interpreting functional imaging experiments (see below for further discussion).

Nonlinearity introduces further biases

The results presented in the previous two sections rely on the approximation that ΔF/F scales linearly with the firing rate, which is not strictly true in practice (Dana et al., 2018; Greenberg et al., 2018; Éltes et al., 2019). Calcium imaging can be more responsive to bursts of APs rather than isolated spikes, and saturates at high firing rates. As an example for how to examine how nonlinearities between ΔF/F and firing rate could affect the fluorescence SMGM metrics (ISF̂ and IAPF̂ ), we applied a log-sigmoid nonlinearity (Extended Data Fig. 3-3A) to the 10,000 mock GCaMP6f time-series traces described above, based on the real behavior of GCaMP6f in cultured neurons (Dana et al., 2019; see Materials and Methods). While the resulting measurements (Extended Data Fig. 3-3) of ground truth information, as measured by the fluorescence SMGM metrics, are largely consistent with the results observed when using the linear assumption (Figs. 2, 3), some quantitative difference can be seen. Thus, even a relatively simple nonlinearity between ΔF/F and firing rate can add distortions to the amount of information measured using the fluorescence SMGM approach.

Deconvolution may not be sufficient to eliminate biases

The framework presented here for comparing ground truth information with information measured with the SMGM metrics can be extended to test the efficacy of other strategies for extracting MI. In particular, a perfect AP inference method would alleviate the problems associated with applying the SMGM metrics to functional fluorescence recordings. To test the utility of such a strategy in measuring information, we applied a popular deconvolution algorithm, FOOPSI (Vogelstein et al., 2010; see Materials and Methods), to the same 10,000 mock GCaMP6f time-series traces described above. Importantly, this deconvolution algorithm (and other available algorithms) does not recover traces of relative spike probability or exact spikes times, but instead produces sparse traces with arbitrary units, that have non-zero values estimating the relative “intensity” of spike production over time ( d). This signal can be thought of as a scaled estimate of the number of spikes per time bin, and thus the average intensity map will have some similar properties to the florescence intensity maps, that is, we would expect the intensity maps from deconvolution to approximate the relative firing rate scaled by some factor c , which has arbitrary units.

We then measured information in these deconvolved d-traces using the SMGM metrics (Isd̂ and IAPd̂ ), which are identical to (IsF̂ and IAPF̂ ), except SMGM is applied to d-traces instead of functional fluorescence traces. When using the SMGM bits per second measure (Isd̂ ; Fig. 4A), we found a clear scaling of the ground truth information. The scaling factor was very small (c=1.15e-3±1.8e-5 AU), resulting in low predicted information (mean error −11.4 ± 6.92 AU, mean % error −99.8 ± 0.38%). This error was larger than when we measured information directly from the florescence traces using IsF̂ (Fig. 2E; 96.0% absolute error, rank-sum p ≪ 0.01; c = 0.0390). It is worth noting that the deconvolved trace d can be arbitrarily scaled, so in a sense this error is arbitrary. However, these are the results from the scaling chosen by a widely used deconvolution algorithm and the large error emphasize that the scale of d can have a large effect on the bits per second measure (Isd̂ ).

Assuming that the intensity map of the deconvolved d-traces are a scaled version of the true rate maps, we could measure information using the SMGM bits per AP metric IAPd̂ without changing units (Fig. 4B). Compared with the SMGM bits per AP metric applied to florescence (IAPF ), on average there was some reduction in the nonlinearity at higher ground truth information (IAPE ) values when using IAPd̂ , resulting in linear fits closer to the unitary line (IAPd̂slope=1.02±0.0027,R2=0.76 vs IAPF̂ slope = 0.78 ± 0.0011, R² = 0.93). However, information measured with IAPd̂ was still better fit with a saturating exponential (χ12 = 2.3e3, p ∼ 0) converging to a saturation value of 5.51 bits/AP (compared with 5.78 for IAPF̂ ), as expected since the algorithm is not expected to resolve spikes at orders of magnitude shorter timescales than the kernel. This resulted in a positive bias at lower levels of ground truth information. For ground truth information values below 3 bits/AP, the average error for IAPd̂ was 0.556 ± 0.50 bits/AP (68.9 ± 104%) compared with −0.060 ± 0.13 bits/AP (2.75 ± 0.36%) for IAPF̂ . For ground truth information values above 3 bits/AP, the average error for IAPd̂ was −0.127 ± 0.76 bits/AP (−1.0 ± 16.8%) as compared with −1.04 ± 0.59 bits/AP (−0.22 ± 9.4%) for IAPF̂ . Overall, there was more error when the SMGM bits/AP metric was applied to deconvolved data compared with when applied directly to fluorescence traces [IAPd̂ mean absolute error 0.60 ± 0.47 bits/AP (52.7 ± 89.1%) versus IAPF̂ was 0.13 ± 0.21 bits per second (8.1 ± 9% error), rank-sum p ≪ 0.01]. Thus, when comparing the recovery of ground truth information from functional fluorescence traces using either direct application of the SMGM metrics (IsF̂ and IAPF̂ ) or the application of the SMGM metrics to deconvolved z-traces (Isd̂ and IAPd̂ ), we found better recovery using the direct application approach (IsF̂ and IAPF̂ ).

The KSG and binned estimators are poor estimators of MI in functional florescence data

In addition to SMGM, the KSG and binned estimation metrics have been developed for estimating MI between variables. These other two metrics produce information measured in bits per second, so they are only comparable to the SMGM bits per second estimator (IsF̂ ). The KSG metric uses the kth nearest neighbor distances between points in the neural response and behavioral variable space to estimate information (Kraskov et al., 2004). The binned estimation metric uses discrete bins to estimate the full multidimensional joint probability distribution (p(X,Y) in Eq. 1) to estimate MI (Timme and Lapish, 2018). These two metrics estimate information across time samples and therefore are dependent on firing rate like the SMGM bits per second metric considered above. Further, the binned estimator is sensitive to the precise method used for data binning, and thus, we have used two commonly applied binning methods: uniform and occupancy-based bins.

We applied the KSG, binned estimator (uniform bins), and binned estimator (occupancy binned; Materials and Methods) to the same 10,000 mock GCaMP6f time-series traces and behavioral data used to assess the SMGM approach. These methods all behaved similarly when applied to our simulations (Fig. 4C–E), so they will be discussed together here. The information values measured by these techniques correlated with ground truth information in bits per second (ISE ). Interestingly, unlike the SMGM bits per second metric (IsF̂ ; Fig. 2), the KSG and binned estimator results were better fit with a saturating exponential than with a linear fit (χ12=1.70e4 , 3.12e4, and 3.14e4, respectively, p ∼ 0). The KSG and binned estimator methods overestimated the information at lower ground truth (IsE ) values and saturated quickly at higher values. For ground truth information (IsE ) values below 10 bits per second, the mean absolute errors were 10.3 ± 7.82 bits per second (465 ± 4390%), 22.2 ± 12.6 bits per second (821.7 ± 3353%), and 24.2 ± 14.0 bits per second (911 ± 3926%) for the KSG, binned estimator (uniform bins), and binned estimator (occupancy binned), respectively (Fig. 4F). This is in comparison to the 0.35 ± 0.59 bitsΔFFAP (14 ± 103%) mean absolute error found using the SMGM bits per second metric (IsF̂ ) for ground truth information values <10 bits per second. For ground truth information (IsE ) values >10 bits per second, the mean absolute errors were 13.2 ± 8.66 bits per second (85.0 ± 64.4%), 25.4 ± 16.2 bits per second (165.2 ± 118.3%), and 28.9 ± 17.34 bits per second (186.3 ± 124.9%) for the KSG, binned estimator (uniform bins), and binned estimator (occupancy binned), respectively. This is in comparison to the 0.97 ± 1.15 bits per second (5.85 ± 6.73%) mean absolute errors found using the SMGM bits per second metric (IsF̂ ) for ground truth information values >10 bits per second. Over the full range of ground truth values, mean absolute errors of 11.9 ± 8.42 bits per second (254.8%), 23.5 ± 14.8 bits per second (456.7%) and 26.7 ± 16.1 bits per second (509.0%) were found for the KSG, binned estimator (uniform bins), and binned estimator (occupancy binned), respectively, an order magnitude larger than the 2.4 ± 2.97 AU (22 ± 27% error) error seen using the SMGM bits per second metric (IsF̂ ). As a control, we applied the binned estimator s to AP traces and compared the estimated information to the ground truth information to verify that the large errors observed (Fig. 4D,E) were caused when the estimators were applied to fluorescence data (rather than simply a difference between the binned estimator, which do not rely on a Poisson firing assumption, and the ground truth information established using SMGM, which does rely on a Poisson firing assumption). We found the errors when applying the binned estimators to AP traces were relatively small [mean absolute error 2.72 ± 3.38 bits per second (41.1%) and 2.70 ± 3.33 bits per second (41.0%) for the uniform and occupancy-based binning; respectively; Extended Data Fig. 4-2]. Therefore, when comparing the recovery of ground truth information from functional fluorescence traces using either the SMGM metric IsF̂ or the KSG and binned estimator metrics, we found better recovery using the SMGM approach (IsF̂ ).

An analytic approximation can reproduce some qualitative, but not quantitative, results of the numeric solutions

Some of the general features of the relationship between ground truth information and fluorescence SMGM metrics can also be seen using an analytic approximation. For example, if we approximate the rate map as a Gaussian firing field with mean rate λ¯ , simplify the kernel approximation to a single exponential with falloff τ, assume constant, normalized movement speed v, and assume that the convolution between the kernel and the mean rate map is nearly Gaussian, we can approximate the relationship between the bits per second ground truth information ISE and measured fluorescence information ISF as ISF≈−Avτλ¯log(4−ISEλ¯+2πev2τ2)log(4), (9)and between the bits per AP ground truth information IAPE and measured fluorescence information IAPF as IAPF≈−log(4−IAPE+2eπv2τ2)log(4). (10)

Similar to the numerical solution, the analytic approximation provided by these equations (Eqs. 9, 10) predict that the fluorescence bits per second metric is dominated by a prefactor (Aτvλ⁻ in the analytical case), and that the fluorescence bits per AP metric saturates at larger information values. Our numerical solutions provide more accurate measures for the magnitude of these effects, and for the magnitude of information values themselves, given that they include the more accurate double exponential kernel, signal noise, and the realistic nonstationary speed, position and fluorescence signals. These quantitative differences can be seen in Extended Data Figure 4-3, where we directly compared this analytic approximation to our numerical approach by simulating 10,000 neurons with Gaussian rate maps (λ(x) ) with known (ground truth) information. We found a significant difference in the slope of the bits per second estimator (c, 0.041 vs 0.0036 ΔF/FHz for the numeric vs analytical, respectively), likely because of the non-stationarities present in behavior and florescence signals. For the fluorescence bits per AP measure, the analytic approximation predicts a large positive bias for ground truth values up to ∼4 bits/AP. This is in contrast to the numeric solution, which has <10% error for ground truth information values below 3.12 bits/AP.

Guidelines for application of information metrics to functional fluorescence imaging data

Taken together, the above results suggest that across the information metrics applied directly to functional fluorescence traces, the SMGM metrics provide the most reliable and interpretable information measurements. We thus suggest the following guidelines for use and interpretation of the SMGM metrics as applied to fluorescence MI metrics (IsF̂ and IAPF̂ ) defined in Equations 7, 8.

The SMGM bits per second metric (IsF̂ ) is likely attractive to imaging researchers because the units suggest that precise knowledge of AP numbers and times are not required for its use. However, there are several challenges when applying the SMGM bits per second metric to functional fluorescence imaging data. First, the substitution of the change in fluorescence map ( f) for the AP firing rate map ( λ) introduces a change in units, from bits per second to ΔFFbitsAP , which is difficult to interpret and relate back to bits per second. Second, the transformation of AP firing rate to change in fluorescence can be approximated by a c scaling factor (f=cλ ), which is measured in ΔFFHz , a quantity that is unknown a priori. If c is not consistent between the neurons of a population of interest, then the information values will be scaled differently and cannot be directly compared (Fig. 2). Since c is dependent on the width and height of the indicator response to a single AP (the kernel), it can vary from neuron to neuron based on difference in indicator expression level, intracellular calcium buffering, and many other factors (Park and Dunlap, 1998; Aponte et al., 2008; Helmchen and Tank, 2015; Greenberg et al., 2018). More research will be needed to measure these parameters (Chen et al., 2012, 2013), and thus c, across neurons. Some results suggest that there may be non-trivial amounts of variability within a populations of neurons (Greenberg et al., 2018; Éltes et al., 2019). With the impact of c on the SMGM bits per second metric, and the possible variability of c across a population of neurons, how can researchers properly extract useful measurements of information using IsF̂ ?

Example: application of information metrics to functional fluorescence imaging data from hippocampus during spatial behavior

In this section, we demonstrate use of the above guidelines for proper application and interpretation of the SMGM fluorescence MI metrics (IsF̂ and IAPF̂ ) defined in Equations 7, 8. We applied these metrics to functional fluorescence recordings (gCaMP6f) from pyramidal neurons in CA1 of the hippocampus acquired during mouse spatial behavior.

CA1 neurons expressing gCaMP6f (viral transfection, Camk2a promoter) were imaged with two-photon microscopy through a chronic imaging window during mouse navigation along a familiar 1D virtual linear track, as described previously (Fig. 5A,B; Dombeck et al., 2010; Sheffield et al., 2017; Radvansky and Dombeck, 2018). Eight fields of view from four mice were recorded in eight total sessions (recording duration 8.8 ± 1.3 min, number of traversals/session: 29 ± 2.5, 3.6 ± 0.3 laps/min, 3-m-long track). From these eight sessions, 1500 neurons were identified from our segmentation algorithm (see Materials and Methods), and analysis was restricted to the 964 neurons that displayed at least one calcium transient on at least 1/3 of the traversals during the session. Among these 964 neurons, 304 (31.5%) had significant place fields and were thus identified as place cells (see Materials and Methods), while the remaining 660 (68.5%) did not pass a place field test and were thus identified as non-place cells.

By applying Equation 7 (using 5-cm sized spatial bins), we found a continuum of spatial information values measured by the fluorescence SMGM bits per second metric (IsF̂ ) across the 964 CA1 neurons, with an average value of IsF̂=0.14±0.0040 bitsΔFFHz (Fig. 5C). The units for IsF̂ make direct use and interpretation of these values difficult, however, because these recordings were all from pyramidal neurons in a single area, here for illustrative purposes, we will presume that variations in c (discussed above) across the 964 neurons of interest are small and acceptable, with the absolute value of c unknown. This allows for comparisons of information ratios across different subsets of the population. For example, place cells had 2.8±0.20 times more information than non-place cells using the SMGM bits per second metric (IsF̂=0.18±0.0047 for place vs 0.063 ± 0.0029 bitsΔFFHz for non-place, rank-sum p = 1.7e-63; Fig. 5D–F), although there was substantial overlap in information between the populations (see distributions in Fig. 5D and individual examples in Fig. 5E,F). This also allows for accurate division of the 964 neurons into three quantiles based on information values, which we use below for spatial location decoding.

By applying Equation 8 (using 5-cm sized spatial bins), we found a continuum of spatial information values measured by the fluorescence SMGM bits per AP metric (IAPF̂ ) across the 964 CA1 neurons, with an average value of IAPF̂=1.65±0.023 bits/AP (Fig. 5C). The units for IAPF̂ allow for direct use and interpretation of these values, and notably, because most (97%) of the neurons had values <3 bits/AP, a mean absolute error of <10% can be assumed across the distribution of SMGM bits per AP values. When applied to the place and non-place cell populations, we found that place cells had higher information than nonplace cells using the fluorescence SMGM bits per AP metric (IAPF̂=1.8±0.026 and 1.35 ± 0.042 bits/AP, rank-sum p = 4.6e-21; Fig. 5D–F). This is consistent with mock fluorescence traces generated from real neuron AP datasets (Extended Data Fig. 5-1B).

As a demonstration of the usefulness of using information metrics to analyze large functional fluorescence population recordings, we explored the accuracy of decoding the animal’s track position using different subsets of neurons. We divided the 964 neurons into nine groups: all neurons, place cells, non-place cells, three quantiles based on the fluorescence SMGM bits per second metric, and three quantiles based on the fluorescence SMGM bits per AP metric. We then used a Bayesian decoder of the animals’ position (see Materials and Methods) separately for each of the nine neuron groups in each of the eight sessions (Fig. 5G,H). An individual session decoding example can be seen in Figure 5G. We quantified decoding accuracy using the absolute position decoding error (% of track), and pooled this measure across sessions for each neuron group (Fig. 5H). The means and standard errors for each group are: all neurons (7.33 ± 2.5%), place cells (6.97 ± 1.9%), non-place cells (20.9 ± 1.8%), SMGM bits per second Q1 (21.9 ± 1.5%), SMGM bits per second Q2 (13.2 ± 2.4%), SMGM bits per second Q3 (8.97 ± 2.4%), SMGM bits per AP Q1 (17.6 ± 2.7%), SMGM bits per AP Q2 (17.8 ± 3.1%), SMGM bits per AP Q3 (10.4 ± 3.0%). Interestingly, even the lowest quantile information groups still could be used to determine animal track location to within ∼1/5 of the track. This supports the idea that the hippocampal code for space is carried by a large population of active neurons (Meshulam et al., 2016), and not just by a select subpopulation with the highest information or most well-defined tuning curves. As could be expected, place cells encoded the position of the animal better than nonplace cells and better than the lowest quantile information groups (Holm–Bonferroni corrected rank-sum, α=0.05 ), and neurons in the higher quantiles provided more accurate decoding. Thus, the fluorescence information metrics provide a means to compare the relative contribution of hippocampal neurons with different information values to decoding animal position.