Speed Estimation for Visual Tracking Emerges Dynamically from Nonlinear Frequency Interactions

Rationale for probing motion integration properties for ocular tracking. a, Motion stimuli are defined by their mean spatial and temporal frequencies in Fourier space. A set of 15 component stimuli were generated, paving the spatiotemporal frequency space (1 to 15 correspond to c1 to c15 detailed in Table 1). The component motion inputs were selected to cover three ranges of spatiotemporal frequency scales (low, medium, high, with a red-green-blue color code). They were also aligned (in groups of 3) along five different speed axes (from 11°/s to 53°/s, hue saturation code). b, Ocular tracking responses to either a component Motion Cloud (MC, top) or a component Drifting Grating (DG, bottom) of same mean spatiotemporal frequencies and speed (stimulus c1 in panel a). Gray curves are all single trials. Green curves are average across ∼150 valid trials, for participant S01. c, Pattern stimuli were generated by summing several (2 or 3) of these components. For instance, a triplet can be oriented along the iso-velocity line or along the scale axis. Components were built from either component DG or component MC. Notice that both pattern stimuli have the same mean spatial and temporal frequency as well as the mean speed (here 24°/s). d, Individual and mean eye velocity profiles of tracking responses to either pattern stimulus. Continuous blue/orange lines are the observed mean eye velocity profiles. Broken lines correspond to the linear prediction, computed as the mean of the responses to each component.

Dynamics of spatiotemporal frequency tuning of ocular tracking responses. a, Mean eye velocity profiles of tracking responses to each of the component MC (continuous lines) or DG (dotted lines). At each location in the spatiotemporal frequency space, grating, and MC stimuli have identical mean spatial frequency, temporal frequency, and speed, but different spreads around these means. b, Early (left) and late (right) temporal frequency (TF) tuning of ocular tracking responses to either component MCs (top) or DG (bottom). Data are mean (±SD) eye velocities across participants. c, Same plots but for spatial frequency (SF) tuning.

a, Distribution of 15 component DG and MC, relative to both the scale and speed axes (same as Fig. 1a with added labels for speed and scale). b, Mean eye velocity profiles for component MC (top) and DG (bottom) of mean speed increasing from 11°/s to 56°/s, but sharing the same medium scale range. c, Mean (±SD across participants) eye velocity, as a function of stimulus speed, for three different scale ranges. d, Mean eye velocity profiles for component MC (top) and DG (bottom) of identical mean speed but increasing scale range. e, Scale tuning of ocular tracking, when presented with either component DGs or MCs. Same color hue/saturation code.

Temporal dynamics of ocular tracking tuning. a, Spatiotemporal tuning of response amplitude computed across the 15 component DG for a given temporal window (here [250–300 ms]). The spatiotemporal tuning was approximated by a quadric surface (surface plot on the left, contour plot on the right). From these quadratic fits, we extracted the main axis (blue curve) that tracks pairs of spatiotemporal frequency coordinates along which the amplitude changes the least for each scale within the fitted range. We also estimated the maximum scale-speed axis (red curve) by collecting the scale corresponding to the maximum eye velocity for each stimulus speed. b, Best fitted spatiotemporal tuning surfaces for component MC stimuli computed across six participants and for four successive time windows, from [100–150 ms] to [250–300 ms] after stimulus onset. c, Same as in b but for component DG stimuli. d, For each participant, the angle Θ of the maximum scale-speed axis is plotted for five time windows, for both DG (pink) and MC (green) stimuli. Larger dots with error bars show mean ± SD (across participants). The first time window is indicated as baseline, before response onsets. e, For each participant, the angle Θ of the main axis of the spatiotemporal tuning surface is plotted for five time windows.

a, b, Best fit 2D polynomial function describing the relationship between ocular following amplitude and both spatial and temporal frequencies. Data are mean response amplitudes across six participants. a, Component DGs. b, Component MCs. c, Q index can be derived from the best-fit 2D polynomial functions, as indicated with the two equations. Individual Q indices were computed for each participant, and each time window. Individual and mean (±SD, across participants) are plotted. Note that one outlier (MC condition) with a Q value of ∼22 at time 150 ms is not plotted to allow comparison between the two conditions.

Variability across trials of tracking responses. a, Distributions of early eye velocities, for participant S02. Data are shown for three mean speeds (from left to right) and for each speed, with three different spatiotemporal frequencies and therefore three different scale ranges (color code). Motion stimuli were component MCs (top) or DGs (bottom). Continuous curves are best-fit Gaussian functions. b, Relationship between best-fit parameters σ and μ. Data are mean (±SD) across six participants for each of the 15 motion components, for either MCs (top) or DGs (bottom). Color code indicates the scale range of each data point. From left to right, plots illustrate results obtained for the five successive time windows. The first time window ([50–100 ms], gray shaded) provides an estimate of baseline variability, as most of this measurement window covered preresponse time. Continuous black lines indicate the theoretical relationship where σ increases with the square root of the μ value.

Speed or scale tuning of tracking responses variability. CV values estimated for ocular responses to component MCs or DGs are plotted one against the other, for six time windows, starting at 50 ms (left column) and spanning 50 ms each. The first time window ([50–100 ms]) estimates the relative variabilities over a baseline, preresponse epoch (gray shaded). a, Each dot corresponds to a given component stimulus pair, for all participants. Colors correspond to the scale range. b, CV values are averaged across speeds, and participants, for each of the three scale ranges. Continuous contour lines enclose the variance across participants. c, CV values are averaged across scales, and participants, for each of the stimulus speeds.

Tracking responses to pattern DGs (pDG) and MCs (pMC). a, Top, Three examples of triplets, aligned along the scale axis (pattern d), the speed axis (pattern i), or in between (pattern e). Next two rows illustrate mean eye velocity profiles to these pattern MCs (middle, continuous lines) or DGs (bottom, dotted lines). Colored lines show the observed response profiles for participant S04. Gray lines show the linear prediction computed by averaging the responses to each component of the triplet. b, Top and Bottom Rows, Component distributions of the 15 patterns (from a to i), in the spatiotemporal space, for both pMCs and pDGs, respectively. Different orientations and distances between components were used. c, The two rows plot, for each pattern type, the ratio between observed and predicted mean eye velocities, over time. A ratio of 1 indicates that observed responses are equal to the linear prediction. Values above or below 1 indicate the observed responses are larger or smaller than predicted, respectively. Data are mean (±SD) across six participants.

Latency of nonlinear effects. a, Two examples of tracking responses for one representative participant, for pattern e composed of three components, either MC (upper plot) or DG (lower plot). Gray lines plot the theoretical eye velocity profiles obtained by averaging the tracking responses to each component, independently. Vertical dotted lines indicate the first point in time (target motion onset) at which observed and predicted tracking responses are significantly different. b, The time of separation between predicted and observed eye velocity profile is plotted for each participant. For comparison, similar patterns MC (square) and DG (circle) are shown together. Larger symbols with error bars are mean (±SD) across six participants for one given condition. The gray shaded area indicates the mean open-loop period of tracking responses. c, Distributions of separation times, across nine patterns and six participants, for both pattern DG (red) and MC (green) conditions. Closed red circles and green squares plot the median values (±IQR, that is the interquartile range) of separation times.

Model. a, Tiling of spatiotemporal channels. Each channel is a bivariate normal function with its main axis oriented along a line of constant speed. All channels coding for the same speed are shown by the same color: blue for slow speed and pink for fast speed. The colored dots represent the center of the channels. The black dots represent the coordinates of the component stimuli. b, Receptive field of a channel. The contour lines show the receptive field in log-log Fourier space of a channel centered at 0.5 c/° and 11.31 Hz. The white dots represent the coordinates of the component stimuli. c, Speed tuning of a channel. Each curve shows the speed tuning of the channel in b at one spatial frequency; different colors indicate different spatial frequencies and the colors of the curves match the vertical lines in b. d–g, Representation of different stimuli types in the log-log Fourier space. Component DG 1 (d), PDG d (e), component MC 1 (f), and PMC d (g). h–k, Normalized responses of all channels of the network to the respective stimuli of d–g.

Model performance. a, Distribution of channel weights across the network. b, Best-fit interaction pattern. c, Comparison of responses probabilities across time windows between experimental and model data, for one participant (S03). d, Ratios between observed and predicted mean eye velocities (plotted on a Log scale) to pattern stimuli over time plotted for experimental data, the model with interaction and the model without interaction. Colors and labels are identical to Figure 6.

Comparison between data and model for component stimuli. a, Distribution of observed (data) and predicted (model) eye velocity of ocular responses to a component MC moving at 24°/s, for participant S04. Color indicates scale range. First and second row illustrate early ([150–200 ms]) and late ([250–300 ms]) time windows. b, Same plots, but now for component DG of the same mean spatial and temporal frequency and mean speed (24°/s). c, d, Relationship between predicted and observed CV (coefficient of variation) across the three scales (color) and five speeds (saturation), for three windows. Insets plot the same relationships between model and data but now for mean eye velocities. Oblique lines indicate a slope of 1 for the linear regression.