Humans Can Track But Fail to Predict Accelerating Objects

Target presentation duration affects ability to track but not to predict accelerating targets

Observers tracked the target with a combination of smooth pursuit and saccadic eye movements during the initial period, in which the target was visible. Two example trials, in which the target either accelerated (Fig. 2A) or decelerated (Fig. 2B) show typical eye movement position and velocity during the task. After pursuit initiation, an initial catch-up saccade aligned the eyes with the target and was typically followed by a period of closed-loop smooth pursuit, during which eye velocity matched the continuously changing target velocity. In some trials, smooth pursuit was supported by additional catch-up saccades (Fig. 2B). Around the time of target occlusion, observers stopped smoothly tracking the target and pursuit velocity decreased to 0°/s. Observers then made a distinctly identifiable predictive saccade of relatively large amplitude to the right border of the occluder, where they then intercepted the target with a pointing movement of their right index finger (see 3D hand trajectory from a single trial in Fig. 2C). To investigate how finely-tuned these different eye movement responses are to target acceleration, we analyzed the effect of presentation duration on the ability to smoothly track the visible, accelerating target and to predict its time of reappearance (TTC) with a predictive saccade.

Observers’ ability to accurately and smoothly track the accelerating target increased with increasing presentation duration, here taken as a direct measure of the availability of target motion signals (Fig. 3A,B). Whereas no difference in average pursuit velocity was observed in the shortest presentation duration (200 ms), average pursuit velocity sensitively reflected different target velocities for both longer presentation durations (500, 800 ms; Fig. 3A). This observation was confirmed by a 5 (acceleration) × 3 (presentation duration) rmANOVA on average pursuit velocity, yielding a significant acceleration × presentation duration interaction (F_(8,120) = 18.52; p < 0.001; η_p² = 0.55). Both main effects of presentation duration (F_(2,30) = 184.76; p < 0.001; η_p² = 0.92) and target acceleration (F_(4,60) = 40.75; p < 0.001; η_p² = 0.73) reached significance. We performed follow-up, one-way rmANOVA with factor acceleration for each presentation duration and found a significant main effect of acceleration in both the 500- and 800-ms conditions (F_(4,60) = 8.31; p < 0.001; η_p² = 0.36 and F_(4,60) = 50.08; p < 0.001; η_p² = 0.77, respectively), but not in the 200-ms condition (F_(4,60) = 1.05; p = 1; η_p² = 0.07).

We next asked whether observers continuously tracked the changing target velocity over time. To this end we normalized pursuit velocity traces in the acceleration conditions relative to the control condition (0°/s²) for each observer. This additional analysis accounts for imperfect pursuit velocity gain and individual differences.

The normalized pursuit velocity over time revealed how closely observers’ pursuit velocity matched the continuously changing target velocity for the two longer presentation durations (Fig. 3B). After the first catch-up saccade, the eye continuously accelerated in response to accelerating targets and decelerated in response to decelerating targets. These findings show that visually-guided pursuit closely matches the continuously changing target velocity, and that these effects are amplified with longer presentation duration where differences in target velocities were more pronounced. In contrast, pursuit velocity was similar across target accelerations in the 200-ms condition. This similarity might have been because of the small differences in the velocity of a target shown only very briefly.

Next, we asked whether the ability to accurately track the accelerating targets with increased presentation duration also affected the ability to predict its reappearance (TTC) with a predictive saccade. Predictive saccades are typically made several hundred milliseconds before target reappearance or interception. Because their timing is tuned to expectations of target motion (Diaz et al., 2013) and reflect decision outcomes in manual interception (Fooken and Spering, 2020), they can provide a sensitive indicator of target motion prediction. Although observers were not instructed to make such a predictive saccade, it was clearly identifiable in virtually every trial (>99% of all trials). These saccades were initiated on average 137 ms (±71 ms; mean ± 1 SD across observers; Fig. 3A) after occlusion onset and landed clustered around the right border of the occluder (Fig. 3C) ∼262 ms (±70 ms) after occlusion onset. In most trials, observers made one large predictive saccade with an average amplitude of 12.1° (±0.8°) across observers and trials, and this average amplitude was similar across acceleration conditions. On average, saccades were initiated 1.2° (±0.8°) from within the occluder and landed close to the occluder’s right border at 13.2° (±0.3°; Figs. 2A,B, 3C, upper panels). In trials where a second predictive saccade was made to correct for undershooting, initiation and landing time of the first predictive saccade were substantially later (319 ± 94 and 414 ± 93 ms, respectively) than in trials with only one predictive saccade.

Interestingly, the landing time of predictive saccades (SLT) did not scale with target acceleration, i.e., the eye did not land earlier at the border of the occluder for accelerating targets and did not land later for decelerating targets. Instead, the SLT showed a consistent bias in the opposite direction, i.e., later for accelerating targets and earlier for decelerating targets. This observation was confirmed by a main effect of target acceleration (F_(4,60) = 8.17; p = 0.008; η_p² = 0.35; Fig. 3D) in a 5 (acceleration) × 3 (presentation duration) rmANOVA. Note that this small bias was also present in the predictive saccade initiation time (Fig. 3A) and was neither caused by differences in saccade duration nor in amplitude across acceleration conditions. We also found a main effect of presentation duration on SLT of the predictive saccades (F_(2,30) = 12.54; p = 0.003; η_p² = 0.46). Saccades landed later in the 200-ms condition, compared with the other presentation durations (Fig. 3D). Despite the short presentation duration of 200 ms, observers typically made one early catch-up saccade followed by a predictive saccade (Fig. 3A, upper panel). This often delayed the onset of the predictive saccade and observers tracked the target until shortly after target occlusion onset. In addition, in a subset of trials (22.7%) in the 200-ms condition, observers only made one saccade. These saccades had slightly larger amplitudes, which might have contributed to the later SLTs in the 200-ms condition. There was no significant interaction between target acceleration and presentation duration (F_(8,120) = 1.31; p = 0.284; η_p² = 0.08). To further analyze which acceleration conditions were significantly different from one another, we performed pairwise comparisons after averaging SLTs across presentation durations. Post hoc t tests revealed significant differences between the −8°/s² and the +4°/s² as well as the +8°/s² conditions (t₍₁₅₎ = −4.04; p = 0.01 and t₍₁₅₎ = −3.35; p = 0.044, respectively), between the −4°/s² and +4°/s² (t₍₁₅₎ = −3.90; p = 0.001), and between the 0°/s² and +4°/s² (t₍₁₅₎ = −3.29; p = 0.050) acceleration conditions. All other comparisons did not reach significance (all p > 0.10).

Taken together, these findings show that predictive saccades appear to follow a bias in the opposite direction to what we would expect if target acceleration was used to estimate TTC. Moreover, the landing time of predictive saccades was relatively less affected by the presentation duration of the target than what was observed for visually-guided pursuit. Next, we asked whether observers considered target acceleration to time their interceptive hand movement.

Target acceleration causes systematic manual interception errors

To intercept the target on the screen, observers moved their hand from the designated start position on the table to the screen (Fig. 2C). On average, hand movement paths did not differ between the different target acceleration conditions (Fig. 4A). Because of the long occlusion times, hand movements were largely executed during the occlusion and were thus not systematically corrected mid-flight. As instructed, observers intercepted the target close to the occluder’s right border (mean distance 1.7° ± 1.1°; Fig. 4B). Distributions of horizontal interception positions largely overlapped across target acceleration conditions (Fig. 4C). Yet, there was a small tendency to hit further to the right for accelerating targets (main effect of acceleration on horizontal interception position: F_(4,60) = 9.21; p = 0.006; η_p² = 0.38). This effect was primarily driven by the +8°/s² condition because targets often reappeared before interception in this condition (Fig. 5B, green data points). In these trials, observers tended to adjust the interception position, resulting in a more skewed distribution for the +8°/s² condition (Fig. 4C, bottom panel).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Hand movement paths and interception position. A, Top view of the hand movement paths in the x-z plane. The gray bar illustrates the x-position of the occluder on the screen. Hand movement paths were shifted along the z-axis for better visibility. B, Distribution of 2D interception positions. Histogram shows the distribution of horizontal interception positions. C, Kernel density plots of horizontal interception positions for the different target acceleration condition.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Manual interception time. A, Mean TTC_hand (interception time relative to occlusion onset). B, Mean and individual observers’ constant interception errors. Semi-transparent dots represent individual observers’ median performance. Negative values indicate interceptions that occurred before the target reached the end of the occluder (too early), and positive values indicate interceptions occurring after the target reached the end of the occluder (too late). Error bars represent ± 1 SEM.

In parallel to investigating effects of acceleration on the predictive saccade, we next analyzed the effect of target acceleration on the timing of the interceptive hand movement (TTC_hand). If observers accounted for target acceleration when timing their hand movement, we would expect them to intercept earlier for accelerating targets and later for decelerating targets, relative to the zero-acceleration (control) condition. In contrast to this hypothesis, we found that observers intercepted later in response to target acceleration and earlier for deceleration (main effect of acceleration: F_(4,60) = 10.80; p = 0.002; η_p² = 0.42; Fig. 5A). This effect was more pronounced in the 500- and 800-ms presentation duration conditions, confirmed by a significant acceleration × presentation duration interaction (F_(8,120) = 3.15; p = 0.037; η_p² = 0.17). Accordingly, main effects of acceleration were found for both the 500-ms (follow-up, one-way rmANOVA: F_(4,60) = 10.61; p < 0.001; η_p² = 0.41) and 800-ms (F_(4,60) = 5.94; p = 0.048; η_p² = 0.28) conditions but not for the 200-ms condition (F_(4,60) = 2.62; p = 0.13; η_p² = 0.15).

Observers also tended to hit the targets later in the 200-ms condition compared with the 500- and 800-ms conditions, as indicated by a main effect of presentation duration (F_(2,30) = 16.21; p < 0.001; η_p² = 0.52).

The failure to take acceleration into account to time the interceptive hand movement resulted in systematic temporal constant interception errors. Relative to target reappearance, observers intercepted too late (i.e., the target had already reappeared) for accelerating targets and too early for decelerating targets (i.e., the target had not yet reappeared). The main effect of target acceleration on the constant interception error was significant (F_(4,60) = 492.61; p < 0.001; η_p² = 0.97; Fig. 5B). We also observed a main effect of presentation duration (F_(2,30) = 16.47; p < 0.001; η_p² = 0.52), which was caused by a general tendency to intercept targets later in the 200-ms condition (see also Fig. 5A). Although, there was a significant acceleration × presentation duration interaction term (F_(8,120) = 3.28; p = 0.032; η_p² = 0.18), follow-up, one-way rmANOVA showed main effects of target acceleration on the constant interception error for all three presentation durations (all p < 0.001).

Overall, observers were not able to accurately adjust the timing of their interceptive hand movement (TTC_hand) to target acceleration, causing systematic constant interception timing errors. Notably, the observed opposite bias in interception time is similar as the bias found for predictive saccades (Fig. 3D): the eye lands later and the finger intercepts later for accelerating targets, and both eye and hand intercept earlier for decelerating targets.

We found that interception time (TTC_hand) and SLT followed strikingly similar biases opposite to what we would expect if observers accounted for target acceleration. Given that the eye landed at the location of subsequent target reappearance several hundred milliseconds before the hand, we next asked whether SLT was a predictor of TTC_hand on a trial-by-trial basis. Trial-by-trial correlations imply a similarity in the trial-based variability between eye and hand movements and are interpreted as evidence for common information sources in the signals that drive eye and hand movements (Sailer et al., 2000). Using a linear mixed model, we found that SLT indeed significantly predicted TTC_hand on a trial-by-trial basis (b = 0.351; F_(1,15.73) = 66.96; p < 0.001). Note, however, that the relation between SLT and TTC_hand varied substantially across observers, indicated by individual trial-by-trial Person’s correlations ranging from r = 0.20 to r = 0.57 (Fig. 6A–C).

• Download figure
• Open in new tab
• Download powerpoint

Figure 6.

Trial-by-trial correlation across acceleration conditions and presentation durations between SLT and TTC_hand. A, Distribution of individual correlation coefficients. B, C, Scatterplot and trend lines of the trial-by-trial correlation for an observer with a strong (B) and for an observer with a weak (C) correlation between SLT and TTC_hand. Dots represent individual trials. Thin lines represent trend lines for the different acceleration conditions, thick line shows the trend across conditions.

Our findings of similar biases in predictive saccades and prediction-based interceptive hand movements and of a medium trial-by-trial correlation between saccade and hand movement timing suggest that both systems relied on similar motion prediction. Notably, despite the ability to closely track accelerating objects with visually-guided pursuit, predictive eye and hand movements appear to be insensitive to target acceleration, raising the question which target features were used to predict TTC for manual interception.

Which target features determine TTC predictions?

The observed constant interception errors and the systematic biases in TTC_hand (later interception for accelerating targets and earlier interception for decelerating targets) suggest that observers were not able to correctly adjust their interception timing according to target acceleration. We next asked which target features observers used instead to estimate TTC for manual interception. We compared three competing models, describing which target motion signals observers might have used to estimate TTC of accelerating targets (Bennett et al., 2007; Heinen, 2007; Fig. 7A–C).

• Download figure
• Open in new tab
• Download powerpoint

Figure 7.

Model comparison. A–C, Three competing models of how observers might predict TTC for predictive eye and interceptive hand movements. A, The final velocity model postulates that observers predict TTC based on v_final (identical for all targets in our design, hence predicting a fixed TTC). B, The average velocity model predicts interception timing based on the average target velocity before occlusion, yielding a negative correlation between veridical and measured TTC. C, The acceleration model suggests that observers use target acceleration for interception and predicts the veridical TTC. D, Comparison of model predictions and measured TTC_hand data. Dashed line shows the TTC prediction based on the mean hand prediction times. E, Root mean squared errors for the competing models on TTC_hand. F, Individual and mean hand prediction times. Asterisks represent significant differences (p < 0.05).

As one possibility, observers could continuously update target velocity and estimate TTC based on the last available target velocity sample before occlusion (final velocity model; Fig. 7A). Alternatively, observers might use the average velocity during the visible period to estimate target reappearance (average velocity model; Fig. 7B). Finally, if observers indeed considered target acceleration, we would predict accurate scaling of the estimated TTC with target acceleration (acceleration model; Fig. 7C). We compared the TTC predictions of the different models to the observed biases in TTC_hand (Fig. 7D) and quantified the performance of each model by calculating the RMSE for each observer.

Using the acceleration model to predict TTC data in eye and hand interception confirms that interception does not use acceleration (Fig. 7D, bright gray line). Although the average velocity model captures the small, reversed trend we observed in TTC_hand (Fig. 7D, dark gray line), this model performs poorly at predicting the measured TTC_hand. The final velocity model produced the lowest RMSEs (Fig. 7D, black line). These observations were confirmed statistically by a main effect of model on RMSEs in a one-way rmANOVA (F_(2,30) = 18.00; p < 0.001; η_p² = 0.55) and post hoc pairwise comparisons. These showed that the final velocity model produced significantly lower RMSEs compared with the acceleration (t₍₁₅₎ = 9.64; p < 0.001; d = 2.41) and average velocity models (t₍₁₅₎ = 3.46; p = 0.011; d = 0.86; Fig. 7E).

The final velocity model predicts that observers rely on the last available velocity sample. To pinpoint the approximate time sample observers relied on when estimating TTC, we determined the time point along each target’s velocity trajectory that best accounted for each observer’s bias in TTC_hand. We termed this the hand prediction time. A negative value indicates that the observer based the TTC prediction on a velocity sample before occlusion, whereas a positive value would indicate that the TTC prediction was based on a partial extrapolation of the veridical target trajectory during occlusion. We found an average hand prediction time of −96.4 ms (Fig. 7F), which captures the small opposite bias we observed in TTC_hand (Fig. 7D, dashed in line).

Final velocity model prediction generalizes to different target configurations

Our model comparison indicates that observers continuously updated their prediction of target velocity until shortly before occlusion. One possible shortcoming of our experimental design was that the target moved at the same final velocity in all acceleration conditions. This might have induced a bias to always hit the target at the same time, favoring the final velocity model. Moreover, our data also showed a small bias toward the average velocity model. We conducted a control experiment to address whether the predictions of the final velocity model hold true when targets moved with different final velocities. In the control experiment, all targets moved with the same average velocities and different final velocities (Fig. 1C). Importantly, these target configurations predict the same difference in RMSEs between the average and final velocity models as in experiment 1, allowing us to directly compare the model fits between the two experiments. Note that in contrast to experiment 1, the average velocity model here predicts no TTC adjustments, whereas the final velocity model predicts earlier TTC for accelerating targets and later TTC for decelerating targets (Fig. 8A,B).

• Download figure
• Open in new tab
• Download powerpoint

Figure 8.

Results from experiment 2. A, B, The model predictions of the final (A) and average velocity (B) models in experiment 2. C, Comparison of TTC_hand to the model predictions. E, Comparison of model fits. Asterisks represent significant differences (p < 0.05).

In line with the prediction of the final velocity model, we found that observers intercepted accelerating targets earlier (Fig. 8C, negative TTC values) and decelerating targets later (Fig. 8C, positive TTC values). This observation was supported by a significant one-sided paired t test, testing whether the final velocity model produced significantly lower RMSEs compared with the average velocity model (t₍₉₎ = 2.12; p = 0.031; d = 0.67; Fig. 8D). We also estimated the target velocity sample that best predicted observers’ TTC bias. We found a mean hand prediction time of 65.8 ms before occlusion onset, which was similar but slightly smaller than the hand prediction time in experiment 1. Together, these results replicate our findings from experiment 1 and confirm the use of the final target velocity to predict TTC.