Abstract
Objects in our visual environment often move unpredictably and can suddenly speed up or slow down. The ability to account for acceleration when interacting with moving objects can be critical for survival. Here, we investigate how human observers track an accelerating target with their eyes and predict its time of reappearance after a temporal occlusion by making an interceptive hand movement. Before occlusion, observers smoothly tracked the accelerating target with their eyes. At the time of occlusion, observers made a predictive saccade to the location where they subsequently intercepted the target with a quick pointing movement. We tested how observers integrated target motion information by comparing three alternative models that describe time-to-contact (TTC) based on the (1) final target velocity sample before occlusion, (2) average target velocity before occlusion, or (3) final target velocity and the rate of target acceleration. We show that observers were able to accurately track the accelerating target with visually-guided smooth pursuit eye movements. However, the timing of the predictive saccade and manual interception revealed inability to act on target acceleration when predicting TTC. Instead, interception timing was best described by the final velocity model that relies on extrapolating the last available target velocity sample before occlusion. Moreover, predictive saccades and manual interception showed similar insensitivity to target acceleration and were correlated on a trial-by-trial basis. These findings provide compelling evidence for the failure of integrating target acceleration into predictive models of target motion that drive both interceptive eye and hand movements.
- acceleration
- eye-hand coordination
- manual interception
- prediction
- saccades
- smooth pursuit
Significance Statement
Acceleration is an essential feature of most moving objects in our environment, but the human visual system is surprisingly insensitive to acceleration. We investigated observers’ ability to track an accelerating and disappearing target with their eyes and to predict its time of reappearance by making an interceptive hand movement. Despite the ability to accurately track the accelerating target with their eyes, observers failed to consider acceleration when predicting target reappearance, resulting in systematic interception errors. The magnitude of the error can be explained by a model that describes interception timing based on extrapolation of the last available target velocity signal. Observers fail to account for acceleration during target interception and instead update target velocity while the target is visible.
Introduction
Seeing and perceiving object motion is a vital capability of the primate visual system. Animals hunting for prey or pedestrians crossing a street must be able to act on a target’s speed, direction, and sudden target accelerations or decelerations. Although acceleration is an important feature that describes the behavior of many moving objects, it is well established that the primate perceptual system is largely insensitive to it (Gottsdanker et al., 1961; Calderone and Kaiser, 1989; Werkhoven et al., 1992; Brouwer et al., 2002; Benguigui et al., 2003; Watamaniuk and Heinen, 2003; Mueller et al., 2017). How we incorporate visual acceleration signals into motor commands to interact with moving objects is still not fully understood. Here, we evaluate human observers’ ability to track accelerating targets with their eyes and to predict accelerating target trajectories for manual target interception.
Tracking visual object motion with the eyes and predicting an object’s motion path are two fundamental abilities that rely on decoding visual motion (Fiehler et al., 2019) and can inform interceptive hand movements (Mrotek and Soechting, 2007; Diaz et al., 2013; Mrotek, 2013; Cesqui et al., 2015; Fooken et al., 2016, 2021; De la Malla et al., 2017; Goettker et al., 2019; AJ De Brouwer et al., 2021). Whereas tracking relies heavily on visual signals (Lisberger, 2015), predicting a trajectory also requires memory of previously seen motion (Orban de Xivry et al., 2013; Kowler et al., 2019; Rust and Palmer, 2021).
When tracking moving objects, humans rely on smooth pursuit eye movements, slow rotations of the eyes, to keep the object close to the fovea. Neurons in motion-sensitive extrastriate cortex (area MT) provide the sensory input that drives pursuit. Neurophysiological studies in macaque monkeys found that single neurons in MT are not tuned to a target’s acceleration (Lisberger and Movshon, 1999); instead, acceleration can be decoded indirectly from populations of speed-sensitive MT neurons (Lisberger and Movshon, 1999; Price et al., 2005). These acceleration signals can drive pursuit eye movements in monkeys (Krauzlis and Lisberger, 1994; Churchland and Lisberger, 2001).
Evidence for the use of acceleration signals for human pursuit comes from studies investigating eye movements in response to small perturbations in target velocity (Tavassoli and Ringach, 2010; Brostek et al., 2017) and those investigating eye movements during temporary target occlusion (Bennett et al., 2007). When tracking an accelerating target throughout a brief occlusion period, eye movements (pursuit and saccades) scale with object acceleration in anticipation of target reappearance (Bennett and Barnes, 2006; Bennett et al., 2007; Bennett and Benguigui, 2013). These findings indicate that the pursuit system can extract target acceleration and use this signal to predictively drive continuous tracking eye movements. Importantly, the ability to extract target acceleration signals improves with presentation times (Bennett et al., 2007), indicating that the pursuit system requires a relatively long initial exposure to target acceleration to form an acceleration-based prediction of target motion. However, whether this ability translates to tasks, in which observers predict the reappearance of accelerating objects, is unclear.
Tasks in which observers are asked to intercept accelerating objects with a quick pointing movement, a projectile, or by pressing a button reveal systematic errors, indicating inability to consider target acceleration for action-related motion prediction (Port et al., 1997; Dubrowski and Carnahan, 2002; Benguigui et al., 2003; Benguigui and Bennett, 2010; Bennett and Benguigui, 2013; Brenner and Smeets, 2015; Brenner et al., 2016). These errors were caused by observers’ failure to adjust the timing or position of their hand movement to the target’s acceleration, resulting in interceptions that are too early and ahead of the target when it decelerates, and interceptions that are too late and behind the target when it accelerates.
Whereas eye movements to accelerating targets appear to be responsive to acceleration, hand movements are prone to systematic errors that indicate inability to account for acceleration. Here, we directly compare eye movements and manual interception of accelerating targets in a track-intercept task. We varied the initial target presentation duration and the rates of target acceleration to test observers’ sensitivity to track the accelerating target with their eyes and to predict its time-to-contact (TTC) for manual interception. We hypothesized that observers make systematic interception errors, indicating inability to account for target acceleration. Moreover, we predicted an improved ability to track target acceleration with longer presentation durations, which would also result in better adjustments of interception timing.
Materials and Methods
We report the results of one main experiment (experiment 1) and one control experiment (experiment 2). Apparatus, procedures, and analyses were identical between both experiments (unless otherwise stated). The critical difference between experiments was that all targets reached the occluder with the exact same final velocity in experiment 1 (Fig. 1B), whereas in experiment 2, all targets moved with the exact same average velocity but different final velocities (Fig. 1C). The study design and parts of the analyses of experiment 1 were preregistered (https://osf.io/adq9v)
Observers
Experiment 1
We tested 16 human adults (seven females; mean age 26.8 ± 5.1 years, range 19–37 years; including two authors) in this study. All participants had normal or corrected-to-normal visual acuity and reported no history of neurologic, psychiatric, or eye disease. The study protocol was conducted in accordance with the Declaration of Helsinki and was approved by the local Behavioural Research Ethics Board. Participants gave written informed consent before participating and were compensated at the rate of $10/h.
Experiment 2
Ten healthy adults were recruited for participation in the control experiments (six females; mean age 28.7 ± 6.7 years, range 21–45 years; four of whom also participated in experiment 1; two authors). Experiment 2 was designed to replicate the model comparison from experiment 1. Specifically, based on experiment 1, we hypothesized that the final velocity model would produce smaller errors, compared with the average velocity model. We used the effect size from experiment 1 (d = 0.86) to determine our sample size of n = 10 using an a priori power analysis in G*Power (Faul et al., 2009; one-sided, paired t test; power = 0.80; α = 0.05). Observers gave written informed consent and were compensated at the rate of $10/h.
Apparatus
Participants performed the task in a dimly lit laboratory, viewing the stimuli binocularly at a distance of 44 cm. A PROPixx video-projector with a resolution of 1280 × 1024 pixels and a refresh rate of 120 Hz (VPixx Technologies) was used to back-project the stimuli onto a 41.8 × 33.4 cm translucent screen. The position of participants’ right eye was recorded using a video-based eye tracker (Eyelink 1000 Tower Mount, SR Research) with a sampling rate of 1 kHz. A combined chin and forehead rest minimized head movements during the experiment. A small magnetic sensor was attached to the tip of participants’ right index finger to record their 3D hand movements with a 3D Guidance trakSTAR (Ascension Technology) at a sampling rate of 120 Hz. The experiment was programmed in MATLAB (MathWorks), using the Psychophysics toolbox (version 3.0.16; Brainard, 1997; Pelli, 1997; Kleiner et al., 2007) and EyeLink toolbox (Cornelissen et al., 2002). Stimulus display and data collection were controlled by a PC (graphics card: NVIDIA GeForce GTX 1060).
Stimuli and procedure
Experiment 1
Participants viewed and intercepted a small black disk (0.35° in diameter; 6.23 cd/m2) that moved across a light gray background (229.8 cd/m2) and then disappeared behind an occluder, a gray (181.3 cd/m2) bar, that extended 13.4° from the horizontal midline into the right half of the screen. Each trial started with the disk shown on the left side of the screen. Participants had to fixate the target (400–800 ms) and place their index finger on a designated start position on the table located 36 cm below and 23 cm in front of the screen center (Fig. 2C). Upon successful fixation, the target started moving to the right, either with a fixed velocity (0°/s2) or constantly accelerating at different rates (−8, −4, 4, 8°/s2). The target was shown for 200, 500, or 800 ms before occlusion. Participants were instructed to follow the disk closely with their eyes during the initial presentation and to manually intercept the target at the time they expected it to reappear behind the occluder. Target presentation ended either with the time of interception, or 100 ms after target reappearance. Initial target position and velocity were matched for each presentation time and acceleration so that all targets reached the same position and velocity at the time of occlusion. The time of target reappearance (equivalent to TTC) depended on the target’s acceleration rate (−8°/s2: 797 ms, −4°/s2: 722 ms, 0°/s2: 670 ms, +4°/s2: 630 ms, +8°/s2: 598 ms). Therefore, successful interception required adjusting the timing of manual interception to target acceleration. Observers were instructed to intercept as closely to the border of the occluder as possible at the time they expected the target to reappear. Feedback about the interception performance was provided at the time of interception by showing a red and a black dot, indicating the interception position and the actual target location at the time of interception, respectively. The combination of acceleration and presentation time resulted in 15 experimental conditions, presented in random order within each block of trials. Each participant completed 40 trials per condition, resulting in 600 trials total, presented in eight blocks of 75 trials each. The experiment took ∼90 min.
Although TTC typically refers to the time at which a moving object contacts a secondary, stationary object, we use the term TTC to refer to the time of target reappearance behind the occluder. Furthermore, we use the term TTChand to refer to observers’ estimate of target reappearance as indicated by the time of manual interception.
Experiment 2
Stimuli and procedure were the same as in experiment 1 with the following exceptions. (1) Only the 800-ms condition was tested in experiment 2. (2) All targets moved with the same average velocity but reached the occluder with different final velocities (Fig. 1C), yielding occlusion times (TTC) of 1070, 797, 670, 588, and 529 ms for the −8, −4, 0, +4, and +8°/s2 acceleration conditions, respectively.
Eye and hand movement recordings and analyses
The data were preprocessed offline using custom-made routines in MATLAB. Eye velocity and acceleration were calculated as the first and second derivatives of the eye position signals over time. Position and velocity profiles were filtered using a low-pass, second-order Butterworth filter with cutoff frequencies of 15 Hz (position) and 30 Hz (velocity). Saccades were detected when five consecutive frames exceeded a fixed velocity criterion of 30°/s; saccade onsets and offsets were then determined as the nearest reversal in the sign of acceleration. For the analyses of the de-saccaded smooth pursuit eye movements, the identified saccades ±25 ms were removed from pursuit velocity traces and replaced by linear extrapolation between the last velocity sample before saccade onset and the first velocity sample after saccade offset (S De Brouwer et al., 2002). Pursuit onset in de-saccaded traces was detected within a 300-ms interval around stimulus motion onset (starting 150 ms before onset) in each individual trace. We first fitted each 2D position trace with a piecewise linear function, consisting of two linear segments and one break point. The least-squares fitting error was then minimized iteratively (using the function lsqnonlin in MATLAB) to identify the best location of the break point, defined as the time of pursuit onset.
The magnetic hand tracker recorded the 2D screen-centered interception position as well as the participant’s hand movement in 3D space. Hand position data were up-sampled to 1 kHz by linear interpolation for precise temporal comparison with eye movement data. Position data were filtered using a second-order Butterworth filter with a cutoff frequency of 15 Hz. Hand latency was computed offline as the first sample that exceeded 5 cm/s following stimulus onset. Hand movement offset was detected online when the finger intercepted the screen (within 0.8 mm from the screen in the z-dimension). If no offset was detected online, hand movement offset was detected offline as the maximum hand position in the z-dimension.
All trials were manually inspected. We excluded trials with blinks during the task and trials in which the eye tracker lost the signal (experiment 1: 2.9% of trials across participants; experiment 2: 3.9%). Trials were also excluded when observers undershot the right concluder’s border by >3.5° or when no interception was detected within 600 ms of target reappearance (experiment 1: 1.5% of all trials, experiment 2: 2.3%).
Data analyses
The primary aim of the current study was to assess whether observers can track and intercept accelerating targets. We assessed observers’ ability to accurately track accelerating targets with smooth pursuit eye movements during the initial target presentation (visually-guided smooth pursuit). We calculated the average de-saccaded pursuit velocity from stimulus or pursuit onset (whichever occurred earlier) and the beginning of target occlusion. Pursuit gain was typically <1 and observers showed anticipatory slowing in the pursuit before target occlusion. To account for these general biases, we normalized pursuit velocity by subtracting the control condition (0°/s2) from the experimental conditions (Fig. 3B). Following target occlusion, observers typically stopped pursuing the target with smooth pursuit eye movements and used predictive saccades to bring the eyes to the interception location. The primary predictive saccade was determined as the saccade that brought the eye within 3.5° from the occluder’s right border (to allow for systematic undershooting of saccades). The landing time of the eye was determined as the offset time of the predictive saccade. If additional saccades were made to correct the undershooting (2.8% of trials), we used the offset time of the last corrective saccade that was initiated from within the occluder. To assess whether target acceleration was taken into account in manual interception, we calculated the constant interception error as the difference between the time of interception and the veridical time of target reappearance. To assess whether target acceleration caused systematic biases in the timing of the interceptive hand movements, we additionally analyzed the interception time (TTChand). Both saccade landing time (SLT) and TTChand were calculated relative to target occlusion onset.
Model comparison
To further analyze which target features were used to estimate TTC for manual interception, we compared the biases in TTChand to three different models on how observers might have predicted TTC. First, the final velocity model postulates that observers continuously update target velocity and predict TTC based on the last available velocity sample. Second, the average velocity model assumes that observers base their TTC estimate on the average target velocity. Finally, the acceleration model suggests that observers consider final target velocity and target acceleration for interception.
Because observers showed a general trend to hit the targets too late, we first normalized TTChand by subtracting the control condition (0°/s2) from the experimental conditions. We then calculated targets’ TTC based on three models. For the final velocity model, TTC was calculated as
To further analyze which target velocity sample best described each observer’s TTChand bias, we modified the final velocity model to include target velocity as a free parameter, set to optimally predict observers’ TTChand. To this end, we first calculated each observer’s median TTC for each acceleration condition. Next, we predicted each observer’s median TTC based on each time point along each target’s velocity trajectory and determined the time point that produced the smallest RMSE. We coined this time point the hand prediction time. Note, for simplicity, we assumed that observers based their TTC estimate on a single velocity sample. Alternatively, observers might have averaged target velocity over a number of samples.
Statistical analyses
All statistical analyses were performed in R (version 3.3.2; R Core Team) with an α level of .05. For all outcome variables, we calculated the condition median across trials for each individual observer. The median was used because it is more robust against outliers and skewed distributions. To assess differences in our experimental conditions, we compared the means across subjects using repeated-measures ANOVAs (rmANOVAs) and post hoc t tests. Normal distribution of dependent variables for all ANOVAs was confirmed using the Shapiro–Wilk test (all p > 0.14). Violation of sphericity was assessed using the Mauchly’s test and p-values were Greenhouse–Geisser corrected in case of significance. To correct the familywise error rate in multiway ANOVA, we applied a sequential Bonferroni procedure to all multiway ANOVA (Cramer et al., 2016). In case of significant interaction effects, we ran follow-up, one-way ANOVA with Bonferroni corrections. Bonferroni corrections were also applied to any pairwise post hoc comparisons. To investigate whether SLT predicted TTChand on a trial-by-trial basis, we ran a linear mixed model with random intercepts and slopes between SLT and TTChand per observer and SLT, presentation duration, and acceleration as fixed effects (using the functions mixed and nice of the R package afex; Singmann et al., 2021):
Results
Observers performed a track-intercept task in which they viewed a moving disk that disappeared behind an occluder after a presentation time of 200, 500, or 800 ms and then reappeared for 100 ms (Fig. 1A). We instructed observers to intercept the target at the estimated time of reappearance (equivalent to TTC) with a quick pointing movement of their right index finger. In each trial, the target moved along a horizontal linear path either at a constant velocity (no acceleration, 0°/s2: control condition) or at a constant rate of velocity change (deceleration: −8 or −4°/s2; acceleration: +4 or +8°/s2).
We analyzed the eye and hand movement data in three parts. First, we investigated visually-guided smooth pursuit and predictive eye movements in response to accelerating targets. We asked whether target acceleration was reflected in the visually-guided eye movement response during visible target presentation, and in the timing of the predictive eye movement response during occlusion. Second, we assessed the effect of target acceleration on the timing error of the interceptive hand movement. These two analysis components are congruent with the preregistered analysis plan for this study. Third, we compared the performance of three models predicting TTC based on different target signals to observers’ hand movement data. This exploratory model comparison was confirmed in a control experiment, in which all targets moved with the same average velocity, but with different final velocities (experiment 2). We restricted the exploratory model comparison to the 800-ms conditions, because it yielded the most reliable biases in both eye and hand movements and allowed for the longest integration of the changing target velocities over time.
Target presentation duration affects ability to track but not to predict accelerating targets
Observers tracked the target with a combination of smooth pursuit and saccadic eye movements during the initial period, in which the target was visible. Two example trials, in which the target either accelerated (Fig. 2A) or decelerated (Fig. 2B) show typical eye movement position and velocity during the task. After pursuit initiation, an initial catch-up saccade aligned the eyes with the target and was typically followed by a period of closed-loop smooth pursuit, during which eye velocity matched the continuously changing target velocity. In some trials, smooth pursuit was supported by additional catch-up saccades (Fig. 2B). Around the time of target occlusion, observers stopped smoothly tracking the target and pursuit velocity decreased to 0°/s. Observers then made a distinctly identifiable predictive saccade of relatively large amplitude to the right border of the occluder, where they then intercepted the target with a pointing movement of their right index finger (see 3D hand trajectory from a single trial in Fig. 2C). To investigate how finely-tuned these different eye movement responses are to target acceleration, we analyzed the effect of presentation duration on the ability to smoothly track the visible, accelerating target and to predict its time of reappearance (TTC) with a predictive saccade.
Observers’ ability to accurately and smoothly track the accelerating target increased with increasing presentation duration, here taken as a direct measure of the availability of target motion signals (Fig. 3A,B). Whereas no difference in average pursuit velocity was observed in the shortest presentation duration (200 ms), average pursuit velocity sensitively reflected different target velocities for both longer presentation durations (500, 800 ms; Fig. 3A). This observation was confirmed by a 5 (acceleration) × 3 (presentation duration) rmANOVA on average pursuit velocity, yielding a significant acceleration × presentation duration interaction (F(8,120) = 18.52; p < 0.001; ηp2 = 0.55). Both main effects of presentation duration (F(2,30) = 184.76; p < 0.001; ηp2 = 0.92) and target acceleration (F(4,60) = 40.75; p < 0.001; ηp2 = 0.73) reached significance. We performed follow-up, one-way rmANOVA with factor acceleration for each presentation duration and found a significant main effect of acceleration in both the 500- and 800-ms conditions (F(4,60) = 8.31; p < 0.001; ηp2 = 0.36 and F(4,60) = 50.08; p < 0.001; ηp2 = 0.77, respectively), but not in the 200-ms condition (F(4,60) = 1.05; p = 1; ηp2 = 0.07).
We next asked whether observers continuously tracked the changing target velocity over time. To this end we normalized pursuit velocity traces in the acceleration conditions relative to the control condition (0°/s2) for each observer. This additional analysis accounts for imperfect pursuit velocity gain and individual differences.
The normalized pursuit velocity over time revealed how closely observers’ pursuit velocity matched the continuously changing target velocity for the two longer presentation durations (Fig. 3B). After the first catch-up saccade, the eye continuously accelerated in response to accelerating targets and decelerated in response to decelerating targets. These findings show that visually-guided pursuit closely matches the continuously changing target velocity, and that these effects are amplified with longer presentation duration where differences in target velocities were more pronounced. In contrast, pursuit velocity was similar across target accelerations in the 200-ms condition. This similarity might have been because of the small differences in the velocity of a target shown only very briefly.
Next, we asked whether the ability to accurately track the accelerating targets with increased presentation duration also affected the ability to predict its reappearance (TTC) with a predictive saccade. Predictive saccades are typically made several hundred milliseconds before target reappearance or interception. Because their timing is tuned to expectations of target motion (Diaz et al., 2013) and reflect decision outcomes in manual interception (Fooken and Spering, 2020), they can provide a sensitive indicator of target motion prediction. Although observers were not instructed to make such a predictive saccade, it was clearly identifiable in virtually every trial (>99% of all trials). These saccades were initiated on average 137 ms (±71 ms; mean ± 1 SD across observers; Fig. 3A) after occlusion onset and landed clustered around the right border of the occluder (Fig. 3C) ∼262 ms (±70 ms) after occlusion onset. In most trials, observers made one large predictive saccade with an average amplitude of 12.1° (±0.8°) across observers and trials, and this average amplitude was similar across acceleration conditions. On average, saccades were initiated 1.2° (±0.8°) from within the occluder and landed close to the occluder’s right border at 13.2° (±0.3°; Figs. 2A,B, 3C, upper panels). In trials where a second predictive saccade was made to correct for undershooting, initiation and landing time of the first predictive saccade were substantially later (319 ± 94 and 414 ± 93 ms, respectively) than in trials with only one predictive saccade.
Interestingly, the landing time of predictive saccades (SLT) did not scale with target acceleration, i.e., the eye did not land earlier at the border of the occluder for accelerating targets and did not land later for decelerating targets. Instead, the SLT showed a consistent bias in the opposite direction, i.e., later for accelerating targets and earlier for decelerating targets. This observation was confirmed by a main effect of target acceleration (F(4,60) = 8.17; p = 0.008; ηp2 = 0.35; Fig. 3D) in a 5 (acceleration) × 3 (presentation duration) rmANOVA. Note that this small bias was also present in the predictive saccade initiation time (Fig. 3A) and was neither caused by differences in saccade duration nor in amplitude across acceleration conditions. We also found a main effect of presentation duration on SLT of the predictive saccades (F(2,30) = 12.54; p = 0.003; ηp2 = 0.46). Saccades landed later in the 200-ms condition, compared with the other presentation durations (Fig. 3D). Despite the short presentation duration of 200 ms, observers typically made one early catch-up saccade followed by a predictive saccade (Fig. 3A, upper panel). This often delayed the onset of the predictive saccade and observers tracked the target until shortly after target occlusion onset. In addition, in a subset of trials (22.7%) in the 200-ms condition, observers only made one saccade. These saccades had slightly larger amplitudes, which might have contributed to the later SLTs in the 200-ms condition. There was no significant interaction between target acceleration and presentation duration (F(8,120) = 1.31; p = 0.284; ηp2 = 0.08). To further analyze which acceleration conditions were significantly different from one another, we performed pairwise comparisons after averaging SLTs across presentation durations. Post hoc t tests revealed significant differences between the −8°/s2 and the +4°/s2 as well as the +8°/s2 conditions (t(15) = −4.04; p = 0.01 and t(15) = −3.35; p = 0.044, respectively), between the −4°/s2 and +4°/s2 (t(15) = −3.90; p = 0.001), and between the 0°/s2 and +4°/s2 (t(15) = −3.29; p = 0.050) acceleration conditions. All other comparisons did not reach significance (all p > 0.10).
Taken together, these findings show that predictive saccades appear to follow a bias in the opposite direction to what we would expect if target acceleration was used to estimate TTC. Moreover, the landing time of predictive saccades was relatively less affected by the presentation duration of the target than what was observed for visually-guided pursuit. Next, we asked whether observers considered target acceleration to time their interceptive hand movement.
Target acceleration causes systematic manual interception errors
To intercept the target on the screen, observers moved their hand from the designated start position on the table to the screen (Fig. 2C). On average, hand movement paths did not differ between the different target acceleration conditions (Fig. 4A). Because of the long occlusion times, hand movements were largely executed during the occlusion and were thus not systematically corrected mid-flight. As instructed, observers intercepted the target close to the occluder’s right border (mean distance 1.7° ± 1.1°; Fig. 4B). Distributions of horizontal interception positions largely overlapped across target acceleration conditions (Fig. 4C). Yet, there was a small tendency to hit further to the right for accelerating targets (main effect of acceleration on horizontal interception position: F(4,60) = 9.21; p = 0.006; ηp2 = 0.38). This effect was primarily driven by the +8°/s2 condition because targets often reappeared before interception in this condition (Fig. 5B, green data points). In these trials, observers tended to adjust the interception position, resulting in a more skewed distribution for the +8°/s2 condition (Fig. 4C, bottom panel).
In parallel to investigating effects of acceleration on the predictive saccade, we next analyzed the effect of target acceleration on the timing of the interceptive hand movement (TTChand). If observers accounted for target acceleration when timing their hand movement, we would expect them to intercept earlier for accelerating targets and later for decelerating targets, relative to the zero-acceleration (control) condition. In contrast to this hypothesis, we found that observers intercepted later in response to target acceleration and earlier for deceleration (main effect of acceleration: F(4,60) = 10.80; p = 0.002; ηp2 = 0.42; Fig. 5A). This effect was more pronounced in the 500- and 800-ms presentation duration conditions, confirmed by a significant acceleration × presentation duration interaction (F(8,120) = 3.15; p = 0.037; ηp2 = 0.17). Accordingly, main effects of acceleration were found for both the 500-ms (follow-up, one-way rmANOVA: F(4,60) = 10.61; p < 0.001; ηp2 = 0.41) and 800-ms (F(4,60) = 5.94; p = 0.048; ηp2 = 0.28) conditions but not for the 200-ms condition (F(4,60) = 2.62; p = 0.13; ηp2 = 0.15).
Observers also tended to hit the targets later in the 200-ms condition compared with the 500- and 800-ms conditions, as indicated by a main effect of presentation duration (F(2,30) = 16.21; p < 0.001; ηp2 = 0.52).
The failure to take acceleration into account to time the interceptive hand movement resulted in systematic temporal constant interception errors. Relative to target reappearance, observers intercepted too late (i.e., the target had already reappeared) for accelerating targets and too early for decelerating targets (i.e., the target had not yet reappeared). The main effect of target acceleration on the constant interception error was significant (F(4,60) = 492.61; p < 0.001; ηp2 = 0.97; Fig. 5B). We also observed a main effect of presentation duration (F(2,30) = 16.47; p < 0.001; ηp2 = 0.52), which was caused by a general tendency to intercept targets later in the 200-ms condition (see also Fig. 5A). Although, there was a significant acceleration × presentation duration interaction term (F(8,120) = 3.28; p = 0.032; ηp2 = 0.18), follow-up, one-way rmANOVA showed main effects of target acceleration on the constant interception error for all three presentation durations (all p < 0.001).
Overall, observers were not able to accurately adjust the timing of their interceptive hand movement (TTChand) to target acceleration, causing systematic constant interception timing errors. Notably, the observed opposite bias in interception time is similar as the bias found for predictive saccades (Fig. 3D): the eye lands later and the finger intercepts later for accelerating targets, and both eye and hand intercept earlier for decelerating targets.
We found that interception time (TTChand) and SLT followed strikingly similar biases opposite to what we would expect if observers accounted for target acceleration. Given that the eye landed at the location of subsequent target reappearance several hundred milliseconds before the hand, we next asked whether SLT was a predictor of TTChand on a trial-by-trial basis. Trial-by-trial correlations imply a similarity in the trial-based variability between eye and hand movements and are interpreted as evidence for common information sources in the signals that drive eye and hand movements (Sailer et al., 2000). Using a linear mixed model, we found that SLT indeed significantly predicted TTChand on a trial-by-trial basis (b = 0.351; F(1,15.73) = 66.96; p < 0.001). Note, however, that the relation between SLT and TTChand varied substantially across observers, indicated by individual trial-by-trial Person’s correlations ranging from r = 0.20 to r = 0.57 (Fig. 6A–C).
Our findings of similar biases in predictive saccades and prediction-based interceptive hand movements and of a medium trial-by-trial correlation between saccade and hand movement timing suggest that both systems relied on similar motion prediction. Notably, despite the ability to closely track accelerating objects with visually-guided pursuit, predictive eye and hand movements appear to be insensitive to target acceleration, raising the question which target features were used to predict TTC for manual interception.
Which target features determine TTC predictions?
The observed constant interception errors and the systematic biases in TTChand (later interception for accelerating targets and earlier interception for decelerating targets) suggest that observers were not able to correctly adjust their interception timing according to target acceleration. We next asked which target features observers used instead to estimate TTC for manual interception. We compared three competing models, describing which target motion signals observers might have used to estimate TTC of accelerating targets (Bennett et al., 2007; Heinen, 2007; Fig. 7A–C).
As one possibility, observers could continuously update target velocity and estimate TTC based on the last available target velocity sample before occlusion (final velocity model; Fig. 7A). Alternatively, observers might use the average velocity during the visible period to estimate target reappearance (average velocity model; Fig. 7B). Finally, if observers indeed considered target acceleration, we would predict accurate scaling of the estimated TTC with target acceleration (acceleration model; Fig. 7C). We compared the TTC predictions of the different models to the observed biases in TTChand (Fig. 7D) and quantified the performance of each model by calculating the RMSE for each observer.
Using the acceleration model to predict TTC data in eye and hand interception confirms that interception does not use acceleration (Fig. 7D, bright gray line). Although the average velocity model captures the small, reversed trend we observed in TTChand (Fig. 7D, dark gray line), this model performs poorly at predicting the measured TTChand. The final velocity model produced the lowest RMSEs (Fig. 7D, black line). These observations were confirmed statistically by a main effect of model on RMSEs in a one-way rmANOVA (F(2,30) = 18.00; p < 0.001; ηp2 = 0.55) and post hoc pairwise comparisons. These showed that the final velocity model produced significantly lower RMSEs compared with the acceleration (t(15) = 9.64; p < 0.001; d = 2.41) and average velocity models (t(15) = 3.46; p = 0.011; d = 0.86; Fig. 7E).
The final velocity model predicts that observers rely on the last available velocity sample. To pinpoint the approximate time sample observers relied on when estimating TTC, we determined the time point along each target’s velocity trajectory that best accounted for each observer’s bias in TTChand. We termed this the hand prediction time. A negative value indicates that the observer based the TTC prediction on a velocity sample before occlusion, whereas a positive value would indicate that the TTC prediction was based on a partial extrapolation of the veridical target trajectory during occlusion. We found an average hand prediction time of −96.4 ms (Fig. 7F), which captures the small opposite bias we observed in TTChand (Fig. 7D, dashed in line).
Final velocity model prediction generalizes to different target configurations
Our model comparison indicates that observers continuously updated their prediction of target velocity until shortly before occlusion. One possible shortcoming of our experimental design was that the target moved at the same final velocity in all acceleration conditions. This might have induced a bias to always hit the target at the same time, favoring the final velocity model. Moreover, our data also showed a small bias toward the average velocity model. We conducted a control experiment to address whether the predictions of the final velocity model hold true when targets moved with different final velocities. In the control experiment, all targets moved with the same average velocities and different final velocities (Fig. 1C). Importantly, these target configurations predict the same difference in RMSEs between the average and final velocity models as in experiment 1, allowing us to directly compare the model fits between the two experiments. Note that in contrast to experiment 1, the average velocity model here predicts no TTC adjustments, whereas the final velocity model predicts earlier TTC for accelerating targets and later TTC for decelerating targets (Fig. 8A,B).
In line with the prediction of the final velocity model, we found that observers intercepted accelerating targets earlier (Fig. 8C, negative TTC values) and decelerating targets later (Fig. 8C, positive TTC values). This observation was supported by a significant one-sided paired t test, testing whether the final velocity model produced significantly lower RMSEs compared with the average velocity model (t(9) = 2.12; p = 0.031; d = 0.67; Fig. 8D). We also estimated the target velocity sample that best predicted observers’ TTC bias. We found a mean hand prediction time of 65.8 ms before occlusion onset, which was similar but slightly smaller than the hand prediction time in experiment 1. Together, these results replicate our findings from experiment 1 and confirm the use of the final target velocity to predict TTC.
Discussion
The aim of the current study was to investigate how humans integrate visual motion information to track and predict accelerating objects for manual interception. Our task required observers to track an accelerating target before a temporary occlusion, and to predict the time of target reappearance by making an interceptive hand movement. This hand movement was naturally accompanied by a predictive saccade to the interception position, although no explicit instruction to make such a saccade was given. Our results show that observers were insensitive to target acceleration when predicting future target motion. Neither the timing of the predictive saccade nor the interceptive hand movement (TTChand) scaled with acceleration, resulting in systematic constant interception errors. Inability to account for target acceleration was observed regardless of target presentation duration and of whether observers were able to accurately track the accelerating target before the occlusion. TTC estimates were best described by a model that relied on the final velocity of the target just before occlusion, indicating that observers based their prediction on the memory of the last available velocity signal (first-order motion; Benguigui et al., 2003; Benguigui and Bennett, 2010).
Different acceleration sensitivity for tracking and predicting?
To successfully interact with a moving object, we must continuously monitor its dynamically-changing motion trajectory. Because of sensorimotor delays, we need to quickly form a prediction of current and future object motion. This form of prediction is also important when we lose sight of an object, or when it is temporarily occluded. Naturally-moving objects do not necessarily move at constant velocity but can suddenly accelerate or decelerate. Forming a prediction that can capture dynamically-changing object motion is therefore an integral part of everyday actions (Zhao and Warren, 2015; Fiehler et al., 2019).
The abilities to track and predict the motion trajectory of objects that move at constant velocity are often closely linked (Makin and Poliakoff, 2011). Accurate tracking of a moving object with smooth pursuit eye movements can enhance temporal (Bennett et al., 2010) and spatial (Spering et al., 2011) predictions of target trajectories. Yet, our results suggest that tracking an accelerating target does not necessarily extend to predicting accelerating objects when aiming to intercept them. Specifically, we show that observers’ eye movements closely matched the velocity profile of accelerating targets for target presentations of longer than 200 ms. However, regardless of how long observers had time to track and potentially integrate acceleration signals, they did not consider target acceleration when timing their manual interception and predictive saccade.
One explanation for the apparent discrepancy between tracking and predicting accelerating targets could be that visually-guided tracking can rely on detecting and updating the changing target velocity over time and might thus not require a direct consideration of the acceleration signal. Conversely, to intercept accelerating targets, observers would need to consider an explicit readout of target acceleration to form a prediction of target motion to overcome sensorimotor delays or a temporary occlusion of the moving object. The finding that acceleration is not used during manual interception suggests that observers continuously update their judgment of target velocity but cannot integrate acceleration signals to inform their prediction. Instead, they predicted TTC based on a velocity sample ∼100 ms before target occlusion, suggesting the use of velocity memory when predicting future target motion (Soechting et al., 2009; Benguigui and Bennett, 2010; Rust and Palmer, 2021). Continuously tracking the changing target velocity with smooth pursuit eye movements might have thus supported manual interception by continuously updating observers’ prediction of target velocity.
Alternatively, tracking and predicting might exhibit different sensitivity to acceleration signals. A possible dissociation in integrating acceleration signals between tracking and interception is congruent with two sets of literature that have typically tested both behaviors, tracking and interception, separately. First, the smooth pursuit system can be sensitive to acceleration signals when probing it with velocity perturbations (Tavassoli and Ringach, 2010; Brostek et al., 2017). Moreover, predictive pursuit during a target’s occlusion period scales with the target’s acceleration (Bennett and Barnes, 2006; Bennett et al., 2007; Bennett and Benguigui, 2013). These findings suggest that the oculomotor system can extract acceleration signals even to predictively drive pursuit. Second, interceptive hand movements are comparatively unresponsive to visual acceleration, reflected in systematic errors when intercepting accelerating targets (Port et al., 1997; Dubrowski and Carnahan, 2002; Benguigui et al., 2003; Brenner and Smeets, 2015; Brenner et al., 2016). These systematic interception errors can be explained by a failure to predict accelerating target motion to overcome sensorimotor delays (Brenner and Smeets, 2015) or temporary target occlusion, in line with our results (see also Reid and Dessing, 2018). Moreover, we showed that systematic interception errors can be described by a model that relied on target velocity just before target occlusion (first-order motion; Benguigui et al., 2003; Benguigui and Bennett, 2010). Together, our results are congruent with the idea that target velocity estimates are continuously updated for both visually-guided pursuit and prediction-guided interception.
Extending the eye-hand link to prediction-based actions
We observed trial-by-trial correlations between the timing of the predictive saccade and TTChand, extending the known close coupling of eye and hand movements during visually-guided actions (Hayhoe, 2017; AJ De Brouwer et al., 2021) to predictive actions (Binaee and Diaz, 2019). During visually-guided reaching, observers commonly shift their eyes to the reach target before hand movement execution (Ballard et al., 1992; Neggers and Bekkering, 2000; Johansson et al., 2001; Land and Hayhoe, 2001; Horstmann and Hoffmann, 2005; Barany et al., 2020). When intercepting moving targets, observers naturally track the target with their eyes, even when no explicit instruction to do so is given (Mrotek and Soechting, 2007). In interception tasks, eye and hand movement endpoints are also correlated (Kreyenmeier et al., 2017; Li et al., 2018; Fooken et al., 2021). Our results extend these findings in two ways. First, correlations of predictive eye movements and interceptive hand movements reveal that the coordinated control of eye and hand movements also applies to memory-based actions. Second, correlations of temporally-based estimations show that eye and hand movements can be correlated not just in the spatial, but also in the temporal domain. Similarly, previous studies showed that the timing of predictive saccades is finely tuned to stimulus properties (ball speed and elasticity; Diaz et al., 2013) and is a sensitive indicator of decision outcomes in manual interception tasks (Fooken and Spering, 2020). Although the eyes reach the interception location several hundred milliseconds before the hand in our task, the timing of predictive saccades and interceptive hand movements showed strikingly similar biases and were correlated on a trial-by-trial basis. A strong eye-hand link is expected when intercepting targets that move unpredictably and are partially occluded from view (Fooken et al., 2021). If acceleration is indeed not considered in a predictive model of target motion, the extrapolation of accelerating target motion becomes highly inaccurate and observers rely on their eye movements to continuously update their prediction of the target motion (Brenner and Smeets, 2018; De la Malla et al., 2019).
Assessing model predictions of accelerating motion integration
Given the limited perceptual sensitivity to acceleration, and the lack of acceleration tuning in key motion-sensitive cortical areas (Lisberger and Movshon, 1999; Price et al., 2005) the question arises what information observers rely on when interacting with accelerating objects in everyday life. It is well known that humans use physical laws of motion, such as gravity, which are learned throughout the lifespan, as an implicit prior when interacting with real-world objects (Zago and Lacquaniti, 2005; Jörges and López-Moliner, 2017). For instance, observers are more accurate when tracking and predicting simulated fly balls that move with natural gravity compared with balls that do not (0 g), or that are unnaturally impacted by gravity (2 g; Bosco et al., 2012; Russo et al., 2017). Although naturalistic priors influence ocular and perceptual motion prediction (Delle Monache et al., 2019), other studies have also found that observers assume that targets move with constant velocity when predicting object motion (Jörges et al., 2021).
When intercepting targets that are impacted by arbitrary acceleration, we found that observers make systematic interception errors (Port et al., 1997; Dubrowski and Carnahan, 2002; Benguigui et al., 2003). In situations where the target is not occluded from view and remains visible throughout, observers can minimize these interception errors by continuously adjusting their interceptive hand movement online (Brenner and Smeets, 1997; Reid and Dessing, 2018). Nonetheless, even when intercepting visible accelerating targets, systematic interception errors occur because of sensorimotor delays (Brenner and Smeets, 2015; Brenner et al., 2016). One possibility to compensate for the inability to extrapolate accelerating motion, is to quickly adapt movements, given that sufficient trial repetitions are available (Ruttle et al., 2021). For example, improvements in the ability to manually intercept (Brenner et al., 2016) and predictively pursue (Bennett and Barnes, 2006) accelerating targets after a few (eight to twelve) repetitions of the same acceleration rate have been reported. These findings suggest that observers might be able to form short-term and long-term (naturalistic) priors to counteract the lack of acceleration signal integration.
In conclusion, our study shows that observers failed to use an acceleration-based prediction of the target’s motion to inform manual interception. Instead, the timing of manual interception was best predicted by an extrapolation of target velocity shortly before target occlusion. Systematic interception errors occurred regardless of the target presentation duration and how well observer’s visually-guided eye movements matched the different target velocity profiles. Interestingly, the timing of both predictive eye and interceptive hand movements showed strikingly similar biases and were correlated on a trial-by-trial basis, indicating a strong coupling between both effectors during prediction-guided interception tasks.
Citation Diversity Statement
Recent work in several fields of science has identified a bias in citation practices such that papers from women and other minority scholars are under-cited relative to the number of such papers in the field (Zurn et al., 2020). Here, we sought to proactively choose references that reflect the diversity of the field in thought, form of contribution, gender, race, ethnicity, and other factors. First, we obtained the predicted gender of the first and last author of each reference by using databases that store the probability of a first name being carried by a woman (Dworkin et al., 2020; Zhou et al., 2020). By this measure (and excluding self-citations to the first and last authors of our current paper), our references contain 12.9% woman (first)/woman (last), 12.26% man/woman, 22.58% woman/man, and 52.25% man/man. This method is limited in that (1) names, pronouns, and social media profiles used to construct the databases may not, in every case, be indicative of gender identity and (2) it cannot account for intersex, nonbinary, or transgender people. Second, we obtained predicted racial/ethnic category of the first and last author of each reference by databases that store the probability of a first and last name being carried by an author of color (Ambekar et al., 2009; Sood and Laohaprapanon, 2018). By this measure (and excluding self-citations), our references contain 9.24% author of color (first)/author of color (last), 19.02% white author/author of color, 16.68% author of color/white author, and 55.06% white author/white author. This method is limited in that (1) names and Florida Voter Data to make the predictions may not be indicative of racial/ethnic identity, and (2) it cannot account for Indigenous and mixed-race authors, or those who may face differential biases because of the ambiguous racialization or ethnicization of their names. We look forward to future work that could help us to better understand how to support equitable practices in science.
Footnotes
The authors declare no competing financial interests.
This work was supported by a University of British Columbia International Doctoral Fellowship (P.K.), the Deutsche Forschungsgemeinschaft (DFG) Research Fellowships Grant FO 1347/1-1 (J.F.), and a Natural Sciences and Engineering Research Council of Canada Discovery Grant and Accelerator Supplement (M.S.).
Acknowledgments: We thank members of the Spering lab for comments on an earlier version of this manuscript. The study design and parts of the analysis were preregistered (https://osf.io/adq9v), all data and analyses are available on the Open Science Framework (https://osf.io/4fv5m). The model comparison was not included in the preregistration. Preliminary data were presented at the 2021 virtual meeting of the Vision Sciences Society.
- Received May 2, 2022.
- Revision received July 10, 2022.
- Accepted July 27, 2022.
- Copyright © 2022 Kreyenmeier et al.
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.