Abstract
Keeping track of multiple visually identical and independently moving objects is a remarkable feature of the human visual system. Theoretical accounts for this ability focus on resource-based models that describe parametric decreases of performance with increasing demands during the task (i.e., more relevant items, closer distances, higher speed). Additionally, the presence of two central tracking resources, one within each hemisphere, has been proposed, allowing for an independent maintenance of moving targets within each visual hemifield. Behavioral evidence in favor of such a model shows that human subjects are able to track almost twice as many targets across both hemifields compared with within one hemifield. A number of recent publications argue for two separate and parallel tracking mechanisms during standard object tracking tasks that allow for the maintenance of the relevant information in a location-based and object-based manner. Unique electrophysiological correlates for each of those processes have been identified. The current study shows that these electrophysiological components are differentially present during tracking within either the left or right hemifield. The present results suggest that targets are mostly maintained as an object-based representation during left hemifield tracking, while location-based resources are preferentially engaged during right hemifield tracking. Interestingly, the manner of representation does not seem to have an impact on behavioral performance within the subjects, while the electrophysiological component indicating object-based tracking does correlate with performance between subjects. We propose that hemifield independence during multiple-object tracking may be an indication of the underlying hemispheric bias for parallel location-based and object-based tracking mechanisms.
Significance Statement
The human visual system is able to keep track of multiple independently moving objects within the visual field over a period of time. It has been suggested that two separate cognitive resources might exist that are assigned to each of the two visual hemifield. In the current paper, we show that these two resources are implemented within two different representational domains—an object-based within the left hemisphere and a location-based within the right hemisphere. This study bridges the controversy between the previous findings of the parallel involvement of object- and location-based attentional mechanisms during tracking and the intriguing reports of hemispheric independence during tracking.
Introduction
In daily life, our visual system is continuously challenged by the simultaneously incoming information and manages to separate relevant from irrelevant information. This is particularly difficult when presented with dynamically changing visual information such as during tracking multiple relevant independently moving objects among other distracting ones. The multiple-object tracking (MOT) task (Pylyshyn and Storm, 1988) provides a great opportunity to study the involved processes. During this type of experiment, the subjects are required to visually track a certain number of moving items among other, visually identical items. At first glance surprising, participants are able to keep track of at least four individual objects without much effort (Pylyshyn, 1989, 2004; Scholl et al., 2001; Alvarez and Franconeri, 2007).
Several mechanistic explanations were put forward to account for this ability. They mainly focus on the continuous parametric relationship between behavioral tracking ability and properties of the tracking task like number of tracked items (Pylyshyn, 1989; Alvarez and Franconeri, 2007), item speed, or minimal inter-item distance (Shim et al., 2008; Franconeri et al., 2010; Chen et al., 2013). All these suggestions converge on variations of a continuous shared resource model that suggests either a spatial or temporal split of attentional resources across the relevant targets (Alvarez and Franconeri, 2007; Horowitz and Cohen, 2010; Howe et al., 2010; Chen et al., 2013) or a reduction of tracking resources by spatial interference (Shim et al., 2008; Franconeri et al., 2010).
A further intriguing property of the object tracking task is the behavioral advantage of performing the task across hemifields (bilateral) compared with performing within the left or right hemifield only (Alvarez and Cavanagh, 2005). The capacity for the bilateral tracking almost doubles compared with the unilateral task, suggesting at least in part independent tracking mechanisms for each hemisphere (Alvarez and Cavanagh, 2005; Chen et al., 2013; Stormer et al., 2014; Strong and Alvarez, 2020). This advantage is even present when the factor of spatial interference is controlled for (Holcombe and Chen, 2012). Importantly, continuous resource models outlined earlier potentially account for these findings by assuming two distinct but functionally similar tracking resources implemented within each hemisphere. It is important to note that the postulated continuous resource models rely on a close relationship between tracking performance and the maintenance of target representations within or across hemifields (i.e., more targets, less representational acuity based on less resource or more spatial interference).
A very different mechanistic account for MOT was suggested in which the entirety of relevant items during visual tracking is maintained as an integrated object-based representation and thus facilitates tracking performance (Yantis, 1992). Recently, behavioral (Merkel et al., 2014), electrophysiological (Merkel et al., 2014, 2017, 2022), and functional imaging (Merkel et al., 2015) data from our lab provided strong evidence for such a mechanism. In a series of experiments, the subjects tracked four out of eight moving identical items. After the tracking phase, the subjects had to indicate whether a probe consisting of four items identically matched the entire target set of objects or not. These experiments revealed a full-match effect as an object-based advantage for probe sets, which were fully congruent with the target set. For this effect, we also were able to identify a specific electrophysiological marker, an amplitude modulation of the N170 event-related component for the object-based representation. In parallel, we also observed a continuous parametric location-based representation of the single items reflected by an amplitude modulation of the N270 component (Merkel et al., 2014, 2022). These two modulations were accompanied by enhanced γ oscillations during the tracking phase in subjects, whose behavioral data indicated a location-based tracking strategy and a reduction in γ oscillations during the motion phase of subjects that maintained items mostly using an object-based strategy (Merkel et al., 2022).
Interestingly, a functional bias between location-based and object-based processing of visual information toward one of the two hemispheres has been observed in visual perception (Egly et al., 1994; Marsolek, 1995) and attention tasks (Pollmann and Morrillo, 2003; Wilson et al., 2005). Moreover in a few studies investigating the bilateral field effect in object tracking, a slight general advantage for processing information within the right hemifield has been observed (Holcombe et al., 2014; Strong and Alvarez, 2020). The question arises whether the separate hemifield biases for visual representations of locations and objects can potentially account for the bilateral field advantage observed in tracking tasks.
Here we employ electrophysiological recordings in conjunction with a lateralized MOT task along with established markers to show that location- and object-based multiple tracking mechanisms entail a representational hemifield bias. Tracking within the left visual hemifield relies on an object-based representation, while a location-based representation is primarily employed to track multiple objects individually within the right visual hemifield. The integration of these representations underlies the bilateral field advantage in object tracking.
Materials and Methods
Twenty-six subjects (19 female) with a mean age of 26.15 (SD, 4.65) preformed in the lateralized object tracking task. Prior to the experiment, a written informed consent was obtained, after which the subjects were instructed about the task. The study was approved by the local ethics board (no. 141/20).
Stimuli
Throughout the experiment, the subjects were required to fixate a cross of 0.12° located in the middle of the screen. All stimuli were presented in white on a black background. Each trial started with a central directional cue (arrow) for 700 ms instructing the subject to perform the subsequent task either within the left or right hemifield (Fig. 1). After a 200 ms blank period, the outline of 12 squares (0.48°) appeared on the screen. The items were randomly arranged into two sets of six objects allocated within a designated area in the left and right hemifield, respectively. These designated areas occupied a space of 12.19° (vertical) by 10.24° (horizontal) centered at 7.07° left and right of fixation. Thus, the inner vertical border of each area had a distance of 1.95° from fixation. Three of the six items of each set of objects were designated as targets by having them blink twice [changing from an outlined (200 ms) to filled square (200 ms) and back]. At the end of this cueing phase, all 12 items appeared visually indistinguishable from each other in an outlined form again and started to move randomly within their designated (left or right) area with a speed of 4.63°/s for 4 s.
Experimental design. The tracking display contained two sets of six objects each, within the right and left side of the screen, respectively. Each trial started with a directional cue indicating the side of the display the tracking task had to be performed in the current trial. All 12 objects appeared simultaneously on the screen, and three items within the left tracking area as well as three items within the right racking area were cued. Subjects were to attend the cues within the, for the current trial, relevant area only. Subsequently, the two sets of six items moved around their respective areas in a pseudorandom fashion. At the end of the movement phase, three objects of the left and right set of objects each were probed, and the subjects had to indicate whether the probed items within the relevant hemifield fully matched the previously cued items or not.
Trajectories for each item and each trial were calculated offline using a semirandomized approach that is described in detail in previous work (Merkel et al., 2014, 2020). The items moved in a continuous and curved manner while not bouncing off the illusory border formed by the respective area or off each other. They kept a minimum distance to any other item of 0.97° throughout the motion phase and never left their area. A unique set of trajectories was calculated for each trial.
Once the motion ceased, again three items of each set of objects appeared as filled squares (probes) at which point the subjects were asked to indicate whether the three probed items within the relevant attended area (left or right) were the identical items designated as targets at the beginning of the trial. Either none of the three probed items on the relevant side were the target items (M0), one of those (M1), two of those (M2), or all three of those (M3, “full match”). The set of probe items appearing within the irrelevant unattended tracking area always consisted of a none-matching probe (M0).
Procedure
The experiment was performed in a dark shielded chamber, and the subjects were placed 70 cm in front of the display. The subjects were shown up to 10 practice trials to familiarize themselves with the task. Special emphasis was given to the directional cue, and the subjects had to track only the relevant field in each trial. The subjects were instructed to maintain fixation throughout the entire experiment. At the end of each trial, the subjects were to indicate a full match within the relevant tracking area with a button press of the right index finger. Any other partly and nonmatching probes had to be indicated by a button press with the right middle finger. The levels of the two factors “match” (M0/M1/M2/M3) and “hemifield” (left/right) were counterbalanced and presented randomly throughout the experiment. Each of the eight conditions was presented 40 times for a total of 320 trials per subject.
EEG recordings
While the subjects performed the experiment, the ongoing EEG was continuously recorded using a 32-channel actiCAP system (Brain Products) with electrodes placed according to the international 10–20 system and referenced to the right mastoid. Impedances were kept below 5 kΩ, and the data were acquired with a sampling rate of 500 Hz. Offline analyses were performed using the FieldTrip Toolbox. The recorded data were first offline filtered with a 128 Hz low-pass filter and re-referenced to the average of right and left mastoid. Next, epochs from −400 to 1,000 ms were extracted from the continuous dataset relative to the temporal onset of the probe display. An artifact-reduction step was included by performing an independent component analysis (ICA) decomposition on the epoched datasets using the runica function with standard parameters. Components reflecting clear ocular sources were manually selected and consequently rejected. An additional artifact rejection step, removing epochs exceeding individual peak-to-peak thresholds, was performed. Those thresholds were adjusted in order to remove ∼5% of trials within the individual subjects and ranged from 100 to 250 uV.
After extracting epochs reflecting the probe displays at the end of each trial, we additionally focused on the time range of tracking itself. Hereby, epochs were extracted from −400 to 4,000 ms time-locked to the motion onset of each trial. Those epochs were taken from the continuous dataset on which the ICA-based artifact reduction step has been applied to using the components identified during the prior ICA decomposition. In addition, epochs with individualized peak-to-peak amplitudes that included ∼5% of all motion-onset trials per subject were removed.
In order to assess the fixation performance of the subjects during the tracking task especially with respect to the experimental factor “hemifield” (left/right), the mean amplitudes of the bipolar HEOG channel (F8–F7 difference) for trials in which tracking had to be performed within the left hemifield versus trials in which tracking had to be performed within the right hemifield were analyzed. This analysis was performed for the probe-display and the motion-phase epochs separately. Figure 2b displays the mean left hemifield–right hemifield difference waves within the bipolar HEOG over the time course of the probe display for each subject. Two subjects exceeded amplitudes of 10 uV during this timeframe and were therefore excluded from further analysis. The remaining 24 subjects with mean saccade differences of <10 uV over the course of the probe displays are considered to have been sufficiently fixated throughout the task. Peripheral tracking apertures were placed laterally from 12.19° to 1.95° from fixation. HEOG saccade amplitude estimates typically range from 14 to 20 uV per degree of visual angle (Berg and Scherg, 1991; Picton et al., 2000; Dimigen et al., 2009; Keren et al., 2010). Thus, the range of saccades observed in the subjects can be considered not to have influenced the lateral retinal projection of the separate left and right tracking tasks during the presentation of the probe display. Horizontal eye movements (F8–F7) across the time course of the motion phase and the probe display are displayed in Figure 2c for the left hemifield and right hemifield conditions.
a, Behavioral results. Response accuracy decreased for an increasing match between probe items and cued items. This pattern was the same for tracking within the right hemifield and tracking within the left hemifield. b, EOG responses of all 26 subjects toward the appearance of the probe display. All but two subjects (blue traces) exhibited horizontal eye movements of <10 uV. c, Average horizontal eye movements during the tracking phase of the task and the subsequent probe display for tracking within the left and right hemifield (shaded area indicates the standard error). During the probe display, the direction of the horizontal shifts showed a bias toward the relevant tracking area. However, the subjects exhibited sufficient fixation since these shifts were not large enough to modulate the unilateral projection during the lateralized tracking task.
The preprocessed data of the 24 included subjects were further baseline corrected from −200 to 0 ms for the probe-display epochs and the motion-onset epochs. Linear trends within all epochs were equally removed. Subsequently, the preprocessed probe-display epochs were averaged over trials for each of the eight conditions [“hemifield” (left/right) × “match” (M0/M1/M2/M3)] separately.
The motion-onset epochs were analyzed by averaging the contra-minus ipsilateral electrode difference waveforms for each trial across the two hemifield conditions (left/right), thereby quantifying the representational load of the tracked items using the contralateral delay activity (CDA) during the motion phase of the tracking trials for left hemifield trials and right hemifield trials (McCollough et al., 2007; Balaban and Luria, 2019).
Statistical analysis
Subject’s behavioral performance was analyzed as variation in response accuracies toward the different probe displays across the factors “match” and “hemifield” using a two-factorial repeated-measures ANOVA.
During probe-onset epochs, bilateral occipital electrodes exhibited previously described tracking-specific morphologies of N170 and N270 components (Merkel et al., 2014, 2022) in response to the appearance of the probe display. Mean amplitudes were measured for the components within 170–190 ms for the N170 and within 240–300 ms for the N270 across the posterior electrodes P7, P3, Pz, P4, P8, PO9, O1, Oz, O2, and PO10. Repeated-measures ANOVAs were used to analyze mean amplitude variations across the different levels of the factors “match” (M0/M1/M2/M3) and “hemifield” (left/right) for the two components (N170/N270).
The CDA amplitude differences during the motion phase were derived for the left and right hemifield tracking condition at standard occipitoparietal sites P3/4, P7/8, and PO9/10 (Balaban and Luria, 2017) based on the average amplitude between 1,500 and 4,000 ms into the tracking phase.
Next, the subjects were separated into two performance groups based on their relative response accuracies toward the full-match versus the no-match condition [across hemifields (left/right)]. A median split was thus used to categorize subjects as more or less accurate in correctly responding toward the full-match condition. A similar approach to classify subjects by tracking strategy was used previously (Merkel et al., 2015, 2022; Lesch et al., 2020). Subsequently, the ERP variation across factors “match” and “hemifield” of the probe-onset epochs was analyzed for the two groups separately. Additionally, variations of the CDA amplitude during the tracking phase were analyzed with the between-subject factor “group.” Low-performance subjects exhibited a poor accuracy toward the full-match trials, whereas the high-performance subjects showed high accuracies toward the full-matching probes, comparable with the nonmatching probes.
Additionally, hemisphere-specific contributions in amplitude variations toward the N170 and N270 components of the probe-onset epochs were assessed using a three-factorial rANOVA including the factors “hemifield,” “match,” and “electrode sites” [P4, P8, O2, PO10(right) / P3, P7, O1, PO9(left)]. The localization of the N170 component during the left hemifield tracking over left electrode sites and a stronger variation of the N270 amplitudes during right hemifield tracking over right electrode sites motivated an additional analysis considering the differences between left and right electrode amplitudes for the N170 and N270 components. Hemisphere-specific amplitude differences were calculated as [mean(P3, P7, O1, PO9) − mean(P4, P8, O2, PO10)] for left hemifield trials and [mean(P4, P8, O2, PO10) − mean(P3, P7, O1, PO9)] for right hemifield trials. This analysis would exclude any bilaterally evoked amplitude for any of the conditions and only reveal the hemispheric bias for each of the specific patterns (more location-based/object-based) evoked by right and left hemifield tracking, respectively. Interestingly, while the later N270 exhibits a stronger parametric variation in this difference wave for the right hemifield tracking, the N170 modulation during trials in which tracking had to be performed within the left hemifield appears considerably earlier than between 170 and 190 ms within the left electrodes compared with all occipitoparietal sites. The observed modulation of the difference waveform was subsequently analyzed in a time window between 110 and 140 ms.
Results
The subjects exhibited a parametric variation of performance with higher error rates for trials with more congruity between probes and targets (F(3,69) = 12.403; p = 0.001; μ2 = 0.35). Interestingly, the overall pattern in behavior did not differ between tracking within the left and right hemifield (F(3,69) = 1.078; p = 0.364; μ2 = 0.045), nor did the mean error rate between left and right tracking (F(1,23) = 1.497; p = 0.233; μ2 = 0.061; Fig. 2a).
Event-related amplitude modulations for the N170 component were compared using two-factorial rANOVAs including the factors “hemifield” and “match” over the posterior electrodes (Fig. 3a). The N170 exhibited a significant amplitude modulation across match conditions (F(3,69) = 3.683; p = 0.017; μ2 = 0.138) which did not show an interaction with the attended hemifield (F(3,69) = 0.952; p = 0.406; μ2 = 0.040). However, more detailed rANOVAs revealed no significant amplitude modulation for right hemifield trials across the match conditions (F(3,69) = 0.438; p = 0.709; μ2 = 0.019), while the main effect for “match” was significant (F(3,69) = 3.737; p = 0.021; μ2 = 0.140) when subjects were required to track within the left hemifield. Importantly, this effect for left hemifield tracking trials is driven by an enhanced negativity present exclusively for the full-match condition (M3) compared with that for the partly matching conditions (M1–M3: t(23) = 2.648, p = 0.014, μ2 = 0.234; M2–M3: t(23) = 2.541, p = 0.018, μ2 = 0.219) with the M0–M3 contrast as a trend (t(23) = 1.956; p = 0.063; μ2 = 0.143). An interaction between the hemifield and match conditions with only two levels (M2/M3) remains nonsignificant as well (F(1,23) = 1.432; p = 0.234; μ2 = 0.061). However this interaction shows also nonequivalency, meaning that the true interaction effect reaches at least a level of a “smallest effect size of interest” (SESOI). This is the effect size that can be determined to be significant given the power of the test (Lakens, 2014). For the current interaction, the effect is significantly larger than the lower bound of the SESOI (d = −0.422: t(23) = 3.29, p = 0.002) but not lower than the upper bound of the SESOI (d = 0.422: t(23) = −0.845, p = 0.203).
Electrophysiological results [amplitudes in microvolts (uV)]. a, Toward the probe display, the patterns of the N170 and N270 amplitudes within the parieto-occipital electrodes toward the different match conditions (M0–M3) were highly dissimilar when compared with left and right hemifield tracking. The visual probes during tracking within the left side of the screen evoked an exclusive N170 response for the full-match condition. In contrast, tracking within the right side of the screen evoked a more parametric response toward the probe across the different match conditions (significant results are denoted using an asterisk while trends are denoted using a dagger). b, During the tracking phase, a CDA emerged over parieto-occipital electrodes with a larger sustained amplitude for tracking within the right hemifield.
The N270 component did show a significant main effect for the factor “match” as well (F(3,69) = 4.660; p = 0.007; μ2 = 0.168), which did not exhibit an interaction with “hemifield” (F(3,69) = 1.310; p = 0.280; μ2 = 0.054). Likewise, employing an equivalence test on the 2 × 2 interaction of the N270 component (left/right × M3/M2) does show nonequivalency, suggesting the effect size to be at least relevant with a nonsignificant lower bound (t(23) = 0.657; p = 0.259) and significant upper bound (t(23) = −3.48; p = 0.001). Employing one-factorial ANOVAs, a very different pattern emerged compared with the N170 component: the probe display for tracking within the left hemifield did not elicit any variation in mean amplitudes between 240 and 300 ms for the different match conditions (F(3,69) = 0.608; p = 0.610; μ2 = 0.026). On the other hand, the probe during tracking within the right hemifield elicited a mostly parametric variation of mean amplitude within the same time range (F(3,69) = 5.477; p = 0.003; μ2 = 0.192) with larger negativity for only partly matching probe displays (M1–M2: t(23) = −2.178 p = 0.040, μ2 = 0.171; M0–M2: t(23) = −3.888 p = 0.001, μ2 = 0.397). No difference was observed for amplitudes elicited by nonmatching and fully matching probes for right hemifield trials within the time range of the N270 component (t(23) = −0.458; p = 0.652; μ2 = 0.009).
During the motion phase of the tracking task, a classical sustained CDA modulation was observed (Fig. 3b). Most interestingly, this CDA showed a larger amplitude for trials in which tracking had to be performed within the right hemifield compared with trials in which tracking had to be performed within the left hemifield (t(23) = 2.139; p = 0.043; μ2 = 0.166).
ERP–amplitude differences toward the probe display across match conditions for left and right hemifield tracking were further analyzed for two different performance groups, which differed in their response patterns toward the match conditions (F(3,66) = 17.021; p < 0.001; μ2 = 0.436). This comparison revealed a performance-dependent modulation of the N170 component exclusively for the left hemifield tracking trials (Fig. 4a). Hereby, the low-performance group did not show any N170 modulation across match conditions (F(3,33) = 1.169; p = 0.331; μ2 = 0.096), whereas the high-performance group did exhibit a significant effect of “match” in the N170 amplitudes during left hemifield tracking (F(3,33) = 3.151; p = 0.040; μ2 = 0.223). This effect is driven by a higher negativity for the fully matching compared with that for the partly matching probes (M3–M0: t(11) = 2.923, p = 0.014, μ2 = 0.437; M3–M1: t(11) = 2.160, p = 0.054, μ2 = 0.298; M3–M2: t(11) = 2.378, p = 0.037, μ2 = 0.340). The pattern of N270 amplitudes however does not show any differences between match conditions in either the low- (F(3,33) = 0.250; p = 0.838; μ2 = 0.022) or high-performance group (F(3,33) = 0.537; p = 0.643; μ2 = 0.047) for left hemifield tracking. A very different picture emerges for tracking within the right hemifield: the N170 does not show any significant match effect for either performance group (low: F(3,33) = 0.912, p = 0.437, μ2 = 0.077; high: F(3,33) = 0.518, p = 0.673, μ2 = 0.045). However, the N270 exhibits a main effect for “match” in both groups (low: F(3,33) = 2.658, p = 0.066, μ2 = 0.195; high: F(3,33) = 2.859, p = 0.052, μ2 = 0.206), displaying a similar modulation that appears to be independent of the performance group (F(3,66) = 0.270; p = 0.832; μ2 = 0.012).
Performance-depending amplitude modulations [amplitudes in microvolts (uV)]. a, During the probe display, the N170 modulation for the left hemifield tracking does show a higher amplitude toward the full match in subjects with better response accuracies in this match condition. The N270 modulation, however, does not show any behavioral correlate between subjects (significant results are denoted using an asterisk while trends are denoted using a dagger). b, No significant interaction can be observed for the CDA amplitude between the left and right hemifield tracking and performance group during the tracking phase.
Most interestingly, within each performance group, the differences in N170 and N270 amplitude patterns between left and right hemifield tracking do not seem to be accompanied with different behavioral patterns for left and right hemifield tracking. Response accuracies across match conditions between trials in which subjects had to track objects on the left side and trials in which subjects had to track objects on the right side are virtually identical within both performance groups (low: F(3,33) = 1.070, p = 0.368, μ2 = 0.089; high: F(3,33) = 0.182, p = 0.884, μ2 = 0.016).
The CDA amplitude during the motion phase for left and right hemifield tracking did not show any significant interaction with the performance group (F(1,22) = 0.085; p = 0.773; μ2 = 0.004; Fig. 4b). Likewise, high- and low-performing subjects do not differ in their overall CDA amplitude for left and right hemifield tracking (F(1,22) = 2.102; p = 0.161; μ2 = 0.87), although the CDA difference between subjects does visually exhibit a different temporal morphology for left and right hemifield tracking.
Next, we sought to investigate potential hemispheric biases for the N170 and N270 components for left hemifield and right hemifield tracking, respectively. Therefore, we included electrode side [PO9, P3, P7, O1 (left) vs PO10, P4, P8, O2 (right)] as a factor in a three-way repeated-measures ANOVA (“hemifield” × “side” × “match”) for the N170 and N270 amplitudes (Fig. 5a). During the early N170 time range, amplitudes were more negative for the left compared with the right occipital electrodes across all trials (F(1,23) = 9.741; p = 0.005; μ2 = 0.298). Importantly, this amplitude difference between left and right electrodes was not different for probes presented within the right and left hemifield (F(1,23) = 0.097; p = 0.758; μ2 = 0.004). The three-way interaction between “hemifield,” “side,” and “match” condition was significant (F(3,69) = 5.399; p = 0.003; μ2 = 0.190) giving an indication together with Figure 5a for the specific full-match effect of the N170 during left hemifield trials being more prominent within the left hemisphere. The mean amplitudes of the N270 component did not differ for left and right electrodes across hemifields and match conditions (F(1,23) = 1.309; p = 0.264; μ2 = 0.054). The N270 shows however a significant variation for the match conditions across all electrodes and hemifield conditions (F(3,69) = 4.849; p = 0.006; μ2 = 0.174). Interestingly, the N270 component appears more negative across match conditions within the right electrodes compared with the left electrodes only for right hemifield trials (F(1,23) = 6.034; p = 0.022; μ2 = 0.208).
Hemisphere-specific amplitude modulations for the probe responses [amplitudes in microvolts (uV)]. a, During left hemifield trials, the N170 component appears within left electrode sites, whereas during the right hemifield tracking, the N270 modulation has a right hemispheric bias. b, Left–right electrode difference waveforms for left hemifield tracking trials and right–left difference waveforms for right hemifield tracking trials. The N170 appears to originate from the left hemisphere, and the N270 has a right hemispheric bias (significant results are denoted using an asterisk while trends are denoted using a dagger).
Given the N170 component exhibiting the match effect during left hemifield tracking mostly within the left hemisphere and the N270 component being more negative for right hemifield tracking within the right hemisphere, N170 and N270 amplitude variations were assessed as a difference wave by subtracting left-minus-right electrode signals for left hemifield trials and right-minus-left electrode signals for right hemifield trials (Fig. 5b). Hemisphere-specific N170 and N270 amplitude modulations were assessed using a three-factorial rANOVA across factors “component,” “hemifield,” and “match.” This analysis confirmed a location bias for the N170 within the left hemisphere and the N270 within the right hemisphere, respectively, as a strong trend toward an interaction between “component” and “hemifield” (F(1,23) = 3.635; p = 0.069; μ2 = 0.136). The detailed pattern of the right-biased N270 difference wave during right probes across match conditions shows a strong linear variation along all four match conditions (linear contrast: F(1,23) = 9.634; p = 0.005; μ2 = 0.295) with a higher amplitude for a greater mismatch between the probe and targets (M0–M3: t(23) = −3.067, p = 0.005, μ2 = 0.290; M1–M3: t(23) = −2.379 p = 0.026 μ2 = 0.197). Please note that the deviant high amplitude for the full-match condition during this time range, observed across all posterior electrodes, disappears in the difference waveforms. The modulation of the early N170 difference wave for the left hemifield trials across the match conditions just fails to become significant (F(3,69) = 2.543; p = 0.071; μ2 = 0.100). During full-match trials, however, the amplitude between 110 and 140 ms of the difference waveform is more negative than that during M0 (t(23) = 2.743, p = 0.012, μ2 = 0.246) and M2 (t(23) = 2.168, p = 0.041, μ2 = 0.17) trials. For the same time range, an amplitude modulation can be observed for the right hemifield trials (F(3,69) = 3.741; p = 0.019; μ2 = 0.140) with a higher negativity for the M2 compared with the M3 condition (t(23) = 3.045; p = 0.006; μ2 = 0.287).
Discussion
The present study investigated potential functional dissociations between location- and object-based representational mechanisms during unilateral MOT within the left or right hemifield. The electrophysiological markers for location-based and object-based tracking strategies established previously (Merkel et al., 2014, 2022) permitted the assessment of the relative contributions of both processes for tracking within the left or right hemifield: while tracking three out of six visually identical objects, the subjects consistently exhibited a sustained small CDA amplitude in combination with an early N170 amplitude modulation for subsequent probes that matched fully the relevant set of target items only when the task was performed within the left hemifield. In contrast, tracking multiple items within the right hemifield leads to an enhanced CDA amplitude throughout the motion phase with a parametric variation of the N270 amplitude modulation for probe displays with a varying degree of target–probe congruities. Most interestingly, the amplitude of the N170 component toward the fully matching probes for the left hemifield tracking task is related to the behavior toward this fully matching probe between subjects. However, the strikingly different amplitude patterns of electrophysiological components elicited during the probes of the left and right hemifield tracking did not translate into differences in performance within the individual subjects.
Recent findings highlight the existence of an object-based tracking mechanism in maintaining the relevant set of targets during a visual tracking task (Merkel et al., 2015, 2017, 2022). Predicated on the polygon idea proposed by Yantis (Yantis, 1992), such a mechanism would build and maintain an abstract, morphing object-based representation of relevant (target) items throughout the tracking phase. A visual probe matching exactly this abstract configuration in a bilateral tracking task was shown to elicit an exclusive modulation of an electrophysiological component—the N170. The sources of this effect originate from the ventral occipital cortical areas (Merkel et al., 2014, 2022). At the same time, the location information for the individual relevant (target) items is maintained during tracking as well, which is indicated by a later component—the N270. Its amplitude modulates with the degree of congruity between probed and tracked items (Merkel et al., 2014, 2022). The current data show a functional and representational dissociation of these two components suggesting the coexistence of two mechanistically different but parallel tracking mechanisms. The N170, indicating an object-based representation of the targets during the probe, was observed when the task was performed within the left hemifield. On the other hand, a modulation of the N270, indicating the maintenance of location-based information, was present during the presentation of the probe when the task was performed within the right hemifield. In the present EEG recordings, these two mechanisms appear to be segregated depending on the spatial location of the tracking task. MOT in the left visual field is performed in an object-based manner while the same task in the right visual field in a location-based manner.
The difference of the sustained CDA modulations following motion onset during the tracking task within the left hemifield and the right hemifield supports this interpretation. The absolute CDA amplitude does correlate with information load during the delay of a working memory task (Vogel and Machizawa, 2004; McCollough et al., 2007) and general bilateral tracking tasks (Drew et al., 2009, 2013). Dynamically changing representations during tracking modulate the CDA hereby “online” (Drew et al., 2012). Compared with the CDA described in visual working memory tasks, the MOT-CDA seems to exhibit a general, load-independent, negative offset which has been argued to be present due to spatial motion (Drew et al., 2011). Such an interpretation would be consistent with the present large negative deflection of the CDA during tracking solely based on item-based spatial information but not abstract object-based information. The currently observed smaller CDA amplitude during left versus right hemifield tracking is furthermore in line with the notion of grouping of the relevant information into an abstract object. Previous work has argued for the CDA to indicate the amount of integrated objects to be maintained (Luria and Vogel, 2011, 2014) and that motion as a feature can facilitate grouping during the retention period (Balaban and Luria, 2015, 2017). Earlier work from our lab dovetails with the currently observed pattern of CDA amplitude modulations for location-based and object-based tracking. In a bilateral tracking task, we observed a sustained γ synchronization during the tracking phase only in subjects employing a location-based tracking strategy, while subjects performing object-based tracking exhibited no such modulation of γ oscillations during the motion phase of the tracking task (Merkel et al., 2022). In general, CDA amplitudes as well as γ oscillations during retention periods were suggested to correlate with information load during the retention period in working memory (Jokisch and Jensen, 2007; Honkanen et al., 2015) and object tracking tasks (Rouhinen et al., 2013).
Hence, maintaining all three individual items during right hemifield tracking indicated by a larger sustained CDA relates to a parametric N270 modulation during the subsequent probe, while during the left hemifield tracking, maintaining the relevant items as one integrated object determines an early match negativity toward the abstract target entity during the probe.
The discussion of hemifield independence in MOT to date revolves around processes with mechanistic restrictions that are equally applicable to both hemifields. In this context, adding relevant items to one of the hemifields during the tracking task adds costs to the contralateral hemisphere in the form of either increased local spatial interference (Alvarez and Cavanagh, 2005; Franconeri et al., 2008; Liu et al., 2009; Stormer et al., 2014), reduced hemifield-specific attentional (multifocal or unifocal) resources (Alvarez and Franconeri, 2007; Howe et al., 2010; Chen et al., 2013; Holcombe and Chen, 2013), or increase in ipsilateral parietal disinhibition (Battelli et al., 2009). The core principle of the bilateral tracking advantage so far is rooted in the idea of a parallel and equally applicable tracking mechanism for the left and right hemifield. Based on the current findings, we propose that the bilateral field advantage during object tracking might arise due to independent modes of representation operating in parallel but with a different functional bias for each hemifield. This bias may be facilitated by hemispheric specialization in representing object-based and location-based visual information.
We observe a hemispheric dissociation of the site of unilateral probe responses with the N170 component mostly originating within the left hemisphere and the N270 amplitude pattern being enhanced within the right hemisphere.
A general perceptual bias for processing location-based and object-based visual information within the right and left hemispheres, respectively, was reported in earlier studies (Marsolek, 1995; Marsolek et al., 1996; Burgund and Marsolek, 1997, 2000). Differential activation patterns within the left and right hemisphere in the processing of abstract and concrete visual information were also observed (McMenamin et al., 2015). Similarly, spatial relations are judged more categorically for stimuli presented to the left hemisphere, while their concrete distance is processed more efficiently when presented to the right hemisphere (Hellige et al., 1989; Kosslyn et al., 1989).
Attentional control mechanisms also exhibit similar differences in hemispherical biases related to object-based and location-based processing. Shifting attention between objects does add additional strain in patients with left parietal lesions but not with right parietal lesion (Egly et al., 1994). Likewise, object-based cueing involves preferentially the cortical attention network of the left hemisphere (Vandenberghe et al., 1997; Arrington et al., 2000; Wilson et al., 2005), while spatial shifts tend to engage the right hemisphere (Fink, 1997; Pollmann and Morrillo, 2003). In a previous functional imaging study, even tracking multiple objects bilaterally seems to show a slight right parietal bias for load-dependent activation over task-dependent activation (Culham et al., 2001; Fig. 3). A study from our group found that patients with left hemisphere lesions showed an N270 response, while it was absent in patients with right hemisphere lesions in a central MOT task. This finding also argues for a functional segregation of representational processes across hemispheres where location-based MOT is dependent on an intact right hemisphere (Lesch et al., 2020).
Importantly, these studies by no means demonstrate a complete functional dissociation between object-based and location-based visual processing, but rather a representational bias for the same information within both hemispheres. Likewise, we observe different amplitude patterns of the N170- and N270-evoked components while tracking within the left or right hemifield across both ipsi- and contralateral, posterior electrodes.
We assume that the extensive interhemispheric communication between visual areas (Dougherty et al., 2005; Shen et al., 2015; Roland et al., 2017) in the current case eventually leads to the very same guided behavioral patterns toward the probe conditions irrespective of the hemifield in which the tracking task occurs. The intriguing situation emerges in which different CDA amplitudes during tracking together with different N170 and N270 amplitude patterns toward probe stimuli within the left or right hemifield do no elicit a different behavioral outcome. Error rates for tracking left and right are almost identical within the subjects. A dissociation of N1 patterns and their associated behavior has been observed in a bilateral tracking task before with different N1 amplitude modulation patterns under different load conditions that nevertheless lead to the same response patterns (Doran and Hoffman, 2010 - Exp. 2/3). Interhemispheric integration of visual processing at later stages of processing would ensure that the differences in representational modes of targets in a hemifield-specific tracking task are compensated for.
In contrast, subjects with improved behavioral responses toward probes that fully match the set of target items (full match) exhibit enhanced N170 response amplitude toward fully matching probes as well. This is consistent with previous descriptions of the N170 during bilateral tracking (Merkel et al., 2014, 2022).
The current results indicate different representational processes for MOT within left and right visual hemifields. These segregated location- and object-based processes are then integrated to underlie a mostly uniform behavioral pattern independent of the spatial location where the task was performed. These findings suggest that the bilateral field advantage during tracking does not necessarily implicate two continuous but separate resources in each hemisphere but rather two different modes of neural representation.
Data Sharing
All data the analyses and interpretation are based on will be made available online via OSF repository: https://osf.io/z63bd/?view_only=3c93ee1dec2048889be49a6dce541453.
Footnotes
The authors declare no competing financial interests.
The current study was supported by the Deutsche Forschungsgemeinschaft Grant SFB1436/B05.
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
References
In this issue
