Introduction

Imagine trying to locate your friends while watching a crowded street parade. If you catch only a glimpse of them in the crowd before they are obscured by others, remembering how far they were from a nearby building (a stable landmark in the external world) may provide a useful cue to help you localize them later. Indeed, this relative (or allocentric) information may prove more valuable than memory of their location within your visual field (egocentric information). However, the nature of storage of allocentric information, and its interaction with other forms of visual memory, have not been clearly established.

Interruptions in sensory input represent a frequent challenge to the visual system, whether due to our own actions, such as an eye-movement or blink, or changes in the external world, such as object occlusions or the disappearance of a transient stimulus. Visuospatial working memory (VSWM) helps bridge these discontinuities, by allowing us to retain sensory information about visual objects even when they are no longer visible. However, the capacity of VSWM to store information is limited. Even when explicitly instructed to remember specific stimuli—in anticipation of an interruption—individuals make substantial errors in both their ability to detect the occurrence of a change [1–3] and to reproduce remembered features [4,5]. Error increases monotonically as the number of items increases, and this holds true for recall of object locations as well as features [6]. This is consistent with models in which objects compete for allocation of a limited representational resource [4,7–9].

Representations of visual information in early visual cortex are inherently egocentric, emerging directly from the projection of the external world onto the retina. Consequently, the spatial information associated with visual processing is at least initially gaze-centered, encoding locations relative to the observer, and decreasing in resolution as the distance from the fovea increases [10]. This retinotopic spatial encoding appears to be preserved throughout much of the brain, particularly in dorsal brain regions that support the execution of actions towards remembered locations [11,12] (but see [13,14] with respect to ventral areas). Indeed, it is actively debated whether spatial information is ever encoded in non-retinotopic reference frames [15–18]. The point of contention is whether separable representations of stimuli are encoded—within distinct neural populations and potentially within different neural pathways—or if the apparent use of other representations merely reflects timely manipulations of egocentric information [16]. Important evidence has come from studies of motor action, which have shown that movement errors are reduced in the presence of visual landmarks, and suggested motor programming reflects the combination of egocentric (retinotopically encoded relative to current gaze) and allocentric (relative to external landmarks) spatial cues [19–21].

In this paper we investigate how egocentric and allocentric VSWM representations interact. Specifically, in view of the limited capacity of VSWM, we examine the impact of encoding additional allocentric spatial information in the form of distance from a visual landmark. We show that the behavioral data is consistent with an optimal integration of an egocentric signal, independent of the landmark, with an allocentric signal that degrades with distance from the landmark. We further show that allocentric information does not compete with egocentric information for storage, indicating that the two sources of information rely on independent memory resources.

Results

A stable landmark enhances spatial working memory precision

In Experiment 1 we investigated the influence of a visual landmark on spatial working memory for different numbers of remembered objects (set size: 1, 2 or 4). Participants used a computer mouse to report the remembered location of one item from a memory array, identified by color (Fig 1A). Examining spatial recall precision in the absence (LM-ABSENT) and presence (LM-PRESENT) of a stable visual landmark, we observed a substantial reduction in the variability of memory reproduction for stimuli located near the landmark, at all set sizes (Fig 1B–1D). These changes occurred in the absence of systematic shifts in bias (S3 Fig) and indicate that the presence of the landmark gave participants access to additional information to facilitate recall.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Experiment one.

(A) LM-PRESENT design. Participants memorized the locations of colored disks in the presence of a landmark (a larger dark gray disk; note object sizes are exaggerated for visibility). (B-D) Data points indicate mean variability in location recall for set sizes 1, 2 and 4 respectively, with predictions of the optimal integration model overlaid (colored lines). Note the model captures both the reduction in variability near the landmark, and the plateau in variability at far landmark-target separations. LM-PRESENT data is shown in red, LM-ABSENT in blue. Errorbars and patches indicate 95% CI. Gray dots indicate size of the landmark on the x-axis scale. (E-H) Box plots depicting parameter estimates for the best-fitting model (notch represents 95% confidence interval on the median). Note the decrease in egocentric precision (E), decrease in allocentric precision (F) and increase in lapse rate (H) associated with increasing set size, while the best-fitting model exhibited no changes in the allocentric scale (rate of decay with distance), which is therefore estimated by a single parameter (G).

https://doi.org/10.1371/journal.pcbi.1006563.g001

We implemented a simple cue-combination model to investigate whether this spatially selective improvement in precision could be captured by optimal integration of independent egocentric and allocentric spatial encodings (Fig 2; see Methods). In the model, the precision of the allocentric signal diminishes with distance to the landmark from a peak A_max at rate A_scale, while precision of the egocentric signal is independent of distance. The model also includes a lapse rate to capture random responding and “swap” errors [4,9]. The fit of the optimal integration model is shown as solid lines in Fig 1B–1D. This model provided a substantially better fit to data than a reduced model with allocentric encoding omitted (ΔAICc = 662).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Ideal observer model.

(A) The visual working memory decoding model, in which egocentric and allocentric estimates are integrated depending on their respective reliabilities. While precision of the egocentric component is set by P_ego, the allocentric precision is determined by two parameters: the peak precision obtained when landmark and target are aligned (A_max), and a scale parameter describing how quickly allocentric precision declines with increasing landmark-target distance (A_scale). The model further incorporates a fixed probability of lapsing (p(lapse); responding at random relative to the target), giving four free parameters in total. (B) Precision of egocentric (blue) and allocentric (green) estimates shown as a function of distance from the landmark. While egocentric precision is constant, the precision of allocentric information decreases exponentially as the distance increases. The precision of the integrated estimate (red) is equal to the sum of precisions of the individual components.

https://doi.org/10.1371/journal.pcbi.1006563.g002

Consistent with previous studies [4,5,22], the precision of the egocentric signal declined with increasing set size (Fig 1E; comparison to model with fixed precision: ΔAICc = 234; linear regression slope = –18.4 ± 2.7 (M ± SE), t(11) = 6.89, p < 0.001). Similarly, model comparison indicated a decrease in peak precision of the allocentric signal with set size (Fig 1F; ΔAICc = 45.94; linear regression slope = -461 ± 78; t(11) = 5.88; p < 0.001). There were no changes across set size in the rate with which precision of the allocentric signal scaled with distance (Fig 1G; ΔAICc = 18.45). The lapse rate increased with set size but accounted for only a very small fraction of trials (Fig 1H; ΔAICc = 139; slope = 0.014 ± 0.003; t(11) = 4.48; p < 0.001).

Landmark persistence

A plausible alternative account of the landmark effect is that the presence of the salient landmark in the initial array biased encoding towards memoranda in its vicinity. In Experiment 2 we tested a condition (LM-ENCODE) in which the landmark was visible only during the presentation of the memory array (set size 4). This condition was interleaved with other conditions such that participants did not know during encoding whether the landmark would disappear. We observed no landmark-related improvement of precision in this condition (Fig 3A) and a reduced model with no allocentric signal provided a better fit to data than the optimal integration model (ΔAICc = 16.56). This confirms that the landmark benefit is a result of the use of allocentric spatial information and not due to encoding bias. If items presented in the vicinity of a landmark were preferentially encoded, or encoded with enhanced precision, we would have seen a benefit for those items even when the landmark was absent during the response phase.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Experiment two.

(A-C) Mean variability in memory recall across participants for LM-ENCODE (A), LM-GAP (B) and LM-PRESENT (C) conditions (with LM-ABSENT shown on the right in blue). There is a substantial reduction in variability in the vicinity of the landmark irrespective of whether the landmark was persistently (LM-PRESENT) or intermittently shown (LM-GAP), but no apparent influence of the visual landmark when it was only visible during encoding (LM-ENCODE). Predictions of the best-fitting model are overlaid.

https://doi.org/10.1371/journal.pcbi.1006563.g003

Does the use of the landmark depend on its continuous presence during the memory delay? We tested a condition (LM-GAP) in which the landmark disappeared at the offset of the sample array and only reappeared at the time of the probe. We found a robust landmark effect in this condition (ΔAICc = 135 compared to reduced model; Fig 3B). Comparing the LM-GAP condition to one in which the landmark was continuously present (LM-PRESENT, as in Exp 1) revealed no difference in peak precision of the allocentric signal (ΔAICc = 9.34 favoring a model with shared A_max parameter between conditions) but some evidence for a difference in the rate of change of precision with distance (ΔAICc = 13.19 favoring a model in which A_scale differed between conditions; median A_scale 20.9% lower in LM-GAP condition). Exp 2 also replicated the finding from Exp 1 that the presence of a landmark incurred no cost to the precision of egocentric memory (model with cost performed worse, ΔAICc = 23.31; parameter estimates for Exp 2 are shown in S4 Fig).

A final possibility is that the benefits observed in the LM-GAP and LM-PRESENT conditions arose from enhanced retrieval of items whose previous locations were close to the landmark’s location at the time of the probe, perhaps due to internal attention being drawn to that location in memory. We therefore carried out an additional control experiment (see S1 Text and S1 Fig) which included an LM-RETRIEVE condition, in which the landmark was visible only at the time of response, and not during the presentation of the memory stimuli. In this condition, allocentric information about the items’ locations relative to the landmark could not be encoded from the memory array, but any effect of the landmark on internal attention at the time of retrieval should still be present. We found no evidence for a landmark-related improvement of precision in this condition, and a reduced model with no allocentric signal for the LM-RETRIEVE condition provided a better fit to data than the optimal integration model (ΔAICc = 39.86).

Considering in combination the results of LM-ENCODE (no benefit if the landmark is present only during encoding), LM-RETRIEVE (no benefit if the landmark is present only during retrieval) and LM-GAP (clear benefit if the landmark is present during both encoding and retrieval), our results strongly indicate that landmark-related benefits are due to encoding and subsequent retrieval of allocentric (relative position) information present in the memory array.

Cue conflict

To provide a strong test of the optimal integration model, in Experiment 3 we implemented a variant of the LM-GAP condition in which the landmark reappeared at a location displaced through a small distance (6° on the circle) from its original position (LM-SHIFT; Fig 4A). According to the model, this manipulation should introduce a conflict between egocentric and allocentric spatial information, with the allocentric estimate shifting with the visual landmark. As a result, we predicted that participants would show systematic biases in their localization responses in the direction of the shift, with the strength of the bias determined by the relative reliability of each cue.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Experiment three.

(A) Example LM-SHIFT trial. When the landmark returned, it was shifted by either 6° clockwise or counter-clockwise (exaggerated above for clarity; light gray disk illustrates previous landmark location and was not visible in the experiment). If participants used the post-shift location to anchor their allocentric estimates, we would expect their responses to be biased in the direction of the displacement, with the magnitude related to the reliability of the allocentric cue. (B) The response bias measured in the direction of the shift (magnitude 6° indicated by gray line), as a function of distance from the landmark. The data reveals a consistent bias in the direction of the displacement, which may be either towards or away from the visible landmark location. Bias magnitude depended on distance from the landmark with a peak of ~80% of the shift. (C) Spatially specific decreases in response variability near the landmark in LM-SHIFT. Note that for clarity the bias was subtracted prior to calculation of the median absolute deviation. The model predictions (overlaid) simultaneously capture landmark effects on both bias (B) and variability (C), without any additional free parameters.

https://doi.org/10.1371/journal.pcbi.1006563.g004

We found a clear landmark effect in the LM-SHIFT condition (ΔAICc = 179 compared to reduced model), and the recalled locations of items presented close to the landmark were strongly shifted in the direction of landmark displacement (Fig 4B). The optimal integration model accurately predicted changes in both bias and variability with landmark distance (Fig 4B&4C). The additional fitting of bias required no extra parameters, relying on the same reliability estimates used to calculate variability. Parameter estimates for Exp 3 are shown in S5 Fig.

Examining the effect of shifting the landmark on precision of the allocentric signal (by contrasting LM-GAP and LM-SHIFT conditions), revealed a reduction in both the peak precision of allocentric information (ΔAICc = 19.03 favoring a model in which A_max differed between conditions; median A_max 74.0% lower in LM-SHIFT condition) and the rate at which it decayed with distance (ΔAICc = 21.6 favoring a model in which A_scale differed between conditions; median A_scale 35.1% lower in LM-SHIFT condition).

Finally, as in previous experiments, we examined whether there was evidence for a precision cost on egocentric encoding. We found a ΔAICc of 28.11 favoring the model without cost, further confirming that allocentric and egocentric information are independently stored.

Discussion

Natural scenes rarely contain only a single item, and are instead frequently populated by multiple stable objects, any of which could act as a visual landmark for locations we need to remember. However, how the brain stores and uses this information is only partially understood. Here, using simple experimental displays, we have demonstrated a spatially specific enhancement of localization precision in the vicinity of a landmark, consistent with observers using not only memory of the egocentric spatial locations of stimuli, but also memory of their locations relative to other objects in the environment (allocentric information). We further investigated the consequences of encoding this additional information into VSWM, in light of established limitations on working memory resources [4,7,9,22].

It is now well established that increasing the number of items to be remembered increases variability in recall of their features and locations [4–6,22,23]. If egocentric and allocentric information compete for access to the same limited memory resource, then the introduction of a landmark (with the consequent encoding of additional allocentric information) should reduce egocentric memory fidelity. While we observed a spatially specific increase in recall precision for items near the landmark, memory items located far from the landmark were recalled just as precisely as when the landmark was absent. Thus, our results demonstrate that the presence of the landmark had no influence on the fidelity of egocentric memory representation. Instead, the presence of a landmark appeared to grant access to an additional allocentric source of spatial information. To confirm this finding, we incorporated a cost parameter into a cue-combination model, which allowed the reliability of egocentric information to be degraded in the presence of a landmark. In three separate experiments, we consistently found a model with no cost provided the best description of the data, a result further supported by a meta-analysis of cost estimates pooled across experiments (S2 Text and S2 Fig). Thus, rather than directly competing, our results suggest that egocentric and allocentric locations are encoded independently and draw upon separate memory resources.

We also examined competition within each representation as the number of items encoded increased. For egocentric spatial information, this competition led to a gradual decrease in the reliability of spatial estimates (P_ego) as set size increased, consistent with previous results [6,7]. Similarly, we found that increasing set size led to a decrease in the maximum reliability of allocentric spatial information (A_max), indicating that the recollection of multiple relative locations also reflects a distribution of limited memory resources. In contrast, set size had no influence on the rate at which allocentric precision diminished with distance (A_scale). So, while the number of items in memory determined the overall reliability of allocentric information, the relationship between landmark distance and reliability appears to be fixed.

We observed a substantial, spatially specific improvement in recall precision even when the landmark was hidden during the memory delay (LM-GAP). While this manipulation did not change the maximum precision of allocentric information (A_max), there was a decrease in the spatial scale over which the precision enhancement was observed (A_scale). The interruption in landmark persistence may have reduced the perceived stability of the visual landmark, introducing uncertainty as to whether the returning landmark had reappeared at the same location or if it should be considered the same object [24–27]. This is consistent with a study of reach programming in which landmark locations were jittered [19], which demonstrated that participants are sensitive to perceived landmark stability and adjust reliance on allocentric information as a result.

Importantly, when the landmark was only present during encoding (LM-ENCODE)—and not during recall—there was no advantage in localization compared to conditions without a landmark (LM-ABSENT). This means the landmark benefit cannot be explained simply by enhanced encoding of items in its vicinity, as this would predict improved localization irrespective of the landmark’s presence at recall. Furthermore, because the interleaved LM-GAP and LM-ENCODE conditions were indistinguishable until the time of recall, we can be certain that the same amount of allocentric information was encoded in both conditions. Therefore, the absence of a benefit in LM-ENCODE must arise from an inability to use this information. A landmark only appears to improve localization performance when at the time of testing the recalled allocentric distance can be anchored to the visible location of the landmark itself.

We also saw no benefit when the landmark was present only at the time of retrieval (LM-RETRIEVE), a condition in which allocentric information relating memory items to the landmark could not have been stored. This also demonstrates that the landmark benefit is not due to an enhancement of retrieval for items whose location in memory falls close to the probe, as such an enhancement would be observed regardless of whether the landmark was visible during encoding.

The lack of difference in recall precision between LM-ENCODE and LM-ABSENT conditions enables two additional observations to be made. First, participants apparently did not encode the egocentric location of the visual landmark itself, as its presence had no influence on precision (i.e. there was no set-size effect diminishing precision in the LM-ENCODE condition). This is consistent with both the task instructions and our conclusion that, to be useful for localization, the landmark had to be present at test. Second, given that we know allocentric information was encoded (but not used) in the LM-ENCODE condition, any competition between allocentric and egocentric information would be readily apparent as a decrease in precision compared to LM-ABSENT. The absence of such a difference is itself strong additional evidence for the independence of egocentric and allocentric spatial representations.

Across a variety of different conditions, an optimal integration model accurately described how allocentric and egocentric information were combined to generate estimates of location. Based on exponential decay of allocentric precision with distance from the landmark, this model captured not only how recall variability changed as a function of distance, but also the distance-dependent recall biases that emerged when egocentric and allocentric cues were put in conflict (Exp 3). Specifically, when we covertly changed the location at which the landmark reappeared (LM-SHIFT), we observed systematic shifts in recall position based on the distance of the recalled item from the landmark, with memoranda near to the landmark biased substantially in the direction of the displacement. Critically, these biases were consistent with a displacement in localization (i.e. relying more strongly on the allocentric information), not with an attractive bias to the landmark’s location. This result adds considerable support for our model, demonstrating that the integration of egocentric and allocentric information was close to optimal, and reinforces the conclusion that allocentric and egocentric estimates are encoded separately and as such associated with independent noise. Such integration models have proved invaluable in the study of multisensory integration (e.g. [28,29]), and several studies have used similar methods to describe the integration of allocentric and egocentric information in reaching and eye-movements to a single target [19–21,30,31]. However, to our knowledge, no previous study has quantitatively examined the consequences of encoding both egocentric and allocentric information on memory fidelity, determined how the precision of allocentric spatial information varies with set size, nor quantified the relationship between distance and the reliability of allocentric information.

The results of the LM-SHIFT condition also provide evidence against any alternative account of our findings based on local changes in the encoding or retrieval of items in the vicinity of the landmark. The observed biases in recall could not be the result of a difference in how items near to the landmark were encoded, because the biases were specifically in the direction of the landmark displacement, which was entirely unpredictable at the time of encoding. Equally, the biases could not be a consequence of proximity of items in memory to the location of the landmark at the time of retrieval, because this location was also randomized with respect to displacement direction. In contrast, biases in the direction of displacement are fully compatible with an account in which observers remember the relative deviation of items from the landmark, and a model in which this allocentric memory provides an additional, independent source of information for item localization provided an excellent quantitative account of both the biases and the enhancements in precision associated with proximity to the landmark (Fig 4B&4C).

Other than the systematic localization shift in the conflict condition (LM-SHIFT)—which was well characterized by our optimal integration model—we observed no consistent biases in localization due to the presence of the landmark in any of our tasks. However, several previous papers have reported biases, both attractive and repulsive, linked to visual landmarks, as well as fixation and attended, non-fixated locations [32–41]. For example, in a task in which participants were required to make a pointing movement to the location of a single flashed target in the presence of a continuous visual landmark, Diedrichsen and colleagues [32] found that movement endpoints were both repulsed from the location of the landmark and less variable in its vicinity. However, in a similar condition to our LM-ENCODE, in which the landmark was only present during the encoding stage, they observed the presence of the same systematic biases without the improvement in precision. This suggests that the systematic biases they observed are independent of the spatially-specific improvements in precision that occur for items near a landmark. The absence of consistent landmark-related biases in the present experiments may be a consequence of preventing eye movements, ensuring both landmarks and stimuli were equally eccentric, and confining responses to the stimulus circle, all of which would tend to minimize the impact of attentional spatial distortions. Some dynamical models of working memory predict attraction or repulsion between items in memory depending on their separation [42–44], but we would not expect the same principles to apply to the landmark, which as discussed above does not appear to itself be stored in memory.

Our experimental manipulations compared recall in the presence and absence of a landmark object. This allowed us to quantify the performance changes resulting from adding a new source of allocentric information to the scene, irrespective of whether allocentric information was also encoded in the LM-ABSENT condition. One possibility is that participants encoded item locations relative to other elements that remained visible throughout the trial, i.e. the screen edges or the fixation spot. Although we elected not to obscure these elements (removing the fixation spot would have made it impractical to control eye movements), we think a contribution of this relative information to our egocentric estimate is unlikely. In our task, the reliability of allocentric information diminished rapidly with distance from the landmark: indeed, the localization of memoranda more than 4.7° of visual angle (46° on the circle) from the landmark received negligible benefit from allocentric information (< 5% change in precision from no-landmark performance). This renders the distance from the memoranda to the screen edges (min 6.5°) or the fixation dot (6°) too far to exert any meaningful influence on localization. Previous studies have attempted to estimate a distance threshold beyond which allocentric information no longer has a significant influence, based on qualitative comparisons of conditions with different spatial separations [30,32,45]. Our approach enabled us to identify and quantify a continuous change in the reliability of relative cues that occurs as the distance from the landmark increases.

The format in which allocentric information is extracted from the array and stored in memory cannot be unambiguously determined from our experiments. The simplest account of our results would posit an internal representation of the vector connecting each memory item with the landmark. However, it is possible that other static elements in the participant’s surroundings, or overarching geometric principles such as the fact all stimuli were displayed in the vertical plane of the monitor (defining an observer-independent coordinate frame), influence the representation format also. These issues have been explored primarily in the context of navigation and large-scale spatial cognition [46–49]. The present design could in future be extended to examine corresponding principles in VSWM, for example by presenting two or more landmarks in a single memory array.

Competition in encoding information within a feature dimension has been linked to the normalization of neural population activity [50], and this model has been successful in accounting for set size effects [51]. While this neural account of resource limitations has been extended to incorporate multiple feature dimensions, including spatial location [52], no attempt has been made to distinguish between different spatial reference frames. This work has, however, both confirmed and provided new evidence for a privileged role of spatial information in binding object features [53,54]. Evidence for a specific contribution of allocentric information to object binding has been revealed in change detection tasks in which individual item locations (egocentric) or global spatial layout (allocentric) are separately manipulated. Here, even when explicitly informed that location information was irrelevant, performance was compromised by individual changes in spatial position unless allocentric information remained veridical [1,55–58].

While our observation of set size effects on the precision of allocentric information suggests a commonality in neural representation with other feature dimensions, relative location information may be unique in that it spans objects rather than being associated with a single object. For this reason it is unclear how object file [53] or slot-based models of VSWM [2,9] would be able to incorporate such spatial information. Our model does not attempt to capture transformations between egocentric and allocentric reference frames (e.g. [59]) and this will be an important direction for future investigation, particularly with respect to the effects of self-motion.

Classically, the division between egocentric and allocentric information has been associated with the neuropsychological distinction between the dorsal and ventral visual processing streams [60,61]. While spatial information is encoded in egocentric coordinates throughout the dorsal pathway, the ventral projections into the inferior temporal cortex represent progressively more complex information about object properties, encoded by neurons with decreasing sensitivity to spatial location [61–64] and little retinotopic organization [13,14]. Contemporary research suggests that, rather than being lost, spatial information along the ventral path is instead increasingly represented in terms of the relations within and between objects in the environment [65–67]. Indeed, neuroimaging studies looking for correlates of allocentric coding have frequently identified higher areas in the ventral stream [68–73] as components of a broader distributed network contributing to allocentric representation [65,71]. Hippocampal structures are also implicated in relative spatial encoding, most clearly in relation to navigation, but with growing evidence for a role in coding visual space [59,71,74–78].

Despite these recent findings, the neural coding of allocentric space remains far more poorly understood than egocentric space. We believe the present work provides a computational and experimental framework within which future studies can explore the neural bases of these spatial memory mechanisms.

Methods

Supporting information