Physical Body Orientation Impacts Virtual Navigation Experience and Performance

Discussion

In this study, we demonstrate that physical BO (standing or supine participants) modulates experienced BO and some navigational measures in VR. Our data also show that a standing BO is preferred during spatial navigation, because participants were more likely to experience a standing BO than a supine BO and because they were faster in retrieving locations when they were standing upright (i.e., shorter navigation time). We also corroborate previous reports by demonstrating that showing a virtual avatar in a supine position reduces the conflict between supine physical BO and preferred BO (i.e., standing) and improves an aspect of spatial navigation (i.e., shorter navigation path). Because many navigation-related neuroimaging studies are done in the MRI scanner (Doeller et al., 2010; Kunz et al., 2015; Horner et al., 2016; Stangl et al., 2018; Bierbrauer et al., 2020; Moon et al., 2022), where physical BO is constrained to be supine (Taube et al., 2013; Steel et al., 2021), our study provides important information about experienced BO and potential biases arising from it, which should be considered when spatial navigation studies in the scanner are designed or interpreted.

First, we demonstrated that participants’ physical BO modulated their experienced BO in VR as measured by subjective questionnaire ratings and by the border effect. Concerning BO ratings, we found that when participants navigated the virtual environment without seeing an avatar, their experienced BO was influenced by their physical BO: standing BO ratings increased and supine BO ratings decreased when they were physically standing upright, and vice versa. These subjective data were corroborated by the border effect in navigation behavior (i.e., the distance from the border of the navigation arena during retrieval trials). When navigating to the target location, our participants kept a larger distance from the border while their physical body was in supine versus standing BO (Fig. 2b,c). We note that both conditions were visually identical (no avatar in either condition) and differed only in physical BO. Thus, the border effect cannot result from visual differences between conditions, but rather by the different physical BO, associated with differences in experienced BO: the space occupied by the body expands forward in the supine BO (but see Alsmith and Longo, 2014; Moon et al., 2022). The finding of a positive correlation between the distance from the border with the strength of the participant’s subjective experience of being supine confirms this interpretation (Extended Data Fig. 2-1). Accordingly, we argue that the border effect is an implicit change in navigational behavior when participants are in supine physical BO versus standing. Importantly, this navigational parameter provides an objective and repeated proxy of one’s experienced BO, eliminating potential biases that may arise from explicit questionnaires. This effect is of direct relevance for spatial navigation studies performed during fMRI (Doeller et al., 2010; Kunz et al., 2015; Stangl et al., 2018; Bierbrauer et al., 2020) where supine BO is unavoidable. Apart from altering some aspects of navigation behavior (as shown by the border effect), the supine BO may also affect place/grid cell-related brain activity (i.e., grid cell-like representation in entorhinal cortex; Doeller et al., 2010; Jacobs et al., 2013; Nadasdy et al., 2017; Maidenbaum et al., 2018; Moon et al., 2022) or the activation of other regions involved in spatial navigation such as retrosplenial cortex (Vann et al., 2009; Mitchell et al., 2018; Bierbrauer et al., 2020; Alexander et al., 2023). For example, the border effect strongly suggests that the reference point to which a reference frame in VR is anchored was shifted, and this could possibly lead to a corresponding shift of place/grid field map encoding the location of a virtual agent in VR. Furthermore, BO of one’s physical body could also affect the orientation of the reference frame anchored to it. For instance, “heading forward” may mean navigation to the ceiling when one is supine in MRI, rather than navigation on the horizontal plane as in the virtual arena. Thus, we believe that most of the human virtual navigation studies describe neural activities from the reference frame anchored to the virtual agent in VR, not the physical body. However, these two distinct reference frames (possibly encoded by distinct cognitive maps in MTL) may conflict in some experimental conditions (e.g., Body condition), and supine physical BO might contribute to the conflict. The influence of physical BO on the experienced BO suggest that a virtual agent (even when it is invisible in Nobody condition) and the participant’s body are functionally linked to each other during virtual navigation. The association could be mediated by the intrinsic signals from the physical body (e.g., vestibular and proprioceptive signals; Pfeiffer et al., 2014; Lenggenhager and Lopez, 2015; Park and Blanke, 2019). Thus, direct changes or disruptions to the bodily inputs could influence both the navigation experience (BO) and related navigation behaviors (i.e., border effect). This hypothesis needs further investigation in future studies using experimental manipulation of those bodily signals, such as electrical vestibular stimulation (Sluydts et al., 2020).

Second, our data show that standing is the preferred experienced BO during virtual navigation. We found that a conflict between the physical BO and the experienced BO in VR (as measured by the conflict score) was larger when our participants were physically supine versus standing upright. Thus, when they were physically standing upright, they felt as if they also were standing in the virtual arena during navigation (supported by bodily signals). In contrast, when they were in supine BO in the real world, their reported experience of BO in the virtual arena was more ambiguous (between supine and standing). Considering the influence of the supine physical BO that we discussed above, the observed ambiguity could be the result of incongruence between the preferred BO in VR (i.e., standing upright) and the supine physical BO (arguably, mediated by vestibular and proprioceptive cues from the supine body). We argue that the preferred standing BO during virtual navigation most likely originates from our daily experience of upright position during physical navigation. Human brain mechanisms of spatial navigation could have adapted to the evolutionary change in BO and been optimized for navigation in standing upright BO (also based on body structure, sensory system, and lifestyle distinguished from other animals; Ekstrom, 2015). This was also indirectly supported by the influences of the physical BO and experienced BO on spatial navigation performance in VR. We observed a decrease in time to retrieve and navigate to the target when participants’ physical BO was standing (Fig. 4a), suggesting that standing BO (i.e., preferred BO during navigation as suggested above) facilitates some spatial navigation processes. Alternatively, this effect could also be linked to changes in space perception (e.g., visual vertical judgment; which is possibly related to the prediction of the destination ahead of straight navigation) that have been reported to be better in the upright position (compared with the supine position), by the contribution of vestibular gravitational signals (Lopez et al., 2008; Lopez and Blanke, 2010). Size or distance perceptions have been reported to be better while upright than supine: supine BO makes the size or the distance more underestimated (Kim et al., 2022). Alternatively, although the VR scene presented in the HMD was the same, our participants might have experienced it from an elevated perspective (i.e., as if they were looking down; which is a more likely situation while standing than while supine) when they were physically upright but not when they were supine. An elevated perspective has been associated with faster response times in visuospatial tasks (vs eye-level or lowered perspective; Schwabe et al., 2009). Indeed, a previous study by Ionta et al. (2011) reported that BO in a virtual space experienced by participants in the MRI scanner (i.e., looking-up vs looking-down) could be altered by multisensory bodily signals while keeping the visual input constant. Altogether, the present findings suggest that the human navigation system “has a preference” for navigation in the upright BO, and that—although virtual navigation can simulate natural navigation (as if upright) with some extent of ambiguity—it may nonetheless impact some aspects of navigation in supine BO (as in MRI). It is possible that the fixed visual angle of 15° in our experiment (adjusted to be more upward than usual, but still closer to the case of upright) could affect the preference for the upright BO and the better performance in the same condition. While this may compromise the generality of our findings, our results are still very important as the majority of virtual navigation research uses a visual angle that is almost horizontal.

Finally, we show that also the presentation of a supine avatar during virtual navigation influences both experienced BO and navigation behaviors in VR. Concerning experienced BO in VR, we report that the avatar in the supine position was associated with a stronger sensation of being in supine BO in VR. This finding is compatible with an important role of bodily multisensory cues in virtual navigation experience. It should be stressed that this rather simple experimental manipulation (i.e., showing a supine avatar during virtual navigation) significantly reduced the conflict between the experienced BO and the physical BO when participants were lying supine (which is the adopted posture in fMRI acquisitions), as highlighted by the lower conflict score in Supine-Body condition compared with Supine-Nobody condition. As discussed above, participants’ experienced BO in the Supine-Nobody condition (i.e., the typical condition in MRI) is experienced as ambiguous, arguably because of the incongruency between preferred BO and the influence of supine physical BO. However, our results suggest that the addition of a seen avatar reduces this ambiguity and the related sensory-experiential conflict (i.e., conflict score). Importantly, we show that these subjective BO changes were reflected in the border effect, reproducing previous results in the MRI scanner (Moon et al., 2022). Of note, this border effect, as induced by a supine avatar (Supine-Body condition), further increased the distance from the border compared with Supine-Nobody condition, where we observed the border effect induced by supine physical BO when compared with Standing-Nobody. Thus, the border effect further increased when the subjective experience of being supine in VR was enhanced by (1) supine physical BO and again by (2) the presence of a supine avatar (Standing-Nobody < Supine-Nobody < Supine-Body). This finding was further supported by the significant correlation between the border effects and Q_supine ratings, and vice versa (Extended Data Fig. 2-1). Moreover, seeing a supine avatar also reduced the navigated distance per trial without increasing retrieval errors, suggesting improved spatial navigation in the conditions with an avatar (Fig. 4b) as was the case in the similar study in MRI (Moon et al., 2022). This is again of relevance to human navigation studies using virtual navigation paradigms, and in particular to those paradigms using fMRI (Taube et al., 2013; Steel et al., 2021). Overall, our data suggest that body-related cues (e.g., view of a body posture-congruent avatar) as well as signals from the body systematically evoke conflicts to different degrees depending on the actual conditions; this knowledge should be used to improve and better understand experienced BO and navigation behaviors in virtual navigation, especially when a supine BO is necessary as in the fMRI studies (Doeller et al., 2008, 2010; Taube et al., 2013; Stangl et al., 2018; Bierbrauer et al., 2020; Moon et al., 2022).

Notably, we elucidate the limitations stemming from the presentation of a supine avatar, despite its merits we have shown above. The introduction of a supine avatar may influence a wide range of neural activities from the rudimentary level visual perception to higher-level cognitions related to sense of self and spatial navigation, as already reported in the previous studies (Iriye and Ehrsson, 2022; Moon et al., 2022, 2023). Therefore, it is essential to properly consider these aspects before adopting the supine avatar in experimental design.

Studies on imagined navigation (Bellmund et al., 2016; Horner et al., 2016), in which participants’ imagined BO was not restricted, showed similar neural correlates to virtual navigation. One might argue that these findings suggest a limited influence of supine BO in the scanner on the human navigation system. However, similar neural correlates between the imagined navigation and virtual navigation might also indicate that virtual navigation without any physical displacement is more similar to “imagined navigation” than to “upright real-world navigation.” Thus, without direct comparison between the neural correlates of supine virtual navigation and those of physical spatial navigation, we cannot really conclude that there is no major influence of the supine BO on the neural activity in the human brain.

Collectively, our results highlight the importance of the physical BO and the relevant visual cue (i.e., BO-indicating body view) on virtual navigation by showing their influence on both subjectively experienced BO and the navigation behaviors in VR. Through this study, we confirm that standing BO is preferred during navigation with solid evidence, which has been considered so far without much evidence. By comparing the conventional condition in MRI with others (including the preferred condition) across a range of aspects, our data provide a more fine-grained understanding of the potential, but often overlooked, effects that could be caused by the constraint in MRI. Importantly, the present data show that the addition of a seen avatar can be a simple but powerful method to resolve the ambiguity of the experience of BO in MRI and possibly to improve some navigation performance (i.e., Moon et al., 2022). Our results strongly suggest that multisensory body-related aspects should also be cautiously considered in the experimental design of any virtual navigation study. Finally, we propose the border effect as a robust measure that allows assessing the potential biases and confounds in experienced BO during virtual navigation, and thereby helps control them.