A hierarchical model of goal directed navigation selects trajectories in a visual environment

Abstract

We have developed a Hierarchical Look-Ahead Trajectory Model (HiLAM) that incorporates the firing pattern of medial entorhinal grid cells in a planning circuit that includes interactions with hippocampus and prefrontal cortex. We show the model’s flexibility in representing large real world environments using odometry information obtained from challenging video sequences. We acquire the visual data from a camera mounted on a small tele-operated vehicle. The camera has a panoramic field of view with its focal point approximately 5 cm above the ground level, similar to what would be expected from a rat’s point of view. Using established algorithms for calculating perceptual speed from the apparent rate of visual change over time, we generate raw dead reckoning information which loses spatial fidelity over time due to error accumulation. We rectify the loss of fidelity by exploiting the loop-closure detection ability of a biologically inspired, robot navigation model termed RatSLAM. The rectified motion information serves as a velocity input to the HiLAM to encode the environment in the form of grid cell and place cell maps. Finally, we show goal directed path planning results of HiLAM in two different environments, an indoor square maze used in rodent experiments and an outdoor arena more than two orders of magnitude larger than the indoor maze. Together these results bridge for the first time the gap between higher fidelity bio-inspired navigation models (HiLAM) and more abstracted but highly functional bio-inspired robotic mapping systems (RatSLAM), and move from simulated environments into real-world studies in rodent-sized arenas and beyond.

Introduction

The ability to successfully navigate to a predefined location is often a life crucial task for many higher order organisms. The goal location might be a food source, a temporary shelter, a nest, or some other desired location. Squirrels are effective at rediscovering their previously stashed food sources (Jacobs & Liman, 1991). Rats can learn to revisit or to avoid known food locations (Brown, 2011, Olton and Schlosberg, 1978). Mice learn to avoid an unpleasant environment, such as a water-maze, by finding an out-of-sight escape platform after only a handful of learning trials (Morris et al., 1982, Redish and Touretzky, 1998, Steele and Morris, 1999). If a visible goal location is in the field-of-view of the agent, the navigation task becomes trivial: The agent proceeds towards the visible goal location avoiding potential obstacles on the way. However, if the goal location is out of visual range or hidden (as in the water-maze) then navigation mechanisms based on cognitive capabilities that can exploit the previously encoded and currently out of view goal location become important to guide the agent to the goal. Such a navigation mechanism would not necessarily need to pinpoint the goal location. It would be sufficient to guide the agent to the general goal location neighborhood such that the goal is in the visual range of the agent. Consequently, the visually driven navigation system can take over to home the agent into the goal location, an approach that has been used successfully by the robotic mapping system used in this research (Milford & Wyeth, 2009).

There is compelling evidence gathered from physiological and behavioral data suggesting the existence of spatial cognitive mechanisms in the brain representing the agent’s spatial environment and aiding it during goal-directed navigation experiments. The entorhinal cortex and hippocampus play a role in goal-directed behavior towards recently learned spatial locations in an environment. Rats show impairments in finding the spatial location of a hidden platform in the Morris water-maze after lesions of the hippocampus, postsubiculum, or entorhinal cortex (Morris et al., 1982, Steele and Morris, 1999, Steffenach et al., 2005; Taube, Kesslak, & Cotman, 1992). Recordings from several brain areas in behaving rats show neural spiking activity relevant to goal-directed spatial behavior, including grid cells in the entorhinal cortex that fire when the rat is in a repeating regular array of locations in the environment falling on the vertices of tightly packed equilateral triangles (Hafting, Fyhn, Molden, Moser, & Moser, 2005), place cells in the hippocampus that respond to mostly unique spatial locations (O’Keefe and Nadel, 1978), head direction cells in the postsubiculum that respond to narrow ranges of allocentric head direction (Taube, 2007), and cells that respond to translational speed of running (O’Keefe, Burgess, Donnett, Jeffery, & Maguire, 1998).

Some of the evidence related to the goal-directed navigation planning include forward sweeping events of spiking activity in rat place cell ensembles that have been observed during vicarious trial and error experiments (Johnson and Redish, 2007, Pfeiffer and Foster, 2013) and sharp wave ripple events during goal-directed spatial tasks (Davidson et al., 2009, Foster and Wilson, 2006, Jadhav et al., 2012, Louie and Wilson, 2001). Furthermore, brief sequences of place cell ensemble activity encoding trajectories from an agent’s current location have been observed to be strongly biased towards the agent’s predicted goal location (Pfeiffer & Foster, 2013).

In this work we combine two biologically inspired models that generate and maintain representations of their environment as collections of simulated spatially tuned neurons such as grid cells and place cells.

The first one of these models is the RatSLAM model (Milford, Wyeth, & Prasser, 2004) which has been implemented on real robotic agents and has been shown to match or outperform the state of the art probabilistic robotic systems in encoding and navigating large environments over long periods of time (Milford and Wyeth, 2009, Prasser et al., 2006). However, the current RatSLAM model is not easily scalable and its goal directed navigation module is less biologically plausible than its Simultaneous Localization and Mapping (SLAM) component.

The second model we use in our work is the HiLAM (Erdem & Hasselmo, 2013), a biologically inspired goal-directed navigation model based on look-ahead trajectories in a hierarchical collection of simulated grid cells and place cells. While HiLAM is highly capable in simulating behavioral goal-directed navigation experiments, it is prone to failure in the presence of noisy and degraded input, since it does not have mechanisms in place to detect and to correct for the stochastic loss of fidelity in its state representation. Consequently, like many other high fidelity computational models, the HiLAM has not been previously tested on real life data.

In this work we combine the RatSLAM model and the HiLAM such that their individual fortes complement each other in generating and maintaining stable spatial maps using real life visual data (RatSLAM) and in using the generated maps for goal-directed path planning in a biologically plausible manner (HiLAM).