Many Paths to the Same Goal: Balancing Exploration and Exploitation during Probabilistic Route Planning

Abstract

During self-guided behaviors, animals identify constraints of the problems they face and adaptively employ appropriate strategies (Marsh, 2002). In the case of foraging, animals must balance sensory-guided exploration of an environment with memory-guided exploitation of known resource locations. Here, we show that animals adaptively shift cognitive resources between sensory and memory systems during foraging to optimize route planning under uncertainty. We demonstrate this using a new, laboratory-based discovery method to define the strategies used to solve a difficult route optimization scenario, the probabilistic “traveling salesman” problem (Raman and Gill, 2017; Fuentes et al., 2018; Mukherjee et al., 2019). Using this system, we precisely manipulated the strength of prior information as well as the complexity of the problem. We find that rats are capable of efficiently solving this route-planning problem, even under conditions with unreliable prior information and a large space of possible solutions. Through analysis of animals’ trajectories, we show that they shift the balance between exploiting known locations and searching for new locations of rewards based on the predictability of reward locations. When compared with a Bayesian search, we found that animal performance is consistent with an approach that adaptively allocates cognitive resources between sensory processing and memory, enhancing sensory acuity and reducing memory load under conditions in which prior information is unreliable. Our findings establish new approaches to understand neural substrates of natural behavior as well as the rational development of biologically inspired approaches for complex real-world optimization.