Neural systems supporting navigation

Highlights: • Recent neuroimaging and electrophysiology studies have begun to shed light on the neural dynamics of navigation systems. • Computational models have advanced theories of how entorhinal grid cells and hippocampal place cells might serve navigation. • Hippocampus and entorhinal cortex provide complementary representations of routes and vectors for navigation. Much is known about how neural systems determine current spatial position and orientation in the environment. By contrast little is understood about how the brain represents future goal locations or computes the distance and direction to such goals. Recent electrophysiology, computational modelling and neuroimaging research have shed new light on how the spatial relationship to a goal may be determined and represented during navigation. This research suggests that the hippocampus may code the path to the goal while the entorhinal cortex represents the vector to the goal. It also reveals that the engagement of the hippocampus and entorhinal cortex varies across the different operational stages of navigation, such as during travel, route planning, and decision-making at waypoints

Establishing the boundaries: the hippocampal contribution to imagining scenes

When we visualize scenes, either from our own past or invented, we impose a viewpoint for our “mind's eye” and we experience the resulting image as spatially coherent from that viewpoint. The hippocampus has been implicated in this process, but its precise contribution is unknown. We tested a specific hypothesis based on the spatial firing properties of neurons in the hippocampal formation of rats, that this region supports the construction of spatially coherent mental images by representing the locations of the environmental boundaries surrounding our viewpoint. Using functional magnetic resonance imaging, we show that hippocampal activation increases parametrically with the number of enclosing boundaries in the imagined scene. In contrast, hippocampal activity is not modulated by a nonspatial manipulation of scene complexity nor to increasing difficulty of imagining the scenes in general. Our findings identify a specific computational role for the hippocampus in mental imagery and episodic recollection

The well-worn route revisited: Striatal and hippocampal system contributions to route learning in human navigation

Parallel spatial memory systems theory posits that there are two types of memory system. One is a flexible, cognitive mapping system subserved by the hippocampal formation, and the other is a system centred on the striatum based on reinforcement learning principles where specific stimuli are associated with rewarded actions (O’Keefe & Nadel, 1978; White & McDonald, 2002). More recently, Khamassi & Humphries (2012) have argued that the division between model-based and model-free spatial learning is a better predictor of whether hippocampal or striatal systems will be recruited, with hippocampal systems associated with model-based responding and striatal systems with model-free responding. Model-free decision-making occurs when responding is based on average reward history associated with a particular cue-action pairing, whereas model-based decision-making allows knowledge of outcomes from previous learning history to be represented. We sought to test these theories by asking participants (N = 24) to navigate within a virtual environment through a previously learned, 9-junction route with distinctive landmarks at each junction, while undergoing functional magnetic resonance imaging. In critical conflict probe trials, a landmark was presented out of sequence such that following the usual sequence of actions would generate an opposite response to following the learned individual landmark-action association, now out of sequence. Participants that made sequence-based responses had higher parahippocampal activations relative to participants that made responses based on the individual landmark-action association, a result that would be predicted by the need to recruit model-based systems to make a sequence-based response. Parallel spatial memory systems theory would not predict hippocampal formation recruitment for either response in the conflict probe, because no cognitive mapping is required when following a prescribed route. In longer probe trials where participants were able to plan a sequence of responses, striatal systems were recruited (caudate and putamen) suggesting a role for striatum in action chunking

Decoding information in the human hippocampus: a user's guide

Multi-voxel pattern analysis (MVPA), or 'decoding', of fMRI activity has gained popularity in the neuroimaging community in recent years. MVPA differs from standard fMRI analyses by focusing on whether information relating to specific stimuli is encoded in patterns of activity across multiple voxels. If a stimulus can be predicted, or decoded, solely from the pattern of fMRI activity, it must mean there is information about that stimulus represented in the brain region where the pattern across voxels was identified. This ability to examine the representation of information relating to specific stimuli (e.g., memories) in particular brain areas makes MVPA an especially suitable method for investigating memory representations in brain structures such as the hippocampus. This approach could open up new opportunities to examine hippocampal representations in terms of their content, and how they might change over time, with aging, and pathology. Here we consider published MVPA studies that specifically focused on the hippocampus, and use them to illustrate the kinds of novel questions that can be addressed using MVPA. We then discuss some of the conceptual and methodological challenges that can arise when implementing MVPA in this context. Overall, we hope to highlight the potential utility of MVPA, when appropriately deployed, and provide some initial guidance to those considering MVPA as a means to investigate the hippocampus

Neural representation in active inference: using generative models to interact with -- and understand -- the lived world

This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from "explicit" models that entail variables for predicting concurrent sensations, like objects, faces, or people - to "action-oriented models" that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities - and the gradual transition from "pragmatic" to "detached" neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation

Model-based fMRI analysis of memory

Recent advances in Model-based fMRI approaches enable researchers to investigate hypotheses about the time course and latent structure in data that were previously inaccessible. Cognitive models, especially when validated on multiple datasets, allow for additional constraints to be marshalled when interpreting neuroimaging data. Models can be related to BOLD response in a variety of ways, such as constraining the cognitive model by neural data, interpreting the neural data in light of behavioural fit, or simultaneously accounting for both neural and behavioural data. Using cognitive models as a lens on fMRI data is complementary to popular multivariate decoding and representational similarity analysis approaches. Indeed, these approaches can realise greater theoretical significance when situated within a model-based approach

Neural and behavioral correlates of flexible navigation in complex space

This thesis focused on a primary aim: investigate behavioral and neural correlates of flexible spatial navigation using a variety of methods. To do so, we combined immersive virtual reality, real-world navigation, and neuroimaging to better understand the nuances in flexible behavior. Across all five studies discussed in this thesis, we utilized Sea Hero Quest as a baseline measure of spatial ability and prospective predictor for both behavioral and neural measures. Importantly, the work presented in this thesis aimed for novelty in methods. First, this thesis presents the first fMRI results from a dynamic navigation task with a continuously moving goal position – to our knowledge. Second, we found no evidence for a relationship between performance in Sea Hero Quest and either real-world wayfinding / spatial memory measures or related neural measures (hippocampal volume ratio) – in contrast to some recent findings. Last, a new task design looking at spatial performance in an urban version of Sea Hero Quest highlighted the importance of realism in task design. Overall, the work presented in this thesis adds to an understanding of flexible navigation and, importantly, highlights areas in which the field might advance

Hippocampal predictive maps of an uncertain world

Humans and other animals can solve a wide variety of decision-making problems with remarkable flexibility. This flexibility is thought to derive from an internal model of the world, or ‘cognitive map’, used to predict the future and plan actions accordingly. A recent theoretical proposal suggests that the hippocampus houses a representation of long-run state expectancies. These “successor representations” (SRs) occupy a middle ground between model-free and model-based reinforcement learning strategies. However, it is not clear whether SRs can explain hippocampal contributions to spatial and model-based behaviour, nor how a putative hippocampal SR might interface with striatal learning mechanisms. More generally, it is not clear how the predictive map should encode uncertainty, and how an uncertainty-augmented predictive map modifies our experimental predictions for animal behaviour. In the first part of this thesis, I investigated whether viewing the hippocampus as an SR can explain experiments contrasting hippocampal and dorsolateral striatal contributions to behaviour in spatial and non-spatial tasks. To do this, I modelled the hippocampus as an SR and DLS as model-free reinforcement learning, combining their outputs via their relative reliability as a proxy for uncertainty. Current SR models do not formally address uncertainty. Therefore I extended the learning of SRs by temporal differences to include managing uncertainty in new observations versus existing knowledge. I generalise this approach to a multi-task setting using a Bayesian nonparametric switching Kalman Filter, allowing the model to learn and maintain multiple task-specific SR maps and infer which one to use at any moment based on the observations. I show that this Bayesian SR model captures animal behaviour in tasks which require contextual memory and generalisation. In conclusion, I consider how the hippocampal contribution to behaviour can be considered as a predictive map when adapted to take account of uncertainty and combined with other behavioural controllers

Neural basis of route-planning and goal-coding during flexible navigation

Animals and humans are remarkable in their ability to flexibly adapt to changes in their surroundings. Navigational flexibility may take many forms and in this thesis we investigate its neural and behavioral underpinnings using a variety of methods and tasks tailored to each specific research aim. These methods include functional resonance magnetic imaging (fMRI), freely moving virtual reality, desktop virtual reality, large-scale online testing, and computational modelling. First, we reanalysed previously collected rodent data in the lab to better under- stand behavioural bias that may occur during goal-directed navigation tasks. Based on finding some biases we designed a new approach of simulating results on maze configurations prior to data collection to select the ideal mazes for our task. In a parallel line of methods development, we designed a freely moving navigation task using large-scale wireless virtual reality in a 10x10 space. We compared human behaviour to that of a select number of reinforcement learning agents to investigate the feasibility of computational modelling approaches to freely moving behaviour. Second, we further developed our new approach of simulating results on maze configuration to design a novel spatial navigation task used in a parallel experiment in both rats and humans. We report the human findings using desktop virtual reality and fMRI. We identified a network of regions including hippocampal, caudate nu- cleus, and lateral orbitofrontal cortex involvement in learning hidden goal locations. We also identified a positive correlation between Euclidean goal distance and brain activity in the caudate nucleus during ongoing navigation. Third, we developed a large online testing paradigm to investigate the role of home environment on wayfinding ability. We extended previous reports that street network complexity is beneficial in improving wayfinding ability as measured using a previously reported virtual navigation game, Sea Hero Quest, as well as in a novel virtual navigation game, City Hero Quest. We also report results of a navigational strategies questionnaire that highlights differences of growing up inside and outside cities in the United States and how this relates to wayfinding ability. Fourth, we investigate route planning in a group of expert navigators, licensed London taxi drivers. We designed a novel mental route planning task, probing 120 different routes throughout the extensive street network of London. We find hip- pocampal and retrosplenial involvement in route planning. We also identify the frontopolar cortex as one of several brain regions parametrically modulated by plan- ning demand. Lastly, I summarize the findings from these studies and how they all come to provide different insights into our remarkable ability to flexibly adapt to naviga- tional challenges in our environment