The Cognitive Architecture of Spatial Navigation: Hippocampal and Striatal Contributions

Spatial navigation can serve as a model system in cognitive neuroscience, in which specific neural representations, learning rules, and control strategies can be inferred from the vast experimental literature that exists across many species, including humans. Here, we review this literature, focusing on the contributions of hippocampal and striatal systems, and attempt to outline a minimal cognitive architecture that is consistent with the experimental literature and that synthesizes previous related computational modeling. The resulting architecture includes striatal reinforcement learning based on egocentric representations of sensory states and actions, incidental Hebbian association of sensory information with allocentric state representations in the hippocampus, and arbitration of the outputs of both systems based on confidence/uncertainty in medial prefrontal cortex. We discuss the relationship between this architecture and learning in model-free and model-based systems, episodic memory, imagery, and planning, including some open questions and directions for further experiments

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available “cached” value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated “Value of Information” exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus – ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation

Changing pattern in the basal ganglia: motor switching under reduced dopaminergic drive

Action selection in the basal ganglia is often described within the framework of a standard model, associating low dopaminergic drive with motor suppression. Whilst powerful, this model does not explain several clinical and experimental data, including varying therapeutic efficacy across movement disorders. We tested the predictions of this model in patients with Parkinson’s disease, on and off subthalamic deep brain stimulation (DBS), focussing on adaptive sensory-motor responses to a changing environment and maintenance of an action until it is no longer suitable. Surprisingly, we observed prolonged perseverance under on-stimulation, and high inter-individual variability in terms of the motor selections performed when comparing the two conditions. To account for these data, we revised the standard model exploring its space of parameters and associated motor functions and found that, depending on effective connectivity between external and internal parts of the globus pallidus and saliency of the sensory input, a low dopaminergic drive can result in increased, dysfunctional, motor switching, besides motor suppression. This new framework provides insight into the biophysical mechanisms underlying DBS, allowing a description in terms of alteration of the signal-to-baseline ratio in the indirect pathway, which better account of known electrophysiological data in comparison with the standard model

Grasp It Loudly! Supporting Actions with Semantically Congruent Spoken Action Words

Evidence for cross-talk between motor and language brain structures has accumulated over the past several years. However, while a significant amount of research has focused on the interaction between language perception and action, little attention has been paid to the potential impact of language production on overt motor behaviour

Spatial navigation deficits — overlooked cognitive marker for preclinical Alzheimer disease?

Detection of incipient Alzheimer disease (AD) pathophysiology is critical to identify preclinical individuals and target potentially disease-modifying therapies towards them. Current neuroimaging and biomarker research is strongly focused in this direction, with the aim of establishing AD fingerprints to identify individuals at high risk of developing this disease. By contrast, cognitive fingerprints for incipient AD are virtually non-existent as diagnostics and outcomes measures are still focused on episodic memory deficits as the gold standard for AD, despite their low sensitivity and specificity for identifying at-risk individuals. This Review highlights a novel feature of cognitive evaluation for incipient AD by focusing on spatial navigation and orientation deficits, which are increasingly shown to be present in at-risk individuals. Importantly, the navigation system in the brain overlaps substantially with the regions affected by AD in both animal models and humans. Notably, spatial navigation has fewer verbal, cultural and educational biases than current cognitive tests and could enable a more uniform, global approach towards cognitive fingerprints of AD and better cognitive treatment outcome measures in future multicentre trials. The current Review appraises the available evidence for spatial navigation and/or orientation deficits in preclinical, prodromal and confirmed AD and identifies research gaps and future research priorities

Sentence processing: Linking language to motor chain. (Special topic, Action and language integration in cognitive systems.)

A growing body of evidence in cognitive science and neuroscience points towards the existence of a deep interconnection between cognition, perception and action. According to this embodied perspective language is grounded in the sensorimotor system and language understanding is based on a mental simulation process (Jeannerod, 2007; Gallese, 2008; Barsalou, 2009). This means that during action words and sentence comprehension the same perception, action, and emotion mechanisms implied during interaction with objects are recruited. Among the neural underpinnings of this simulation process an important role is played by a sensorimotor matching system known as the mirror neuron system (Rizzolatti and Craighero, 2004). Despite a growing number of studies, the precise dynamics underlying the relation between language and action are not yet well understood. In fact, experimental studies are not always coherent as some report that language processing interferes with action execution while others find facilitation. In this work we present a detailed neural network model capable of reproducing experimentally observed influences of the processing of action-related sentences on the execution of motor sequences. The proposed model is based on three main points. The first is that the processing of action-related sentences causes the resonance of motor and mirror neurons encoding the corresponding actions. The second is that there exists a varying degree of crosstalk between neuronal populations depending on whether they encode the same motor act, the same effector or the same action-goal. The third is the fact that neuronal populations' internal dynamics, which results from the combination of multiple processes taking place at different time scales, can facilitate or interfere with successive activations of the same or of partially overlapping pools

A general model of hippocampal and dorsal striatal learning and decision making

Humans and other animals use multiple strategies for making decisions. Reinforcement-learning theory distinguishes between stimulus–response (model-free; MF) learning and deliberative (model-based; MB) planning. The spatial-navigation literature presents a parallel dichotomy between navigation strategies. In “response learning,” associated with the dorsolateral striatum (DLS), decisions are anchored to an egocentric reference frame. In “place learning,” associated with the hippocampus, decisions are anchored to an allocentric reference frame. Emerging evidence suggests that the contribution of hippocampus to place learning may also underlie its contribution to MB learning by representing relational structure in a cognitive map. Here, we introduce a computational model in which hippocampus subserves place and MB learning by learning a “successor representation” of relational structure between states; DLS implements model-free response learning by learning associations between actions and egocentric representations of landmarks; and action values from either system are weighted by the reliability of its predictions. We show that this model reproduces a range of seemingly disparate behavioral findings in spatial and nonspatial decision tasks and explains the effects of lesions to DLS and hippocampus on these tasks. Furthermore, modeling place cells as driven by boundaries explains the observation that, unlike navigation guided by landmarks, navigation guided by boundaries is robust to “blocking” by prior state–reward associations due to learned associations between place cells. Our model, originally shaped by detailed constraints in the spatial literature, successfully characterizes the hippocampal–striatal system as a general system for decision making via adaptive combination of stimulus–response learning and the use of a cognitive map