Learning navigational maps through potentiation and modulation of hippocampal place cells

We analyze a model of navigational map formation based on correlation-based, temporally asymmetric potentiation and depression of synapses between hippocampal place cells. We show that synaptic modification during random exploration of an environment shifts the location encoded by place cell activity in such a way that it indicates the direction from any location to a fixed target avoiding walls and other obstacles. Multiple maps to different targets can be simultaneously stored if we introduce target-dependent modulation of place cell activity. Once maps to a number of target locations in a given environment have been stored, novel maps to previously unknown target locations are automatically constructed by interpolation between existing maps

Coding and Learning of behavioral sequences

A major challenge to understanding behavior is how the nervous system allows the learning of behavioral sequences that can occur over arbitrary timescales, ranging from milliseconds up to seconds, using a fixed millisecond learning rule. This article describes some potential solutions, and then focuses on a study by Mehta et al. that could contribute towards solving this puzzle. They have discovered that an experience-dependent asymmetric shape of hippocampal receptive fields combined with oscillatory inhibition can serve to map behavioral sequences on a fixed timescale

From synaptic plasticity to spatial maps and sequence learning

The entorhinal-hippocampal circuit is crucial for several forms of learning and memory, especially sequence learning, including spatial navigation. The challenge is to understand the underlying mechanisms. Pioneering discoveries of spatial selectivity in this circuit, i.e. place cells and grid cells, provided a major step forward in tackling this challenge. Considerable research has also shown that sequence learning relies on synaptic plasticity, especially the Hebbian or the NMDAR-dependent synaptic plasticity. This raises several questions: Are spatial maps plastic? If so, what is the contribution of Hebbian plasticity to spatial map plasticity? How does the spatial map plasticity contribute to sequence learning? A combination of computational and experimental studies has shown that NMDAR-mediated plasticity and theta rhythm can have specific effects on the formation and experiential modification of spatial maps to facilitate predictive coding. Advances in transgenic techniques have provided further support for these mechanisms. Although many exciting challenges remain, these findings have brought us closer to solving the puzzle of how the hippocampal system contributes to spatial memory, and point to a way forward

Does computational neuroscience need new synaptic learning paradigms?

Computational neuroscience is dominated by a few paradigmatic models, but it remains an open question whether the existing modelling frameworks are sufficient to explain observed behavioural phenomena in terms of neural implementation. We take learning and synaptic plasticity as an example and point to open questions, such as one-shot learning and acquiring internal representations of the world for flexible planning

Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning

Reinforcement learning (RL) provides an influential characterization of the brain's mechanisms for learning to make advantageous choices. An important problem, though, is how complex tasks can be represented in a way that enables efficient learning. We consider this problem through the lens of spatial navigation, examining how two of the brain's location representations—hippocampal place cells and entorhinal grid cells—are adapted to serve as basis functions for approximating value over space for RL. Although much previous work has focused on these systems' roles in combining upstream sensory cues to track location, revisiting these representations with a focus on how they support this downstream decision function offers complementary insights into their characteristics. Rather than localization, the key problem in learning is generalization between past and present situations, which may not match perfectly. Accordingly, although neural populations collectively offer a precise representation of position, our simulations of navigational tasks verify the suggestion that RL gains efficiency from the more diffuse tuning of individual neurons, which allows learning about rewards to generalize over longer distances given fewer training experiences. However, work on generalization in RL suggests the underlying representation should respect the environment's layout. In particular, although it is often assumed that neurons track location in Euclidean coordinates (that a place cell's activity declines “as the crow flies” away from its peak), the relevant metric for value is geodesic: the distance along a path, around any obstacles. We formalize this intuition and present simulations showing how Euclidean, but not geodesic, representations can interfere with RL by generalizing inappropriately across barriers. Our proposal that place and grid responses should be modulated by geodesic distances suggests novel predictions about how obstacles should affect spatial firing fields, which provides a new viewpoint on data concerning both spatial codes

Mathematical Formulations of Hebbian Learning

Several formulations of correlation-based Hebbian learning are reviewed. On the presynaptic side, activity is described either by a firing rate or by presynaptic spike arrival. The state of the postsynaptic neuron can be described by its membrane potential, its firing rate, or the timing of backpropagating action potentials (BPAPs). It is shown that all of the above formulations can be derived from the point of view of an expansion. In the absence of BPAPs potentials, it is natural to correlate presynaptic spikes with the postsynaptic membrane potential. Time windows of spike time dependent plasticity arise naturally, if the timing of postsynaptic spikes is available at the site of the synapse as it is the case in the presence of BPAPs. With an appropriate choice of parameters, Hebbian synaptic plasticity has intrinsic normalization properties that stabilizes postsynaptic firing rates and leads to subtractive weight normalization

Dual coding with STDP in a spiking recurrent neural network model of the hippocampus.

The firing rate of single neurons in the mammalian hippocampus has been demonstrated to encode for a range of spatial and non-spatial stimuli. It has also been demonstrated that phase of firing, with respect to the theta oscillation that dominates the hippocampal EEG during stereotype learning behaviour, correlates with an animal's spatial location. These findings have led to the hypothesis that the hippocampus operates using a dual (rate and temporal) coding system. To investigate the phenomenon of dual coding in the hippocampus, we examine a spiking recurrent network model with theta coded neural dynamics and an STDP rule that mediates rate-coded Hebbian learning when pre- and post-synaptic firing is stochastic. We demonstrate that this plasticity rule can generate both symmetric and asymmetric connections between neurons that fire at concurrent or successive theta phase, respectively, and subsequently produce both pattern completion and sequence prediction from partial cues. This unifies previously disparate auto- and hetero-associative network models of hippocampal function and provides them with a firmer basis in modern neurobiology. Furthermore, the encoding and reactivation of activity in mutually exciting Hebbian cell assemblies demonstrated here is believed to represent a fundamental mechanism of cognitive processing in the brain

Unsupervised Learning of Reflexive and Action-Based Affordances to Model Adaptive Navigational Behavior

Here we introduce a cognitive model capable to model a variety of behavioral domains and apply it to a navigational task. We used place cells as sensory representation, such that the cells’ place fields divided the environment into discrete states. The robot learns knowledge of the environment by memorizing the sensory outcome of its motor actions. This is composed of a central process, learning the probability of state-to-state transitions by motor actions and a distal processing routine, learning the extent to which these state-to-state transitions are caused by sensory-driven reflex behavior (obstacle avoidance). Navigational decision making integrates central and distal learned environmental knowledge to select an action that leads to a goal state. Differentiating distal and central processing increases the behavioral accuracy of the selected actions and the ability of behavioral adaptation to a changed environment. We propose that the system can canonically be expanded to model other behaviors, using alternative definitions of states and actions. The emphasis of this paper is to test this general cognitive model on a robot in a real-world environment