Neuromorphic Learning towards Nano Second Precision

Temporal coding is one approach to representing information in spiking neural networks. An example of its application is the location of sounds by barn owls that requires especially precise temporal coding. Dependent upon the azimuthal angle, the arrival times of sound signals are shifted between both ears. In order to deter- mine these interaural time differences, the phase difference of the signals is measured. We implemented this biologically inspired network on a neuromorphic hardware system and demonstrate spike-timing dependent plasticity on an analog, highly accelerated hardware substrate. Our neuromorphic implementation enables the resolution of time differences of less than 50 ns. On-chip Hebbian learning mechanisms select inputs from a pool of neurons which code for the same sound frequency. Hence, noise caused by different synaptic delays across these inputs is reduced. Furthermore, learning compensates for variations on neuronal and synaptic parameters caused by device mismatch intrinsic to the neuromorphic substrate.Comment: 7 pages, 7 figures, presented at IJCNN 2013 in Dallas, TX, USA. IJCNN 2013. Corrected version with updated STDP curves IJCNN 201

The spectro-contextual encoding and retrieval theory of episodic memory.

The spectral fingerprint hypothesis, which posits that different frequencies of oscillations underlie different cognitive operations, provides one account for how interactions between brain regions support perceptual and attentive processes (Siegel etal., 2012). Here, we explore and extend this idea to the domain of human episodic memory encoding and retrieval. Incorporating findings from the synaptic to cognitive levels of organization, we argue that spectrally precise cross-frequency coupling and phase-synchronization promote the formation of hippocampal-neocortical cell assemblies that form the basis for episodic memory. We suggest that both cell assembly firing patterns as well as the global pattern of brain oscillatory activity within hippocampal-neocortical networks represents the contents of a particular memory. Drawing upon the ideas of context reinstatement and multiple trace theory, we argue that memory retrieval is driven by internal and/or external factors which recreate these frequency-specific oscillatory patterns which occur during episodic encoding. These ideas are synthesized into a novel model of episodic memory (the spectro-contextual encoding and retrieval theory, or "SCERT") that provides several testable predictions for future research

Neural models of learning and visual grouping in the presence of finite conduction velocities

The hypothesis of object binding-by-synchronization in the visual cortex has been supported by recent experiments in awake monkeys. They demonstrated coherence among gamma-activities (30–90 Hz) of local neural groups and its perceptual modulation according to the rules of figure-ground segregation. Interactions within and between these neural groups are based on axonal spike conduction with finite velocities. Physiological studies confirmed that the majority of transmission delays is comparable to the temporal scale defined by gamma-activity (11–33 ms). How do these finite velocities influence the development of synaptic connections within and between visual areas? What is the relationship between the range of gamma-coherence and the velocity of signal transmission? Are these large temporal delays compatible with recently discovered phenomenon of gamma-waves traveling across larger parts of the primary visual cortex? The refinement of connections in the immature visual cortex depends on temporal Hebbian learning to adjust synaptic efficacies between spiking neurons. The impact of constant, finite, axonal spike conduction velocities on this process was investigated using a set of topographic network models. Random spike trains with a confined temporal correlation width mimicked cortical activity before visual experience. After learning, the lateral connectivity within one network layer became spatially restricted, the width of the connection profile being directly proportional to the lateral conduction velocity. Furthermore, restricted feedforward divergence developed between neurons of two successive layers. The size of this connection profile matched the lateral connection profile of the lower layer neuron. The mechanism in this network model is suitable to explain the emergence of larger receptive fields at higher visual areas while preserving a retinotopic mapping. The influence of finite conduction velocities on the local generation of gamma-activities and their spatial synchronization was investigated in a model of a mature visual area. Sustained input and local inhibitory feedback was sufficient for the emergence of coherent gamma-activity that extended across few millimeters. Conduction velocities had a direct impact on the frequency of gamma-oscillations, but did neither affect gamma-power nor the spatial extent of gamma-coherence. Adding long-range horizontal connections between excitatory neurons, as found in layer 2/3 of the primary visual cortex, increased the spatial range of gamma-coherence. The range was maximal for zero transmission delays, and for all distances attenuated with finite, decreasing lateral conduction velocities. Below a velocity of 0.5 m/s, gamma-power and gamma-coherence were even smaller than without these connections at all, i.e., slow horizontal connections actively desynchronized neural populations. In conclusion, the enhancement of gamma-coherence by horizontal excitatory connections critically depends on fast conduction velocities. Coherent gamma-activity in the primary visual cortex and the accompanying models was found to only cover small regions of the visual field. This challenges the role of gamma-synchronization to solve the binding problem for larger object representations. Further analysis of the previous model revealed that the patches of coherent gamma-activity (1.8 mm half-height decline) were part of more globally occurring gamma-waves, which coupled over much larger distances (6.3 mm half-height decline). The model gamma-waves observed here are very similar to those found in the primary visual cortex of awake monkeys, indicating that local recurrent inhibition and restricted horizontal connections with finite axonal velocities are sufficient requirements for their emergence. In conclusion, since the model is in accordance with the connectivity and gamma-processes in the primary visual cortex, the results support the hypothesis that gamma-waves provide a generalized concept for object binding in the visual cortex

Synaptic potentiation facilitates memory-like attractor dynamics in cultured in vitro hippocampal networks

Collective rhythmic dynamics from neurons is vital for cognitive functions such as memory formation but how neurons self-organize to produce such activity is not well understood. Attractor-based models have been successfully implemented as a theoretical framework for memory storage in networks of neurons. Activity-dependent modification of synaptic transmission is thought to be the physiological basis of learning and memory. The goal of this study is to demonstrate that using a pharmacological perturbation on in vitro networks of hippocampal neurons that has been shown to increase synaptic strength follows the dynamical postulates theorized by attractor models. We use a grid of extracellular electrodes to study changes in network activity after this perturbation and show that there is a persistent increase in overall spiking and bursting activity after treatment. This increase in activity appears to recruit more "errant" spikes into bursts. Lastly, phase plots indicate a conserved activity pattern suggesting that the network is operating in a stable dynamical state

A Pulse-Gated, Predictive Neural Circuit

Recent evidence suggests that neural information is encoded in packets and may be flexibly routed from region to region. We have hypothesized that neural circuits are split into sub-circuits where one sub-circuit controls information propagation via pulse gating and a second sub-circuit processes graded information under the control of the first sub-circuit. Using an explicit pulse-gating mechanism, we have been able to show how information may be processed by such pulse-controlled circuits and also how, by allowing the information processing circuit to interact with the gating circuit, decisions can be made. Here, we demonstrate how Hebbian plasticity may be used to supplement our pulse-gated information processing framework by implementing a machine learning algorithm. The resulting neural circuit has a number of structures that are similar to biological neural systems, including a layered structure and information propagation driven by oscillatory gating with a complex frequency spectrum.Comment: This invited paper was presented at the 50th Asilomar Conference on Signals, Systems and Computer

Storage of phase-coded patterns via STDP in fully-connected and sparse network: a study of the network capacity

We study the storage and retrieval of phase-coded patterns as stable dynamical attractors in recurrent neural networks, for both an analog and a integrate-and-fire spiking model. The synaptic strength is determined by a learning rule based on spike-time-dependent plasticity, with an asymmetric time window depending on the relative timing between pre- and post-synaptic activity. We store multiple patterns and study the network capacity. For the analog model, we find that the network capacity scales linearly with the network size, and that both capacity and the oscillation frequency of the retrieval state depend on the asymmetry of the learning time window. In addition to fully-connected networks, we study sparse networks, where each neuron is connected only to a small number z << N of other neurons. Connections can be short range, between neighboring neurons placed on a regular lattice, or long range, between randomly chosen pairs of neurons. We find that a small fraction of long range connections is able to amplify the capacity of the network. This imply that a small-world-network topology is optimal, as a compromise between the cost of long range connections and the capacity increase. Also in the spiking integrate and fire model the crucial result of storing and retrieval of multiple phase-coded patterns is observed. The capacity of the fully-connected spiking network is investigated, together with the relation between oscillation frequency of retrieval state and window asymmetry