Learning and stabilization of winner-take-all dynamics through interacting excitatory and inhibitory plasticity

Winner-Take-All (WTA) networks are recurrently connected populations of excitatory and inhibitory neurons that represent promising candidate microcircuits for implementing cortical computation. WTAs can perform powerful computations, ranging from signal restoration to state-dependent processing. However, such networks require fine tuned connectivity parameters to keep the network dynamics within stable operating regimes. In this article, we show how such stability can emerge autonomously through an interaction of biologically plausible plasticity mechanisms that operate simultaneously on all excitatory and inhibitory synapses of the network. A weight-dependent plasticity rule is derived from the triplet spike-timing dependent plasticity model, and its stabilization properties in the mean field case are analyzed using contraction theory. Our main result provides simple constraints on the plasticity rule parameters, rather than on the weights themselves, which guarantee stable WTA behavior. The plastic network we present is able to adapt to changing input conditions, and to dynamically adjust its gain, therefore exhibiting self-stabilization mechanisms that are crucial for maintaining stable operation in large networks of interconnected subunits. We show how distributed neural assemblies can adjust their parameters for stable WTA function autonomously while respecting anatomical constraints on neural wiring

A superconducting nanowire spiking element for neural networks

As the limits of traditional von Neumann computing come into view, the brain's ability to communicate vast quantities of information using low-power spikes has become an increasing source of inspiration for alternative architectures. Key to the success of these largescale neural networks is a power-efficient spiking element that is scalable and easily interfaced with traditional control electronics. In this work, we present a spiking element fabricated from superconducting nanowires that has pulse energies on the order of ~10 aJ. We demonstrate that the device reproduces essential characteristics of biological neurons, such as a refractory period and a firing threshold. Through simulations using experimentally measured device parameters, we show how nanowire-based networks may be used for inference in image recognition, and that the probabilistic nature of nanowire switching may be exploited for modeling biological processes and for applications that rely on stochasticity.Comment: 5 main figures; 7 supplemental figure

Winner-take-all in a phase oscillator system with adaptation.

We consider a system of generalized phase oscillators with a central element and radial connections. In contrast to conventional phase oscillators of the Kuramoto type, the dynamic variables in our system include not only the phase of each oscillator but also the natural frequency of the central oscillator, and the connection strengths from the peripheral oscillators to the central oscillator. With appropriate parameter values the system demonstrates winner-take-all behavior in terms of the competition between peripheral oscillators for the synchronization with the central oscillator. Conditions for the winner-take-all regime are derived for stationary and non-stationary types of system dynamics. Bifurcation analysis of the transition from stationary to non-stationary winner-take-all dynamics is presented. A new bifurcation type called a Saddle Node on Invariant Torus (SNIT) bifurcation was observed and is described in detail. Computer simulations of the system allow an optimal choice of parameters for winner-take-all implementation

A Hippocampal Model for Behavioral Time Acquisition and Fast Bidirectional Replay of Spatio-Temporal Memory Sequences

The hippocampus is known to play a crucial role in the formation of long-term memory. For this, fast replays of previously experienced activities during sleep or after reward experiences are believed to be crucial. But how such replays are generated is still completely unclear. In this paper we propose a possible mechanism for this: we present a model that can store experienced trajectories on a behavioral timescale after a single run, and can subsequently bidirectionally replay such trajectories, thereby omitting any specifics of the previous behavior like speed, etc, but allowing repetitions of events, even with different subsequent events. Our solution builds on well-known concepts, one-shot learning and synfire chains, enhancing them by additional mechanisms using global inhibition and disinhibition. For replays our approach relies on dendritic spikes and cholinergic modulation, as supported by experimental data. We also hypothesize a functional role of disinhibition as a pacemaker during behavioral time

Steep, Spatially Graded Recruitment of Feedback Inhibition by Sparse Dentate Granule Cell Activity

The dentate gyrus of the hippocampus is thought to subserve important physiological functions, such as 'pattern separation'. In chronic temporal lobe epilepsy, the dentate gyrus constitutes a strong inhibitory gate for the propagation of seizure activity into the hippocampus proper. Both examples are thought to depend critically on a steep recruitment of feedback inhibition by active dentate granule cells. Here, I used two complementary experimental approaches to quantitatively investigate the recruitment of feedback inhibition in the dentate gyrus. I showed that the activity of approximately 4% of granule cells suffices to recruit maximal feedback inhibition within the local circuit. Furthermore, the inhibition elicited by a local population of granule cells is distributed non-uniformly over the extent of the granule cell layer. Locally and remotely activated inhibition differ in several key aspects, namely their amplitude, recruitment, latency and kinetic properties. Finally, I show that net feedback inhibition facilitates during repetitive stimulation. Taken together, these data provide the first quantitative functional description of a canonical feedback inhibitory microcircuit motif. They establish that sparse granule cell activity, within the range observed in-vivo, steeply recruits spatially and temporally graded feedback inhibition