A Pulse-Gated, Predictive Neural Circuit

Recent evidence suggests that neural information is encoded in packets and may be flexibly routed from region to region. We have hypothesized that neural circuits are split into sub-circuits where one sub-circuit controls information propagation via pulse gating and a second sub-circuit processes graded information under the control of the first sub-circuit. Using an explicit pulse-gating mechanism, we have been able to show how information may be processed by such pulse-controlled circuits and also how, by allowing the information processing circuit to interact with the gating circuit, decisions can be made. Here, we demonstrate how Hebbian plasticity may be used to supplement our pulse-gated information processing framework by implementing a machine learning algorithm. The resulting neural circuit has a number of structures that are similar to biological neural systems, including a layered structure and information propagation driven by oscillatory gating with a complex frequency spectrum.Comment: This invited paper was presented at the 50th Asilomar Conference on Signals, Systems and Computer

Graded, Dynamically Routable Information Processing with Synfire-Gated Synfire Chains

Coherent neural spiking and local field potentials are believed to be signatures of the binding and transfer of information in the brain. Coherent activity has now been measured experimentally in many regions of mammalian cortex. Synfire chains are one of the main theoretical constructs that have been appealed to to describe coherent spiking phenomena. However, for some time, it has been known that synchronous activity in feedforward networks asymptotically either approaches an attractor with fixed waveform and amplitude, or fails to propagate. This has limited their ability to explain graded neuronal responses. Recently, we have shown that pulse-gated synfire chains are capable of propagating graded information coded in mean population current or firing rate amplitudes. In particular, we showed that it is possible to use one synfire chain to provide gating pulses and a second, pulse-gated synfire chain to propagate graded information. We called these circuits synfire-gated synfire chains (SGSCs). Here, we present SGSCs in which graded information can rapidly cascade through a neural circuit, and show a correspondence between this type of transfer and a mean-field model in which gating pulses overlap in time. We show that SGSCs are robust in the presence of variability in population size, pulse timing and synaptic strength. Finally, we demonstrate the computational capabilities of SGSC-based information coding by implementing a self-contained, spike-based, modular neural circuit that is triggered by, then reads in streaming input, processes the input, then makes a decision based on the processed information and shuts itself down

Spiking LCA in a Neural Circuit with Dictionary Learning and Synaptic Normalization

The Locally Competitive Algorithm (LCA) [17, 18] was put forward as a model of primary visual cortex [14, 17] and has been used extensively as a sparse coding algorithm for multivariate data. LCA has seen implementations on neuromorphic processors, including IBM’s TrueNorth processor [10], and Intel’s neuromorphic research processor, Loihi, which show that it can be very efficient with respect to the power resources it consumes [8]. When combined with dictionary learning [13], the LCA algorithm encounters synaptic instability [24], where, as a synapse’s strength grows, its activity increases, further enhancing synaptic strength, leading to a runaway condition, where synapses become saturated [3, 15]. A number of approaches have been suggested to stabilize this phenomenon [1, 2, 5, 7, 12]. Previous work demonstrated that, by extending the cost function used to generate LCA updates, synaptic normalization could be achieved, eliminating synaptic runaway [7]. It was also shown that the resulting algorithm could be implemented in a firing rate model [7]. Here, we implement a probabilistic approximation to this firing rate model as a spiking LCA algorithm that includes dictionary learning and synaptic normalization. The algorithm is based on a synfire-gated synfire chain-based information control network in concert with Hebbian synapses [16, 19]. We show that this algorithm results in correct classification on numeric data taken from the MNIST datase

Machine Learning Changes the Rules for Flux Limiters

Learning to integrate non-linear equations from highly resolved direct numerical simulations (DNSs) has seen recent interest for reducing the computational load for fluid simulations. Here, we focus on determining a flux-limiter for shock capturing methods. Focusing on flux limiters provides a specific plug-and-play component for existing numerical methods. Since their introduction, an array of flux limiters has been designed. Using the coarse-grained Burgers' equation, we show that flux-limiters may be rank-ordered in terms of their log-error relative to high-resolution data. We then develop theory to find an optimal flux-limiter and present flux-limiters that outperform others tested for integrating Burgers' equation on lattices with $2\times$, $3\times$, $4\times$, and $8\times$ coarse-grainings. We train a continuous piecewise linear limiter by minimizing the mean-squared misfit to 6-grid point segments of high-resolution data, averaged over all segments. While flux limiters are generally designed to have an output of $\phi(r) = 1$ at a flux ratio of $r = 1$, our limiters are not bound by this rule, and yet produce a smaller error than standard limiters. We find that our machine learned limiters have distinctive features that may provide new rules-of-thumb for the development of improved limiters. Additionally, we use our theory to learn flux-limiters that outperform standard limiters across a range of values (as opposed to at a specific fixed value) of coarse-graining, number of discretized bins, and diffusion parameter. This demonstrates the ability to produce flux limiters that should be more broadly useful than standard limiters for general applications.Comment: fixed erratum: one corrected figure and some minor text update