246 research outputs found
An Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks
In this article, we propose a novel Winner-Take-All (WTA) architecture
employing neurons with nonlinear dendrites and an online unsupervised
structural plasticity rule for training it. Further, to aid hardware
implementations, our network employs only binary synapses. The proposed
learning rule is inspired by spike time dependent plasticity (STDP) but differs
for each dendrite based on its activation level. It trains the WTA network
through formation and elimination of connections between inputs and synapses.
To demonstrate the performance of the proposed network and learning rule, we
employ it to solve two, four and six class classification of random Poisson
spike time inputs. The results indicate that by proper tuning of the inhibitory
time constant of the WTA, a trade-off between specificity and sensitivity of
the network can be achieved. We use the inhibitory time constant to set the
number of subpatterns per pattern we want to detect. We show that while the
percentage of successful trials are 92%, 88% and 82% for two, four and six
class classification when no pattern subdivisions are made, it increases to
100% when each pattern is subdivided into 5 or 10 subpatterns. However, the
former scenario of no pattern subdivision is more jitter resilient than the
later ones.Comment: 11 pages, 10 figures, journa
Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations
In this paper, we describe a new neuro-inspired, hardware-friendly readout
stage for the liquid state machine (LSM), a popular model for reservoir
computing. Compared to the parallel perceptron architecture trained by the
p-delta algorithm, which is the state of the art in terms of performance of
readout stages, our readout architecture and learning algorithm can attain
better performance with significantly less synaptic resources making it
attractive for VLSI implementation. Inspired by the nonlinear properties of
dendrites in biological neurons, our readout stage incorporates neurons having
multiple dendrites with a lumped nonlinearity. The number of synaptic
connections on each branch is significantly lower than the total number of
connections from the liquid neurons and the learning algorithm tries to find
the best 'combination' of input connections on each branch to reduce the error.
Hence, the learning involves network rewiring (NRW) of the readout network
similar to structural plasticity observed in its biological counterparts. We
show that compared to a single perceptron using analog weights, this
architecture for the readout can attain, even by using the same number of
binary valued synapses, up to 3.3 times less error for a two-class spike train
classification problem and 2.4 times less error for an input rate approximation
task. Even with 60 times larger synapses, a group of 60 parallel perceptrons
cannot attain the performance of the proposed dendritically enhanced readout.
An additional advantage of this method for hardware implementations is that the
'choice' of connectivity can be easily implemented exploiting address event
representation (AER) protocols commonly used in current neuromorphic systems
where the connection matrix is stored in memory. Also, due to the use of binary
synapses, our proposed method is more robust against statistical variations.Comment: 14 pages, 19 figures, Journa
Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites
This paper presents a spike-based model which employs neurons with
functionally distinct dendritic compartments for classifying high dimensional
binary patterns. The synaptic inputs arriving on each dendritic subunit are
nonlinearly processed before being linearly integrated at the soma, giving the
neuron a capacity to perform a large number of input-output mappings. The model
utilizes sparse synaptic connectivity; where each synapse takes a binary value.
The optimal connection pattern of a neuron is learned by using a simple
hardware-friendly, margin enhancing learning algorithm inspired by the
mechanism of structural plasticity in biological neurons. The learning
algorithm groups correlated synaptic inputs on the same dendritic branch. Since
the learning results in modified connection patterns, it can be incorporated
into current event-based neuromorphic systems with little overhead. This work
also presents a branch-specific spike-based version of this structural
plasticity rule. The proposed model is evaluated on benchmark binary
classification problems and its performance is compared against that achieved
using Support Vector Machine (SVM) and Extreme Learning Machine (ELM)
techniques. Our proposed method attains comparable performance while utilizing
10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio
Delay Learning Architectures for Memory and Classification
We present a neuromorphic spiking neural network, the DELTRON, that can
remember and store patterns by changing the delays of every connection as
opposed to modifying the weights. The advantage of this architecture over
traditional weight based ones is simpler hardware implementation without
multipliers or digital-analog converters (DACs) as well as being suited to
time-based computing. The name is derived due to similarity in the learning
rule with an earlier architecture called Tempotron. The DELTRON can remember
more patterns than other delay-based networks by modifying a few delays to
remember the most 'salient' or synchronous part of every spike pattern. We
present simulations of memory capacity and classification ability of the
DELTRON for different random spatio-temporal spike patterns. The memory
capacity for noisy spike patterns and missing spikes are also shown. Finally,
we present SPICE simulation results of the core circuits involved in a
reconfigurable mixed signal implementation of this architecture.Comment: 27 pages, 20 figure
- …