9,882 research outputs found
Training Multi-layer Spiking Neural Networks using NormAD based Spatio-Temporal Error Backpropagation
Spiking neural networks (SNNs) have garnered a great amount of interest for
supervised and unsupervised learning applications. This paper deals with the
problem of training multi-layer feedforward SNNs. The non-linear
integrate-and-fire dynamics employed by spiking neurons make it difficult to
train SNNs to generate desired spike trains in response to a given input. To
tackle this, first the problem of training a multi-layer SNN is formulated as
an optimization problem such that its objective function is based on the
deviation in membrane potential rather than the spike arrival instants. Then,
an optimization method named Normalized Approximate Descent (NormAD),
hand-crafted for such non-convex optimization problems, is employed to derive
the iterative synaptic weight update rule. Next, it is reformulated to
efficiently train multi-layer SNNs, and is shown to be effectively performing
spatio-temporal error backpropagation. The learning rule is validated by
training -layer SNNs to solve a spike based formulation of the XOR problem
as well as training -layer SNNs for generic spike based training problems.
Thus, the new algorithm is a key step towards building deep spiking neural
networks capable of efficient event-triggered learning.Comment: 19 pages, 10 figure
Signal Propagation in Feedforward Neuronal Networks with Unreliable Synapses
In this paper, we systematically investigate both the synfire propagation and
firing rate propagation in feedforward neuronal network coupled in an
all-to-all fashion. In contrast to most earlier work, where only reliable
synaptic connections are considered, we mainly examine the effects of
unreliable synapses on both types of neural activity propagation in this work.
We first study networks composed of purely excitatory neurons. Our results show
that both the successful transmission probability and excitatory synaptic
strength largely influence the propagation of these two types of neural
activities, and better tuning of these synaptic parameters makes the considered
network support stable signal propagation. It is also found that noise has
significant but different impacts on these two types of propagation. The
additive Gaussian white noise has the tendency to reduce the precision of the
synfire activity, whereas noise with appropriate intensity can enhance the
performance of firing rate propagation. Further simulations indicate that the
propagation dynamics of the considered neuronal network is not simply
determined by the average amount of received neurotransmitter for each neuron
in a time instant, but also largely influenced by the stochastic effect of
neurotransmitter release. Second, we compare our results with those obtained in
corresponding feedforward neuronal networks connected with reliable synapses
but in a random coupling fashion. We confirm that some differences can be
observed in these two different feedforward neuronal network models. Finally,
we study the signal propagation in feedforward neuronal networks consisting of
both excitatory and inhibitory neurons, and demonstrate that inhibition also
plays an important role in signal propagation in the considered networks.Comment: 33pages, 16 figures; Journal of Computational Neuroscience
(published
- …