15,986 research outputs found
Supervised Learning in Multilayer Spiking Neural Networks
The current article introduces a supervised learning algorithm for multilayer
spiking neural networks. The algorithm presented here overcomes some
limitations of existing learning algorithms as it can be applied to neurons
firing multiple spikes and it can in principle be applied to any linearisable
neuron model. The algorithm is applied successfully to various benchmarks, such
as the XOR problem and the Iris data set, as well as complex classifications
problems. The simulations also show the flexibility of this supervised learning
algorithm which permits different encodings of the spike timing patterns,
including precise spike trains encoding.Comment: 38 pages, 4 figure
Adiabatic Quantum Optimization for Associative Memory Recall
Hopfield networks are a variant of associative memory that recall information
stored in the couplings of an Ising model. Stored memories are fixed points for
the network dynamics that correspond to energetic minima of the spin state. We
formulate the recall of memories stored in a Hopfield network using energy
minimization by adiabatic quantum optimization (AQO). Numerical simulations of
the quantum dynamics allow us to quantify the AQO recall accuracy with respect
to the number of stored memories and the noise in the input key. We also
investigate AQO performance with respect to how memories are stored in the
Ising model using different learning rules. Our results indicate that AQO
performance varies strongly with learning rule due to the changes in the energy
landscape. Consequently, learning rules offer indirect methods for
investigating change to the computational complexity of the recall task and the
computational efficiency of AQO.Comment: 22 pages, 11 figures. Updated for clarity and figures, to appear in
Frontiers of Physic
Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
Recent studies have shown that synaptic unreliability is a robust and
sufficient mechanism for inducing the stochasticity observed in cortex. Here,
we introduce Synaptic Sampling Machines, a class of neural network models that
uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised
learning. Similar to the original formulation of Boltzmann machines, these
models can be viewed as a stochastic counterpart of Hopfield networks, but
where stochasticity is induced by a random mask over the connections. Synaptic
stochasticity plays the dual role of an efficient mechanism for sampling, and a
regularizer during learning akin to DropConnect. A local synaptic plasticity
rule implementing an event-driven form of contrastive divergence enables the
learning of generative models in an on-line fashion. Synaptic sampling machines
perform equally well using discrete-timed artificial units (as in Hopfield
networks) or continuous-timed leaky integrate & fire neurons. The learned
representations are remarkably sparse and robust to reductions in bit precision
and synapse pruning: removal of more than 75% of the weakest connections
followed by cursory re-learning causes a negligible performance loss on
benchmark classification tasks. The spiking neuron-based synaptic sampling
machines outperform existing spike-based unsupervised learners, while
potentially offering substantial advantages in terms of power and complexity,
and are thus promising models for on-line learning in brain-inspired hardware
Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites
This paper presents a spike-based model which employs neurons with
functionally distinct dendritic compartments for classifying high dimensional
binary patterns. The synaptic inputs arriving on each dendritic subunit are
nonlinearly processed before being linearly integrated at the soma, giving the
neuron a capacity to perform a large number of input-output mappings. The model
utilizes sparse synaptic connectivity; where each synapse takes a binary value.
The optimal connection pattern of a neuron is learned by using a simple
hardware-friendly, margin enhancing learning algorithm inspired by the
mechanism of structural plasticity in biological neurons. The learning
algorithm groups correlated synaptic inputs on the same dendritic branch. Since
the learning results in modified connection patterns, it can be incorporated
into current event-based neuromorphic systems with little overhead. This work
also presents a branch-specific spike-based version of this structural
plasticity rule. The proposed model is evaluated on benchmark binary
classification problems and its performance is compared against that achieved
using Support Vector Machine (SVM) and Extreme Learning Machine (ELM)
techniques. Our proposed method attains comparable performance while utilizing
10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio
Stochastic firing rate models
We review a recent approach to the mean-field limits in neural networks that
takes into account the stochastic nature of input current and the uncertainty
in synaptic coupling. This approach was proved to be a rigorous limit of the
network equations in a general setting, and we express here the results in a
more customary and simpler framework. We propose a heuristic argument to derive
these equations providing a more intuitive understanding of their origin. These
equations are characterized by a strong coupling between the different moments
of the solutions. We analyse the equations, present an algorithm to simulate
the solutions of these mean-field equations, and investigate numerically the
equations. In particular, we build a bridge between these equations and
Sompolinsky and collaborators approach (1988, 1990), and show how the coupling
between the mean and the covariance function deviates from customary
approaches
- …