6,493 research outputs found
Delay Learning Architectures for Memory and Classification
We present a neuromorphic spiking neural network, the DELTRON, that can
remember and store patterns by changing the delays of every connection as
opposed to modifying the weights. The advantage of this architecture over
traditional weight based ones is simpler hardware implementation without
multipliers or digital-analog converters (DACs) as well as being suited to
time-based computing. The name is derived due to similarity in the learning
rule with an earlier architecture called Tempotron. The DELTRON can remember
more patterns than other delay-based networks by modifying a few delays to
remember the most 'salient' or synchronous part of every spike pattern. We
present simulations of memory capacity and classification ability of the
DELTRON for different random spatio-temporal spike patterns. The memory
capacity for noisy spike patterns and missing spikes are also shown. Finally,
we present SPICE simulation results of the core circuits involved in a
reconfigurable mixed signal implementation of this architecture.Comment: 27 pages, 20 figure
Memory and information processing in neuromorphic systems
A striking difference between brain-inspired neuromorphic processors and
current von Neumann processors architectures is the way in which memory and
processing is organized. As Information and Communication Technologies continue
to address the need for increased computational power through the increase of
cores within a digital processor, neuromorphic engineers and scientists can
complement this need by building processor architectures where memory is
distributed with the processing. In this paper we present a survey of
brain-inspired processor architectures that support models of cortical networks
and deep neural networks. These architectures range from serial clocked
implementations of multi-neuron systems to massively parallel asynchronous ones
and from purely digital systems to mixed analog/digital systems which implement
more biological-like models of neurons and synapses together with a suite of
adaptation and learning mechanisms analogous to the ones found in biological
nervous systems. We describe the advantages of the different approaches being
pursued and present the challenges that need to be addressed for building
artificial neural processing systems that can display the richness of behaviors
seen in biological systems.Comment: Submitted to Proceedings of IEEE, review of recently proposed
neuromorphic computing platforms and system
Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
Recent studies have shown that synaptic unreliability is a robust and
sufficient mechanism for inducing the stochasticity observed in cortex. Here,
we introduce Synaptic Sampling Machines, a class of neural network models that
uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised
learning. Similar to the original formulation of Boltzmann machines, these
models can be viewed as a stochastic counterpart of Hopfield networks, but
where stochasticity is induced by a random mask over the connections. Synaptic
stochasticity plays the dual role of an efficient mechanism for sampling, and a
regularizer during learning akin to DropConnect. A local synaptic plasticity
rule implementing an event-driven form of contrastive divergence enables the
learning of generative models in an on-line fashion. Synaptic sampling machines
perform equally well using discrete-timed artificial units (as in Hopfield
networks) or continuous-timed leaky integrate & fire neurons. The learned
representations are remarkably sparse and robust to reductions in bit precision
and synapse pruning: removal of more than 75% of the weakest connections
followed by cursory re-learning causes a negligible performance loss on
benchmark classification tasks. The spiking neuron-based synaptic sampling
machines outperform existing spike-based unsupervised learners, while
potentially offering substantial advantages in terms of power and complexity,
and are thus promising models for on-line learning in brain-inspired hardware
Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites
This paper presents a spike-based model which employs neurons with
functionally distinct dendritic compartments for classifying high dimensional
binary patterns. The synaptic inputs arriving on each dendritic subunit are
nonlinearly processed before being linearly integrated at the soma, giving the
neuron a capacity to perform a large number of input-output mappings. The model
utilizes sparse synaptic connectivity; where each synapse takes a binary value.
The optimal connection pattern of a neuron is learned by using a simple
hardware-friendly, margin enhancing learning algorithm inspired by the
mechanism of structural plasticity in biological neurons. The learning
algorithm groups correlated synaptic inputs on the same dendritic branch. Since
the learning results in modified connection patterns, it can be incorporated
into current event-based neuromorphic systems with little overhead. This work
also presents a branch-specific spike-based version of this structural
plasticity rule. The proposed model is evaluated on benchmark binary
classification problems and its performance is compared against that achieved
using Support Vector Machine (SVM) and Extreme Learning Machine (ELM)
techniques. Our proposed method attains comparable performance while utilizing
10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio
Memcapacitive Devices in Logic and Crossbar Applications
Over the last decade, memristive devices have been widely adopted in
computing for various conventional and unconventional applications. While the
integration density, memory property, and nonlinear characteristics have many
benefits, reducing the energy consumption is limited by the resistive nature of
the devices. Memcapacitors would address that limitation while still having all
the benefits of memristors. Recent work has shown that with adjusted parameters
during the fabrication process, a metal-oxide device can indeed exhibit a
memcapacitive behavior. We introduce novel memcapacitive logic gates and
memcapacitive crossbar classifiers as a proof of concept that such applications
can outperform memristor-based architectures. The results illustrate that,
compared to memristive logic gates, our memcapacitive gates consume about 7x
less power. The memcapacitive crossbar classifier achieves similar
classification performance but reduces the power consumption by a factor of
about 1,500x for the MNIST dataset and a factor of about 1,000x for the
CIFAR-10 dataset compared to a memristive crossbar. Our simulation results
demonstrate that memcapacitive devices have great potential for both Boolean
logic and analog low-power applications
- …