565 research outputs found
Memory and information processing in neuromorphic systems
A striking difference between brain-inspired neuromorphic processors and
current von Neumann processors architectures is the way in which memory and
processing is organized. As Information and Communication Technologies continue
to address the need for increased computational power through the increase of
cores within a digital processor, neuromorphic engineers and scientists can
complement this need by building processor architectures where memory is
distributed with the processing. In this paper we present a survey of
brain-inspired processor architectures that support models of cortical networks
and deep neural networks. These architectures range from serial clocked
implementations of multi-neuron systems to massively parallel asynchronous ones
and from purely digital systems to mixed analog/digital systems which implement
more biological-like models of neurons and synapses together with a suite of
adaptation and learning mechanisms analogous to the ones found in biological
nervous systems. We describe the advantages of the different approaches being
pursued and present the challenges that need to be addressed for building
artificial neural processing systems that can display the richness of behaviors
seen in biological systems.Comment: Submitted to Proceedings of IEEE, review of recently proposed
neuromorphic computing platforms and system
Real time unsupervised learning of visual stimuli in neuromorphic VLSI systems
Neuromorphic chips embody computational principles operating in the nervous
system, into microelectronic devices. In this domain it is important to
identify computational primitives that theory and experiments suggest as
generic and reusable cognitive elements. One such element is provided by
attractor dynamics in recurrent networks. Point attractors are equilibrium
states of the dynamics (up to fluctuations), determined by the synaptic
structure of the network; a `basin' of attraction comprises all initial states
leading to a given attractor upon relaxation, hence making attractor dynamics
suitable to implement robust associative memory. The initial network state is
dictated by the stimulus, and relaxation to the attractor state implements the
retrieval of the corresponding memorized prototypical pattern. In a previous
work we demonstrated that a neuromorphic recurrent network of spiking neurons
and suitably chosen, fixed synapses supports attractor dynamics. Here we focus
on learning: activating on-chip synaptic plasticity and using a theory-driven
strategy for choosing network parameters, we show that autonomous learning,
following repeated presentation of simple visual stimuli, shapes a synaptic
connectivity supporting stimulus-selective attractors. Associative memory
develops on chip as the result of the coupled stimulus-driven neural activity
and ensuing synaptic dynamics, with no artificial separation between learning
and retrieval phases.Comment: submitted to Scientific Repor
Hardware-Amenable Structural Learning for Spike-based Pattern Classification using a Simple Model of Active Dendrites
This paper presents a spike-based model which employs neurons with
functionally distinct dendritic compartments for classifying high dimensional
binary patterns. The synaptic inputs arriving on each dendritic subunit are
nonlinearly processed before being linearly integrated at the soma, giving the
neuron a capacity to perform a large number of input-output mappings. The model
utilizes sparse synaptic connectivity; where each synapse takes a binary value.
The optimal connection pattern of a neuron is learned by using a simple
hardware-friendly, margin enhancing learning algorithm inspired by the
mechanism of structural plasticity in biological neurons. The learning
algorithm groups correlated synaptic inputs on the same dendritic branch. Since
the learning results in modified connection patterns, it can be incorporated
into current event-based neuromorphic systems with little overhead. This work
also presents a branch-specific spike-based version of this structural
plasticity rule. The proposed model is evaluated on benchmark binary
classification problems and its performance is compared against that achieved
using Support Vector Machine (SVM) and Extreme Learning Machine (ELM)
techniques. Our proposed method attains comparable performance while utilizing
10 to 50% less computational resources than the other reported techniques.Comment: Accepted for publication in Neural Computatio
Adaptive motor control and learning in a spiking neural network realised on a mixed-signal neuromorphic processor
Neuromorphic computing is a new paradigm for design of both the computing
hardware and algorithms inspired by biological neural networks. The event-based
nature and the inherent parallelism make neuromorphic computing a promising
paradigm for building efficient neural network based architectures for control
of fast and agile robots. In this paper, we present a spiking neural network
architecture that uses sensory feedback to control rotational velocity of a
robotic vehicle. When the velocity reaches the target value, the mapping from
the target velocity of the vehicle to the correct motor command, both
represented in the spiking neural network on the neuromorphic device, is
autonomously stored on the device using on-chip plastic synaptic weights. We
validate the controller using a wheel motor of a miniature mobile vehicle and
inertia measurement unit as the sensory feedback and demonstrate online
learning of a simple 'inverse model' in a two-layer spiking neural network on
the neuromorphic chip. The prototype neuromorphic device that features 256
spiking neurons allows us to realise a simple proof of concept architecture for
the purely neuromorphic motor control and learning. The architecture can be
easily scaled-up if a larger neuromorphic device is available.Comment: 6+1 pages, 4 figures, will appear in one of the Robotics conference
Six networks on a universal neuromorphic computing substrate
In this study, we present a highly configurable neuromorphic computing substrate and use it for emulating several types of neural networks. At the heart of this system lies a mixed-signal chip, with analog implementations of neurons and synapses and digital transmission of action potentials. Major advantages of this emulation device, which has been explicitly designed as a universal neural network emulator, are its inherent parallelism and high acceleration factor compared to conventional computers. Its configurability allows the realization of almost arbitrary network topologies and the use of widely varied neuronal and synaptic parameters. Fixed-pattern noise inherent to analog circuitry is reduced by calibration routines. An integrated development environment allows neuroscientists to operate the device without any prior knowledge of neuromorphic circuit design. As a showcase for the capabilities of the system, we describe the successful emulation of six different neural networks which cover a broad spectrum of both structure and functionality
Design and Implementation of BCM Rule Based on Spike-Timing Dependent Plasticity
The Bienenstock-Cooper-Munro (BCM) and Spike Timing-Dependent Plasticity
(STDP) rules are two experimentally verified form of synaptic plasticity where
the alteration of synaptic weight depends upon the rate and the timing of pre-
and post-synaptic firing of action potentials, respectively. Previous studies
have reported that under specific conditions, i.e. when a random train of
Poissonian distributed spikes are used as inputs, and weight changes occur
according to STDP, it has been shown that the BCM rule is an emergent property.
Here, the applied STDP rule can be either classical pair-based STDP rule, or
the more powerful triplet-based STDP rule. In this paper, we demonstrate the
use of two distinct VLSI circuit implementations of STDP to examine whether BCM
learning is an emergent property of STDP. These circuits are stimulated with
random Poissonian spike trains. The first circuit implements the classical
pair-based STDP, while the second circuit realizes a previously described
triplet-based STDP rule. These two circuits are simulated using 0.35 um CMOS
standard model in HSpice simulator. Simulation results demonstrate that the
proposed triplet-based STDP circuit significantly produces the threshold-based
behaviour of the BCM. Also, the results testify to similar behaviour for the
VLSI circuit for pair-based STDP in generating the BCM
MorphIC: A 65-nm 738k-Synapse/mm Quad-Core Binary-Weight Digital Neuromorphic Processor with Stochastic Spike-Driven Online Learning
Recent trends in the field of neural network accelerators investigate weight
quantization as a means to increase the resource- and power-efficiency of
hardware devices. As full on-chip weight storage is necessary to avoid the high
energy cost of off-chip memory accesses, memory reduction requirements for
weight storage pushed toward the use of binary weights, which were demonstrated
to have a limited accuracy reduction on many applications when
quantization-aware training techniques are used. In parallel, spiking neural
network (SNN) architectures are explored to further reduce power when
processing sparse event-based data streams, while on-chip spike-based online
learning appears as a key feature for applications constrained in power and
resources during the training phase. However, designing power- and
area-efficient spiking neural networks still requires the development of
specific techniques in order to leverage on-chip online learning on binary
weights without compromising the synapse density. In this work, we demonstrate
MorphIC, a quad-core binary-weight digital neuromorphic processor embedding a
stochastic version of the spike-driven synaptic plasticity (S-SDSP) learning
rule and a hierarchical routing fabric for large-scale chip interconnection.
The MorphIC SNN processor embeds a total of 2k leaky integrate-and-fire (LIF)
neurons and more than two million plastic synapses for an active silicon area
of 2.86mm in 65nm CMOS, achieving a high density of 738k synapses/mm.
MorphIC demonstrates an order-of-magnitude improvement in the area-accuracy
tradeoff on the MNIST classification task compared to previously-proposed SNNs,
while having no penalty in the energy-accuracy tradeoff.Comment: This document is the paper as accepted for publication in the IEEE
Transactions on Biomedical Circuits and Systems journal (2019), the
fully-edited paper is available at
https://ieeexplore.ieee.org/document/876400
A differential memristive synapse circuit for on-line learning in neuromorphic computing systems
Spike-based learning with memristive devices in neuromorphic computing
architectures typically uses learning circuits that require overlapping pulses
from pre- and post-synaptic nodes. This imposes severe constraints on the
length of the pulses transmitted in the network, and on the network's
throughput. Furthermore, most of these circuits do not decouple the currents
flowing through memristive devices from the one stimulating the target neuron.
This can be a problem when using devices with high conductance values, because
of the resulting large currents. In this paper we propose a novel circuit that
decouples the current produced by the memristive device from the one used to
stimulate the post-synaptic neuron, by using a novel differential scheme based
on the Gilbert normalizer circuit. We show how this circuit is useful for
reducing the effect of variability in the memristive devices, and how it is
ideally suited for spike-based learning mechanisms that do not require
overlapping pre- and post-synaptic pulses. We demonstrate the features of the
proposed synapse circuit with SPICE simulations, and validate its learning
properties with high-level behavioral network simulations which use a
stochastic gradient descent learning rule in two classification tasks.Comment: 18 Pages main text, 9 pages of supplementary text, 19 figures.
Patente
Neuro-memristive Circuits for Edge Computing: A review
The volume, veracity, variability, and velocity of data produced from the
ever-increasing network of sensors connected to Internet pose challenges for
power management, scalability, and sustainability of cloud computing
infrastructure. Increasing the data processing capability of edge computing
devices at lower power requirements can reduce several overheads for cloud
computing solutions. This paper provides the review of neuromorphic
CMOS-memristive architectures that can be integrated into edge computing
devices. We discuss why the neuromorphic architectures are useful for edge
devices and show the advantages, drawbacks and open problems in the field of
neuro-memristive circuits for edge computing
- …