1,018 research outputs found
Neuro-memristive Circuits for Edge Computing: A review
The volume, veracity, variability, and velocity of data produced from the
ever-increasing network of sensors connected to Internet pose challenges for
power management, scalability, and sustainability of cloud computing
infrastructure. Increasing the data processing capability of edge computing
devices at lower power requirements can reduce several overheads for cloud
computing solutions. This paper provides the review of neuromorphic
CMOS-memristive architectures that can be integrated into edge computing
devices. We discuss why the neuromorphic architectures are useful for edge
devices and show the advantages, drawbacks and open problems in the field of
neuro-memristive circuits for edge computing
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays.
Resistive RAM crossbar arrays offer an attractive solution to minimize off-chip data transfer and parallelize on-chip computations for neural networks. Here, we report a hardware/software co-design approach based on low energy subquantum conductive bridging RAM (CBRAM®) devices and a network pruning technique to reduce network level energy consumption. First, we demonstrate low energy subquantum CBRAM devices exhibiting gradual switching characteristics important for implementing weight updates in hardware during unsupervised learning. Then we develop a network pruning algorithm that can be employed during training, different from previous network pruning approaches applied for inference only. Using a 512 kbit subquantum CBRAM array, we experimentally demonstrate high recognition accuracy on the MNIST dataset for digital implementation of unsupervised learning. Our hardware/software co-design approach can pave the way towards resistive memory based neuro-inspired systems that can autonomously learn and process information in power-limited settings
A differential memristive synapse circuit for on-line learning in neuromorphic computing systems
Spike-based learning with memristive devices in neuromorphic computing
architectures typically uses learning circuits that require overlapping pulses
from pre- and post-synaptic nodes. This imposes severe constraints on the
length of the pulses transmitted in the network, and on the network's
throughput. Furthermore, most of these circuits do not decouple the currents
flowing through memristive devices from the one stimulating the target neuron.
This can be a problem when using devices with high conductance values, because
of the resulting large currents. In this paper we propose a novel circuit that
decouples the current produced by the memristive device from the one used to
stimulate the post-synaptic neuron, by using a novel differential scheme based
on the Gilbert normalizer circuit. We show how this circuit is useful for
reducing the effect of variability in the memristive devices, and how it is
ideally suited for spike-based learning mechanisms that do not require
overlapping pre- and post-synaptic pulses. We demonstrate the features of the
proposed synapse circuit with SPICE simulations, and validate its learning
properties with high-level behavioral network simulations which use a
stochastic gradient descent learning rule in two classification tasks.Comment: 18 Pages main text, 9 pages of supplementary text, 19 figures.
Patente
A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation
Memristive devices represent a promising technology for building neuromorphic
electronic systems. In addition to their compactness and non-volatility
features, they are characterized by computationally relevant physical
properties, such as state-dependence, non-linear conductance changes, and
intrinsic variability in both their switching threshold and conductance values,
that make them ideal devices for emulating the bio-physics of real synapses. In
this paper we present a spiking neural network architecture that supports the
use of memristive devices as synaptic elements, and propose mixed-signal
analog-digital interfacing circuits which mitigate the effect of variability in
their conductance values and exploit their variability in the switching
threshold, for implementing stochastic learning. The effect of device
variability is mitigated by using pairs of memristive devices configured in a
complementary push-pull mechanism and interfaced to a current-mode normalizer
circuit. The stochastic learning mechanism is obtained by mapping the desired
change in synaptic weight into a corresponding switching probability that is
derived from the intrinsic stochastic behavior of memristive devices. We
demonstrate the features of the CMOS circuits and apply the architecture
proposed to a standard neural network hand-written digit classification
benchmark based on the MNIST data-set. We evaluate the performance of the
approach proposed on this benchmark using behavioral-level spiking neural
network simulation, showing both the effect of the reduction in conductance
variability produced by the current-mode normalizer circuit, and the increase
in performance as a function of the number of memristive devices used in each
synapse.Comment: 13 pages, 12 figures, accepted for Faraday Discussion
Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
Recent studies have shown that synaptic unreliability is a robust and
sufficient mechanism for inducing the stochasticity observed in cortex. Here,
we introduce Synaptic Sampling Machines, a class of neural network models that
uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised
learning. Similar to the original formulation of Boltzmann machines, these
models can be viewed as a stochastic counterpart of Hopfield networks, but
where stochasticity is induced by a random mask over the connections. Synaptic
stochasticity plays the dual role of an efficient mechanism for sampling, and a
regularizer during learning akin to DropConnect. A local synaptic plasticity
rule implementing an event-driven form of contrastive divergence enables the
learning of generative models in an on-line fashion. Synaptic sampling machines
perform equally well using discrete-timed artificial units (as in Hopfield
networks) or continuous-timed leaky integrate & fire neurons. The learned
representations are remarkably sparse and robust to reductions in bit precision
and synapse pruning: removal of more than 75% of the weakest connections
followed by cursory re-learning causes a negligible performance loss on
benchmark classification tasks. The spiking neuron-based synaptic sampling
machines outperform existing spike-based unsupervised learners, while
potentially offering substantial advantages in terms of power and complexity,
and are thus promising models for on-line learning in brain-inspired hardware
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
- …