384 research outputs found
Training large-scale ANNs on simulated resistive crossbar arrays
Accelerating training of artificial neural networks (ANN) with analog
resistive crossbar arrays is a promising idea. While the concept has been
verified on very small ANNs and toy data sets (such as MNIST), more
realistically sized ANNs and datasets have not yet been tackled. However, it is
to be expected that device materials and hardware design constraints, such as
noisy computations, finite number of resistive states of the device materials,
saturating weight and activation ranges, and limited precision of
analog-to-digital converters, will cause significant challenges to the
successful training of state-of-the-art ANNs. By using analog hardware aware
ANN training simulations, we here explore a number of simple algorithmic
compensatory measures to cope with analog noise and limited weight and output
ranges and resolutions, that dramatically improve the simulated training
performances on RPU arrays on intermediately to large-scale ANNs
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
Neuro-memristive Circuits for Edge Computing: A review
The volume, veracity, variability, and velocity of data produced from the
ever-increasing network of sensors connected to Internet pose challenges for
power management, scalability, and sustainability of cloud computing
infrastructure. Increasing the data processing capability of edge computing
devices at lower power requirements can reduce several overheads for cloud
computing solutions. This paper provides the review of neuromorphic
CMOS-memristive architectures that can be integrated into edge computing
devices. We discuss why the neuromorphic architectures are useful for edge
devices and show the advantages, drawbacks and open problems in the field of
neuro-memristive circuits for edge computing
An On-chip Trainable and Clock-less Spiking Neural Network with 1R Memristive Synapses
Spiking neural networks (SNNs) are being explored in an attempt to mimic
brain's capability to learn and recognize at low power. Crossbar architecture
with highly scalable Resistive RAM or RRAM array serving as synaptic weights
and neuronal drivers in the periphery is an attractive option for SNN.
Recognition (akin to reading the synaptic weight) requires small amplitude bias
applied across the RRAM to minimize conductance change. Learning (akin to
writing or updating the synaptic weight) requires large amplitude bias pulses
to produce a conductance change. The contradictory bias amplitude requirement
to perform reading and writing simultaneously and asynchronously, akin to
biology, is a major challenge. Solutions suggested in the literature rely on
time-division-multiplexing of read and write operations based on clocks, or
approximations ignoring the reading when coincidental with writing. In this
work, we overcome this challenge and present a clock-less approach wherein
reading and writing are performed in different frequency domains. This enables
learning and recognition simultaneously on an SNN. We validate our scheme in
SPICE circuit simulator by translating a two-layered feed-forward Iris
classifying SNN to demonstrate software-equivalent performance. The system
performance is not adversely affected by a voltage dependence of conductance in
realistic RRAMs, despite departing from linearity. Overall, our approach
enables direct implementation of biological SNN algorithms in hardware
Memristors for the Curious Outsiders
We present both an overview and a perspective of recent experimental advances
and proposed new approaches to performing computation using memristors. A
memristor is a 2-terminal passive component with a dynamic resistance depending
on an internal parameter. We provide an brief historical introduction, as well
as an overview over the physical mechanism that lead to memristive behavior.
This review is meant to guide nonpractitioners in the field of memristive
circuits and their connection to machine learning and neural computation.Comment: Perpective paper for MDPI Technologies; 43 page
Spatio-temporal Learning with Arrays of Analog Nanosynapses
Emerging nanodevices such as resistive memories are being considered for
hardware realizations of a variety of artificial neural networks (ANNs),
including highly promising online variants of the learning approaches known as
reservoir computing (RC) and the extreme learning machine (ELM). We propose an
RC/ELM inspired learning system built with nanosynapses that performs both
on-chip projection and regression operations. To address time-dynamic tasks,
the hidden neurons of our system perform spatio-temporal integration and can be
further enhanced with variable sampling or multiple activation windows. We
detail the system and show its use in conjunction with a highly analog
nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46
battery of spoken digits. The system achieves nearly perfect (99%) accuracy at
sufficient hidden layer size, which compares favorably with software results.
In addition, the model is extended to a larger dataset, the MNIST database of
handwritten digits. By translating the database into the time domain and using
variable integration windows, up to 95% classification accuracy is achieved. In
addition to an intrinsically low-power programming style, the proposed
architecture learns very quickly and can easily be converted into a spiking
system with negligible loss in performance- all features that confer
significant energy efficiency.Comment: 6 pages, 3 figures. Presented at 2017 IEEE/ACM Symposium on Nanoscale
architectures (NANOARCH
Memristive crossbars as hardware accelerators: modelling, design and new uses
Digital electronics has given rise to reliable, affordable, and scalable computing devices. However, new computing paradigms present challenges. For example, machine learning requires repeatedly processing large amounts of data; this creates a bottleneck in conventional computers, where computing and memory are separated. To add to that, Moore’s “law” is plateauing and is thus unlikely to address the increasing demand for computational power. In-memory computing, and specifically hardware accelerators for linear algebra, may address both of these issues.
Memristive crossbar arrays are a promising candidate for such hardware accelerators. Memristive devices are fast, energy-efficient, and—when arranged in a crossbar structure—can compute vector-matrix products. Unfortunately, they come with their own set of limitations. The analogue nature of these devices makes them stochastic and thus less reliable compared to digital devices. It does not, however, necessarily make them unsuitable for computing. Nevertheless, successful deployment of analogue hardware accelerators requires a proper understanding of their drawbacks, ways of mitigating the effects of undesired physical behaviour, and applications where some degree of stochasticity is tolerable.
In this thesis, I investigate the effects of nonidealities in memristive crossbar arrays, introduce techniques of minimising those negative effects, and present novel crossbar circuit designs for new applications. I mostly focus on physical implementations of neural networks and investigate the influence of device nonidealities on classification accuracy. To make memristive neural networks more reliable, I explore committee machines, rearrangement of crossbar lines, nonideality-aware training, and other techniques. I find that they all may contribute to the higher accuracy of physically implemented neural networks, often comparable to the accuracy of their digital counterparts. Finally, I introduce circuits that extend dot product computations to higher-rank arrays, different linear algebra operations, and quaternion vectors and matrices. These present opportunities for using crossbar arrays in new ways, including the processing of coloured images
- …