54 research outputs found
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
A Biomimetic Model of the Outer Plexiform Layer by Incorporating Memristive Devices
In this paper we present a biorealistic model for the first part of the early vision processing by incorporating memristive nanodevices. The architecture of the proposed network is based on the organisation and functioning of the outer plexiform layer (OPL) in the vertebrate retina. We demonstrate that memristive devices are indeed a valuable building block for neuromorphic architectures, as their highly non-linear and adaptive response could be exploited for establishing ultra-dense networks with similar dynamics to their biological counterparts. We particularly show that hexagonal memristive grids can be employed for faithfully emulating the smoothing-effect occurring at the OPL for enhancing the dynamic range of the system. In addition, we employ a memristor-based thresholding scheme for detecting the edges of grayscale images, while the proposed system is also evaluated for its adaptation and fault tolerance capacity against different light or noise conditions as well as distinct device yields
Energy Efficient Neocortex-Inspired Systems with On-Device Learning
Shifting the compute workloads from cloud toward edge devices can significantly improve the overall latency for inference and learning. On the contrary this paradigm shift exacerbates the resource constraints on the edge devices. Neuromorphic computing architectures, inspired by the neural processes, are natural substrates for edge devices. They offer co-located memory, in-situ training, energy efficiency, high memory density, and compute capacity in a small form factor. Owing to these features, in the recent past, there has been a rapid proliferation of hybrid CMOS/Memristor neuromorphic computing systems. However, most of these systems offer limited plasticity, target either spatial or temporal input streams, and are not demonstrated on large scale heterogeneous tasks. There is a critical knowledge gap in designing scalable neuromorphic systems that can support hybrid plasticity for spatio-temporal input streams on edge devices.
This research proposes Pyragrid, a low latency and energy efficient neuromorphic computing system for processing spatio-temporal information natively on the edge. Pyragrid is a full-scale custom hybrid CMOS/Memristor architecture with analog computational modules and an underlying digital communication scheme. Pyragrid is designed for hierarchical temporal memory, a biomimetic sequence memory algorithm inspired by the neocortex. It features a novel synthetic synapses representation that enables dynamic synaptic pathways with reduced memory usage and interconnects. The dynamic growth in the synaptic pathways is emulated in the memristor device physical behavior, while the synaptic modulation is enabled through a custom training scheme optimized for area and power.
Pyragrid features data reuse, in-memory computing, and event-driven sparse local computing to reduce data movement by ~44x and maximize system throughput and power efficiency by ~3x and ~161x over custom CMOS digital design. The innate sparsity in Pyragrid results in overall robustness to noise and device failure, particularly when processing visual input and predicting time series sequences. Porting the proposed system on edge devices can enhance their computational capability, response time, and battery life
Biologically Plausible Information Propagation in a CMOS Integrate-and-Fire Artificial Neuron Circuit with Memristive Synapses
Neuromorphic circuits based on spikes are currently envisioned as a viable option to achieve brain-like computation capabilities in specific electronic implementations while limiting power dissipation given their ability to mimic energy efficient bio-inspired mechanisms. While several network architectures have been developed to embed in hardware the bio-inspired learning rules found in the biological brain, such as the Spike Timing Dependent Plasticity, it is still unclear if hardware spiking neural network architectures can handle and transfer information akin to biological networks. In this work, we investigate the analogies between an artificial neuron combining memristor synapses and rate-based learning rule with biological neuron response in terms of information propagation from a theoretical perspective. Bio-inspired experiments have been reproduced by linking the biological probability of release with the artificial synapses conductance. Mutual information and surprise have been chosen as metrics to evidence how, for different values of synaptic weights, an artificial neuron allows to develop a reliable and biological resembling neural network in terms of information propagation and analysi
A CMOS Spiking Neuron for Brain-Inspired Neural Networks with Resistive Synapses and In-Situ Learning
Nanoscale resistive memories are expected to fuel dense integration of
electronic synapses for large-scale neuromorphic system. To realize such a
brain-inspired computing chip, a compact CMOS spiking neuron that performs
in-situ learning and computing while driving a large number of resistive
synapses is desired. This work presents a novel leaky integrate-and-fire neuron
design which implements the dual-mode operation of current integration and
synaptic drive, with a single opamp and enables in-situ learning with crossbar
resistive synapses. The proposed design was implemented in a 0.18 m CMOS
technology. Measurements show neuron's ability to drive a thousand resistive
synapses, and demonstrate an in-situ associative learning. The neuron circuit
occupies a small area of 0.01 mm and has an energy-efficiency of 9.3
pJspikesynapse
Cryogenic Neuromorphic Hardware
The revolution in artificial intelligence (AI) brings up an enormous storage
and data processing requirement. Large power consumption and hardware overhead
have become the main challenges for building next-generation AI hardware. To
mitigate this, Neuromorphic computing has drawn immense attention due to its
excellent capability for data processing with very low power consumption. While
relentless research has been underway for years to minimize the power
consumption in neuromorphic hardware, we are still a long way off from reaching
the energy efficiency of the human brain. Furthermore, design complexity and
process variation hinder the large-scale implementation of current neuromorphic
platforms. Recently, the concept of implementing neuromorphic computing systems
in cryogenic temperature has garnered intense interest thanks to their
excellent speed and power metric. Several cryogenic devices can be engineered
to work as neuromorphic primitives with ultra-low demand for power. Here we
comprehensively review the cryogenic neuromorphic hardware. We classify the
existing cryogenic neuromorphic hardware into several hierarchical categories
and sketch a comparative analysis based on key performance metrics. Our
analysis concisely describes the operation of the associated circuit topology
and outlines the advantages and challenges encountered by the state-of-the-art
technology platforms. Finally, we provide insights to circumvent these
challenges for the future progression of research
Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks
Artificial neural networks are notoriously power- and time-consuming when
implemented on conventional von Neumann computing systems. Consequently, recent
years have seen an emergence of research in machine learning hardware that
strives to bring memory and computing closer together. A popular approach is to
realise artificial neural networks in hardware by implementing their synaptic
weights using memristive devices. However, various device- and system-level
non-idealities usually prevent these physical implementations from achieving
high inference accuracy. We suggest applying a well-known concept in computer
science -- committee machines -- in the context of memristor-based neural
networks. Using simulations and experimental data from three different types of
memristive devices, we show that committee machines employing ensemble
averaging can successfully increase inference accuracy in physically
implemented neural networks that suffer from faulty devices, device-to-device
variability, random telegraph noise and line resistance. Importantly, we
demonstrate that the accuracy can be improved even without increasing the total
number of memristors.Comment: 22 pages, 18 figures, 4 table
- …