53 research outputs found
Neuromorphic Hebbian learning with magnetic tunnel junction synapses
Neuromorphic computing aims to mimic both the function and structure of
biological neural networks to provide artificial intelligence with extreme
efficiency. Conventional approaches store synaptic weights in non-volatile
memory devices with analog resistance states, permitting in-memory computation
of neural network operations while avoiding the costs associated with
transferring synaptic weights from a memory array. However, the use of analog
resistance states for storing weights in neuromorphic systems is impeded by
stochastic writing, weights drifting over time through stochastic processes,
and limited endurance that reduces the precision of synapse weights. Here we
propose and experimentally demonstrate neuromorphic networks that provide
high-accuracy inference thanks to the binary resistance states of magnetic
tunnel junctions (MTJs), while leveraging the analog nature of their stochastic
spin-transfer torque (STT) switching for unsupervised Hebbian learning. We
performed the first experimental demonstration of a neuromorphic network
directly implemented with MTJ synapses, for both inference and
spike-timing-dependent plasticity learning. We also demonstrated through
simulation that the proposed system for unsupervised Hebbian learning with
stochastic STT-MTJ synapses can achieve competitive accuracies for MNIST
handwritten digit recognition. By appropriately applying neuromorphic
principles through hardware-aware design, the proposed STT-MTJ neuromorphic
learning networks provide a pathway toward artificial intelligence hardware
that learns autonomously with extreme efficiency
In-memory computing with emerging memory devices: Status and outlook
Supporting data for "In-memory computing with emerging memory devices: status and outlook", submitted to APL Machine Learning
A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)
Neuromorphic computing systems comprise networks of neurons that use
asynchronous events for both computation and communication. This type of
representation offers several advantages in terms of bandwidth and power
consumption in neuromorphic electronic systems. However, managing the traffic
of asynchronous events in large scale systems is a daunting task, both in terms
of circuit complexity and memory requirements. Here we present a novel routing
methodology that employs both hierarchical and mesh routing strategies and
combines heterogeneous memory structures for minimizing both memory
requirements and latency, while maximizing programming flexibility to support a
wide range of event-based neural network architectures, through parameter
configuration. We validated the proposed scheme in a prototype multi-core
neuromorphic processor chip that employs hybrid analog/digital circuits for
emulating synapse and neuron dynamics together with asynchronous digital
circuits for managing the address-event traffic. We present a theoretical
analysis of the proposed connectivity scheme, describe the methods and circuits
used to implement such scheme, and characterize the prototype chip. Finally, we
demonstrate the use of the neuromorphic processor with a convolutional neural
network for the real-time classification of visual symbols being flashed to a
dynamic vision sensor (DVS) at high speed.Comment: 17 pages, 14 figure
Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications
With the advent of dedicated Deep Learning (DL) accelerators and neuromorphic
processors, new opportunities are emerging for applying deep and Spiking Neural
Network (SNN) algorithms to healthcare and biomedical applications at the edge.
This can facilitate the advancement of the medical Internet of Things (IoT)
systems and Point of Care (PoC) devices. In this paper, we provide a tutorial
describing how various technologies ranging from emerging memristive devices,
to established Field Programmable Gate Arrays (FPGAs), and mature Complementary
Metal Oxide Semiconductor (CMOS) technology can be used to develop efficient DL
accelerators to solve a wide variety of diagnostic, pattern recognition, and
signal processing problems in healthcare. Furthermore, we explore how spiking
neuromorphic processors can complement their DL counterparts for processing
biomedical signals. After providing the required background, we unify the
sparsely distributed research on neural network and neuromorphic hardware
implementations as applied to the healthcare domain. In addition, we benchmark
various hardware platforms by performing a biomedical electromyography (EMG)
signal processing task and drawing comparisons among them in terms of inference
delay and energy. Finally, we provide our analysis of the field and share a
perspective on the advantages, disadvantages, challenges, and opportunities
that different accelerators and neuromorphic processors introduce to healthcare
and biomedical domains. This paper can serve a large audience, ranging from
nanoelectronics researchers, to biomedical and healthcare practitioners in
grasping the fundamental interplay between hardware, algorithms, and clinical
adoption of these tools, as we shed light on the future of deep networks and
spiking neuromorphic processing systems as proponents for driving biomedical
circuits and systems forward.Comment: Submitted to IEEE Transactions on Biomedical Circuits and Systems (21
pages, 10 figures, 5 tables
Spiking CMOS-NVM mixed-signal neuromorphic ConvNet with circuit- and training-optimized temporal subsampling
We increasingly rely on deep learning algorithms to process colossal amount of unstructured visual data. Commonly, these deep learning algorithms are deployed as software models on digital hardware, predominantly in data centers. Intrinsic high energy consumption of Cloud-based deployment of deep neural networks (DNNs) inspired researchers to look for alternatives, resulting in a high interest in Spiking Neural Networks (SNNs) and dedicated mixed-signal neuromorphic hardware. As a result, there is an emerging challenge to transfer DNN architecture functionality to energy-efficient spiking non-volatile memory (NVM)-based hardware with minimal loss in the accuracy of visual data processing. Convolutional Neural Network (CNN) is the staple choice of DNN for visual data processing. However, the lack of analog-friendly spiking implementations and alternatives for some core CNN functions, such as MaxPool, hinders the conversion of CNNs into the spike domain, thus hampering neuromorphic hardware development. To address this gap, in this work, we propose MaxPool with temporal multiplexing for Spiking CNNs (SCNNs), which is amenable for implementation in mixed-signal circuits. In this work, we leverage the temporal dynamics of internal membrane potential of Integrate & Fire neurons to enable MaxPool decision-making in the spiking domain. The proposed MaxPool models are implemented and tested within the SCNN architecture using a modified version of the aihwkit framework, a PyTorch-based toolkit for modeling and simulating hardware-based neural networks. The proposed spiking MaxPool scheme can decide even before the complete spatiotemporal input is applied, thus selectively trading off latency with accuracy. It is observed that by allocating just 10% of the spatiotemporal input window for a pooling decision, the proposed spiking MaxPool achieves up to 61.74% accuracy with a 2-bit weight resolution in the CIFAR10 dataset classification task after training with back propagation, with only about 1% performance drop compared to 62.78% accuracy of the 100% spatiotemporal window case with the 2-bit weight resolution to reflect foundry-integrated ReRAM limitations. In addition, we propose the realization of one of the proposed spiking MaxPool techniques in an NVM crossbar array along with periphery circuits designed in a 130nm CMOS technology. The energy-efficiency estimation results show competitive performance compared to recent neuromorphic chip designs
Toward Reliable Compact Modeling of Multilevel 1T-1R RRAM Devices for Neuromorphic Systems
The authors would like to thank the financial support by Deutsche Forschungsgemeinschaft
(German Research Foundation) with Project-ID SFB1461 and by the Federal Ministry of Education
and Research of Germany under grant numbers 16ES1002, 16FMD01K, 16FMD02 and 16FMD03.
The authors also gratefully acknowledge the support of the Spanish Ministry of Science, Innovation
and Universities and the FEDER program through project TEC2017-84321-C4-3-R and project
A.TIC.117.UGR18 funded by the government of Andalusia (Spain) and the FEDER program. The
publication of this article was funded by the Open Access Fund of the Leibniz Association.The datasets generated during and/or analysed during the current
study are available from the corresponding author on reasonable request.In this work, three different RRAM compact models implemented in Verilog-A are analyzed and evaluated in order to reproduce the multilevel approach based on the switching capability of experimental devices. These models are integrated in 1T-1R cells to control their analog behavior by means of the compliance current imposed by the NMOS select transistor. Four different resistance levels are simulated and assessed with experimental verification to account for their multilevel capability. Further, an Artificial Neural Network study is carried out to evaluate in a real scenario the viability of the multilevel approach under study.German Research Foundation (DFG)
SFB1461Federal Ministry of Education & Research (BMBF)
16ES1002
16FMD01K
16FMD02
16FMD03Spanish Ministry of Science, Innovation and UniversitiesEuropean Commission
TEC2017-84321-C4-3-Rgovernment of Andalusia (Spain)
A.TIC.117.UGR18Leibniz Associatio
Cryogenic Neuromorphic Hardware
The revolution in artificial intelligence (AI) brings up an enormous storage
and data processing requirement. Large power consumption and hardware overhead
have become the main challenges for building next-generation AI hardware. To
mitigate this, Neuromorphic computing has drawn immense attention due to its
excellent capability for data processing with very low power consumption. While
relentless research has been underway for years to minimize the power
consumption in neuromorphic hardware, we are still a long way off from reaching
the energy efficiency of the human brain. Furthermore, design complexity and
process variation hinder the large-scale implementation of current neuromorphic
platforms. Recently, the concept of implementing neuromorphic computing systems
in cryogenic temperature has garnered intense interest thanks to their
excellent speed and power metric. Several cryogenic devices can be engineered
to work as neuromorphic primitives with ultra-low demand for power. Here we
comprehensively review the cryogenic neuromorphic hardware. We classify the
existing cryogenic neuromorphic hardware into several hierarchical categories
and sketch a comparative analysis based on key performance metrics. Our
analysis concisely describes the operation of the associated circuit topology
and outlines the advantages and challenges encountered by the state-of-the-art
technology platforms. Finally, we provide insights to circumvent these
challenges for the future progression of research
- …