19 research outputs found
Design of Resistive Synaptic Devices and Array Architectures for Neuromorphic Computing
abstract: Over the past few decades, the silicon complementary-metal-oxide-semiconductor (CMOS) technology has been greatly scaled down to achieve higher performance, density and lower power consumption. As the device dimension is approaching its fundamental physical limit, there is an increasing demand for exploration of emerging devices with distinct operating principles from conventional CMOS. In recent years, many efforts have been devoted in the research of next-generation emerging non-volatile memory (eNVM) technologies, such as resistive random access memory (RRAM) and phase change memory (PCM), to replace conventional digital memories (e.g. SRAM) for implementation of synapses in large-scale neuromorphic computing systems.
Essentially being compact and “analog”, these eNVM devices in a crossbar array can compute vector-matrix multiplication in parallel, significantly speeding up the machine/deep learning algorithms. However, non-ideal eNVM device and array properties may hamper the learning accuracy. To quantify their impact, the sparse coding algorithm was used as a starting point, where the strategies to remedy the accuracy loss were proposed, and the circuit-level design trade-offs were also analyzed. At architecture level, the parallel “pseudo-crossbar” array to prevent the write disturbance issue was presented. The peripheral circuits to support various parallel array architectures were also designed. One key component is the read circuit that employs the principle of integrate-and-fire neuron model to convert the analog column current to digital output. However, the read circuit is not area-efficient, which was proposed to be replaced with a compact two-terminal oscillation neuron device that exhibits metal-insulator-transition phenomenon.
To facilitate the design exploration, a circuit-level macro simulator “NeuroSim” was developed in C++ to estimate the area, latency, energy and leakage power of various neuromorphic architectures. NeuroSim provides a wide variety of design options at the circuit/device level. NeuroSim can be used alone or as a supporting module to provide circuit-level performance estimation in neural network algorithms. A 2-layer multilayer perceptron (MLP) simulator with integration of NeuroSim was demonstrated to evaluate both the learning accuracy and circuit-level performance metrics for the online learning and offline classification, as well as to study the impact of eNVM reliability issues such as data retention and write endurance on the learning performance.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201
Soft eSkin:distributed touch sensing with harmonized energy and computing
Inspired by biology, significant advances have been made in the field of electronic skin (eSkin) or tactile skin. Many of these advances have come through mimicking the morphology of human skin and by distributing few touch sensors in an area. However, the complexity of human skin goes beyond mimicking few morphological features or using few sensors. For example, embedded computing (e.g. processing of tactile data at the point of contact) is centric to the human skin as some neuroscience studies show. Likewise, distributed cell or molecular energy is a key feature of human skin. The eSkin with such features, along with distributed and embedded sensors/electronics on soft substrates, is an interesting topic to explore. These features also make eSkin significantly different from conventional computing. For example, unlike conventional centralized computing enabled by miniaturized chips, the eSkin could be seen as a flexible and wearable large area computer with distributed sensors and harmonized energy. This paper discusses these advanced features in eSkin, particularly the distributed sensing harmoniously integrated with energy harvesters, storage devices and distributed computing to read and locally process the tactile sensory data. Rapid advances in neuromorphic hardware, flexible energy generation, energy-conscious electronics, flexible and printed electronics are also discussed. This article is part of the theme issue ‘Harmonizing energy-autonomous computing and intelligence’
A Soft-Pruning Method Applied During Training of Spiking Neural Networks for In-memory Computing Applications
Inspired from the computational efficiency of the biological brain, spiking neural networks (SNNs) emulate biological neural networks, neural codes, dynamics, and circuitry. SNNs show great potential for the implementation of unsupervised learning using in-memory computing. Here, we report an algorithmic optimization that improves energy efficiency of online learning with SNNs on emerging non-volatile memory (eNVM) devices. We develop a pruning method for SNNs by exploiting the output firing characteristics of neurons. Our pruning method can be applied during network training, which is different from previous approaches in the literature that employ pruning on already-trained networks. This approach prevents unnecessary updates of network parameters during training. This algorithmic optimization can complement the energy efficiency of eNVM technology, which offers a unique in-memory computing platform for the parallelization of neural network operations. Our SNN maintains ~90% classification accuracy on the MNIST dataset with up to ~75% pruning, significantly reducing the number of weight updates. The SNN and pruning scheme developed in this work can pave the way toward applications of eNVM based neuro-inspired systems for energy efficient online learning in low power applications
Recommended from our members
Charge Trap Transistors (CTT): Turning Logic Transistors into Embedded Non-Volatile Memory for Advanced High-k/Metal Gate CMOS Technologies
While need for embedded non-volatile memory (eNVM) in modern computing systems continues to grow rapidly, the options have been limited due to integration and scaling challenges as well as operational voltage incompatibilities. Introduced in this work is a unique multi-time programmable memory (MTPM) solution for advanced high-k/metal-gate (HKMG) CMOS technologies which turns as-fabricated standard logic transistors into eNVM elements, without the need for any process adders or additional masks. These logic transistors, when employed as eNVM elements, are dubbed “Charge Trap Transistors” (CTTs). The fundamental device physics, principles of operation, and technological breakthroughs required for employing logic transistors as eNVM are presented. Implementation of CTT eNVM in 32 nm, 22 nm, 14 nm, and 7 nm production technologies has been realized and demonstrated in this work. The emerging memory technology landscape and the space that the CTT technology occupies therein are examined.The motivation behind this work is to develop an eNVM technology that is completely process/mask-free, multi-time programmable, operable at low/logic-compatible voltages, scalable, and secure. The CTT technology satisfies all of the aforementioned criteria. CTTs offer a data retention lifetime of > 10 years at 125 �C and an operation temperature range of -55�-125� C. Hardware results demonstrate an endurance of > 10^4 program/erase cycles which is more than adequate for most embedded applications. Hardware security enhancement, on-chip reconfigurable encryption, firmware, BIOS, chip ID, redundancy, repair at wafer and module test and in the field, performance tailoring, and chip configuration are a few of the applications of CTT eNVM. Moreover, the CTT array in its native (unprogrammed) state measures very well as an entropy source for potential PUF (Physically Unclonable Function) applications such as identification, authentication, anti-counterfeiting, secure boot, and cryptographic IP. In addition to the numerous digital applications, CTTs can also be utilized as an analog memory for applications like neuromorphic computing for machine learning (ML) and artificial intelligence (AI)
Simulation and programming strategies to mitigate device non-idealities in memristor based neuromorphic systems
Since its inception, resistive random access memory (RRAM) has widely been regarded as a promising technology, not only for its potential to revolutionize non-volatile data storage by bridging the speed gap between traditional solid state drives (SSD) and dynamic random access memory (DRAM), but also for the promise it brings to in-memory and neuromorphic computing.
Despite the potential, the design process of RRAM neuromorphic arrays still finds itself in its infancy, as reliability (retention, endurance, programming linearity) and variability (read-to-read, cycle-to-cycle and device-to-device) issues remain major hurdles for the mainstream implementation of these systems.
One of the fundamental stages of neuromorphic design is the simulation stage. In this thesis, a simulation framework for evaluating the impact of RRAM non-idealities on NNs, that emphasizes flexibility and experimentation in NN topology and RRAM programming conditions is coded in MATLAB, making full use of its various toolboxes.
Using these tools as the groundwork, various RRAM non-idealities are comprehensively measured and their impact on both inference and training accuracy of a pattern recognition system based on the MNIST handwritten digits dataset are simulated.
In the inference front, variability originated from different sources (read-to-read and programming-to-programming) are statistically evaluated and modelled for two different device types: filamentary and non-filamentary. Based on these results, the impact of various variability sources on inference are simulated and compared, showing much more pronounced variability in the filamentary device compared to its non-filamentary counterpart. The staged programming scheme is introduced as a method to improve linearity and reduce programming variability, leading to negligible accuracy loss in non-filamentary devices. Random telegraph noise (RTN) remains the major source of read variability in both devices. These results can be explained by the difference in switching mechanisms of both devices.
In training, non-idealities such as conductance stepping and cycle-to-cycle variability are characterized and their impact on the training of NNs based on backpropagation are independently evaluated. Analysing the change of weight distributions during training reveals the different impacts on the SET and RESET processes. Based on these findings, a new selective programming strategy is introduced for the suppression of non-idealities impact on accuracy. Furthermore, the impact of these methods are analysed between different NN topologies, including traditional multi-layer perceptron (MLP) and convolutional neural network (CNN) configurations.
Finally, the new dynamic weight range rescaling methodology is introduced as a way of not only alleviating the constraints imposed in hardware due to the limited conductance range of RRAM in training, but also as way of increasing the flexibility of RRAM based deep synaptic layers to different sets of data
Emerging physical unclonable functions with nanotechnology
Physical unclonable functions (PUFs) are increasingly used for authentication and identification applications as well as the cryptographic key generation. An important feature of a PUF is the reliance on minute random variations in the fabricated hardware to derive a trusted random key. Currently, most PUF designs focus on exploiting process variations intrinsic to the CMOS technology. In recent years, progress in emerging nanoelectronic devices has demonstrated an increase in variation as a consequence of scaling down to the nanoregion. To date, emerging PUFs with nanotechnology have not been fully established, but they are expected to emerge. Initial research in this area aims to provide security primitives for emerging integrated circuits with nanotechnology. In this paper, we review emerging nanotechnology-based PUFs