Simulation and implementation of novel deep learning hardware architectures for resource constrained devices

Corey Lammie designed mixed signal memristive-complementary metal–oxide–semiconductor (CMOS) and field programmable gate arrays (FPGA) hardware architectures, which were used to reduce the power and resource requirements of Deep Learning (DL) systems; both during inference and training. Disruptive design methodologies, such as those explored in this thesis, can be used to facilitate the design of next-generation DL systems

Power Profile Obfuscation using RRAMs to Counter DPA Attacks

Side channel attacks, such as Differential Power Analysis (DPA), denote a special class of attacks in which sensitive key information is unveiled through information extracted from the physical device executing a cryptographic algorithm. This information leakage, known as side channel information, occurs from computations in a non-ideal system composed of electronic devices such as transistors. Power dissipation is one classic side channel source, which relays information of the data being processed. DPA uses statistical analysis to identify data-dependent correlations in sets of power measurements. Countermeasures against DPA focus on hiding or masking techniques at different levels of design abstraction and are typically associated with high power and area cost. Emerging technologies such as Resistive Random Access Memory (RRAM), offer unique opportunities to mitigate DPAs with their inherent memristor device characteristics such as variability in write time, ultra low power (0.1-3 pJ/bit), and high density (4F2). In this research, an RRAM based architecture is proposed to mitigate the DPA attacks by obfuscating the power profile. Specifically, a dual RRAM based memory module masks the power dissipation of the actual transaction by accessing both the data and its complement from the memory in tandem. DPA attack resiliency for a 128-bit AES cryptoprocessor using RRAM and CMOS memory modules is compared against baseline CMOS only technology. In the proposed AES architecture, four single port RRAM memory units store the intermediate state of the encryption. The correlation between the state data and sets of power measurement is masked due to power dissipated from inverse data access on dual RRAM memory. A customized simulation framework is developed to design the attack scenarios using Synopsys and Cadence tool suites, along with a Hamming weight DPA attack module. The attack mounted on a baseline CMOS architecture is successful and the full key is recovered. However, DPA attacks mounted on the dual CMOS and RRAM based AES cryptoprocessor yielded unsuccessful results with no keys recovered, demonstrating the resiliency of the proposed architecture against DPA attacks

Low Power Memory/Memristor Devices and Systems

This reprint focusses on achieving low-power computation using memristive devices. The topic was designed as a convenient reference point: it contains a mix of techniques starting from the fundamental manufacturing of memristive devices all the way to applications such as physically unclonable functions, and also covers perspectives on, e.g., in-memory computing, which is inextricably linked with emerging memory devices such as memristors. Finally, the reprint contains a few articles representing how other communities (from typical CMOS design to photonics) are fighting on their own fronts in the quest towards low-power computation, as a comparison with the memristor literature. We hope that readers will enjoy discovering the articles within

In-Memory Computing by Using Nano-ionic Memristive Devices

By reaching to the CMOS scaling limitation based on the Moore’s law and due to the increasing disparity between the processing units and memory performance, the quest is continued to find a suitable alternative to replace the conventional technology. The recently discovered two terminal element, memristor, is believed to be one of the most promising candidates for future very large scale integrated systems. This thesis is comprised of two main parts, (Part I) modeling the memristor devices, and (Part II) memristive computing. The first part is presented in one chapter and the second part of the thesis contains five chapters. The basics and fundamentals regarding the memristor functionality and memristive computing are presented in the introduction chapter. A brief detail of these two main parts is as follows: Part I: Modeling- This part presents an accurate model based on the charge transport mechanisms for nanoionic memristor devices. The main current mechanism in metal/insulator/metal (MIM) structures are assessed, a physic-based model is proposed and a SPICE model is presented and tested for four different fabricated devices. An accuracy comparison is done for various models for Ag/TiO2/ITO fabricated device. Also, the functionality of the model is tested for various input signals. Part II: Memristive computing- Memristive computing is about utilizing memristor to perform computational tasks. This part of the thesis is divided into neuromorphic, analog and digital computing schemes with memristor devices. – Neuromorphic computing- Two chapters of this thesis are about biologicalinspired memristive neural networks using STDP-based learning mechanism. The memristive implementation of two well-known spiking neuron models, Hudgkin-Huxley and Morris-Lecar, are assessed and utilized in the proposed memristive network. The synaptic connections are also memristor devices in this design. Unsupervised pattern classification tasks are done to ensure the right functionality of the system. – Analog computing- Memristor has analog memory property as it can be programmed to different memristance values. A novel memristive analog adder is designed by Continuous Valued Number System (CVNS) scheme and its circuit is comprised of addition and modulo blocks. The proposed analog adder design is explained and its functionality is tested for various numbers. It is shown that the CVNS scheme is compatible with memristive design and the environment resolution can be adjusted by the memristance ratio of the memristor devices. – Digital computing- Two chapters are dedicated for digital computing. In the first one, a development over IMPLY-based logic with memristor is provided to implement a 4:2 compressor circuit. In the second chapter, A novel resistive over a novel mirrored memristive crossbar platform. Different logic gates are designed with the proposed memristive logic method and the simulations are provided with Cadence to prove the functionality of the logic. The logic implementation over a mirrored memristive crossbars is also assessed

Memristors

This Edited Volume Memristors - Circuits and Applications of Memristor Devices is a collection of reviewed and relevant research chapters, offering a comprehensive overview of recent developments in the field of Engineering. The book comprises single chapters authored by various researchers and edited by an expert active in the physical sciences, engineering, and technology research areas. All chapters are complete in itself but united under a common research study topic. This publication aims at providing a thorough overview of the latest research efforts by international authors on physical sciences, engineering, and technology,and open new possible research paths for further novel developments

Memristive crossbars as hardware accelerators: modelling, design and new uses

Digital electronics has given rise to reliable, affordable, and scalable computing devices. However, new computing paradigms present challenges. For example, machine learning requires repeatedly processing large amounts of data; this creates a bottleneck in conventional computers, where computing and memory are separated. To add to that, Moore’s “law” is plateauing and is thus unlikely to address the increasing demand for computational power. In-memory computing, and specifically hardware accelerators for linear algebra, may address both of these issues. Memristive crossbar arrays are a promising candidate for such hardware accelerators. Memristive devices are fast, energy-efficient, and—when arranged in a crossbar structure—can compute vector-matrix products. Unfortunately, they come with their own set of limitations. The analogue nature of these devices makes them stochastic and thus less reliable compared to digital devices. It does not, however, necessarily make them unsuitable for computing. Nevertheless, successful deployment of analogue hardware accelerators requires a proper understanding of their drawbacks, ways of mitigating the effects of undesired physical behaviour, and applications where some degree of stochasticity is tolerable. In this thesis, I investigate the effects of nonidealities in memristive crossbar arrays, introduce techniques of minimising those negative effects, and present novel crossbar circuit designs for new applications. I mostly focus on physical implementations of neural networks and investigate the influence of device nonidealities on classification accuracy. To make memristive neural networks more reliable, I explore committee machines, rearrangement of crossbar lines, nonideality-aware training, and other techniques. I find that they all may contribute to the higher accuracy of physically implemented neural networks, often comparable to the accuracy of their digital counterparts. Finally, I introduce circuits that extend dot product computations to higher-rank arrays, different linear algebra operations, and quaternion vectors and matrices. These present opportunities for using crossbar arrays in new ways, including the processing of coloured images

Spike neural network architecture with memristive synapses using predictive Cmos model

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2020.O seguinte trabalho tem como objetivo propor e avaliar um circuito eletrônico que implemente uma arquitetura de rede neural com neurônios spike. Transistores do tipo MOSFET foram utilizados para implementar os neurônios. A rede proposta possui aprendizado do tipo spike timing dependent plasticity (STDP). Memristors foram utilizados como sinapses. A validação dos módulos de circuitos e do circuito da arquitetura completa foram realizados utilizando modelos SPICE dos dispositivos. Como a maioria dos dados sobre tecnologias de empresas possuem acesso restrito, algumas universidades fornecem modelos preditivos dos dispositivos com o intuito de reproduzir o comportamento real em tecnologias futuras, que foram empregados neste trabalho. Nesse projeto apresentamos dois tipos de Integrate and Fire Neuron(I&F) usando tecnologia CMOS de 32nm simulada no LTspice empregando o modelo BSIM4v4 concebido pela Universidade de Berkley e aplicando parâmetros preditivos fornecidos pelo Predictive Technology Model (PTM). Os resultados da simulação obtidos aqui, reduzem a tensão da fonte e o tamanho do chip em relação aos designs semelhantes mais recentemente implementado. Além disso, a comunicação entre neurônios e sinapses com um aprendizado STDP foi simulada com êxito.CAPESThe following work aims to establish and evaluate an electronic circuit that implements a neural network architecture with spike neurons. This project uses MOSFET-type transistors to achieve the neurons, and the proposed network has a spike-timing-dependent plasticity (STDP) learning aspect. It applies Memristors to function as synapses. The validation of the circuit modules and the circuit for complete architecture were performed using SPICE models of the devices. Since most data of company technologies is restricted, some universities provide predictive models to reproduce the real ones. In this dissertation, we present two types of Integrate and Fire Neuron (IF) (IF) using 32nm CMOS technology simulated in LTspice with BSIM4v4 model designed by Berkley University and applying predictive parameters provided by Predictive Technology Model (PTM). The simulation results obtained here reduces the font tension and the chip size to the most recent designs implemented. Communication between neurons and synapses with STDP learning has been successfully simulated

Bio-inspired learning and hardware acceleration with emerging memories

Machine Learning has permeated many aspects of engineering, ranging from the Internet of Things (IoT) applications to big data analytics. While computing resources available to implement these algorithms have become more powerful, both in terms of the complexity of problems that can be solved and the overall computing speed, the huge energy costs involved remains a significant challenge. The human brain, which has evolved over millions of years, is widely accepted as the most efficient control and cognitive processing platform. Neuro-biological studies have established that information processing in the human brain relies on impulse like signals emitted by neurons called action potentials. Motivated by these facts, the Spiking Neural Networks (SNNs), which are a bio-plausible version of neural networks have been proposed as an alternative computing paradigm where the timing of spikes generated by artificial neurons is central to its learning and inference capabilities. This dissertation demonstrates the computational power of the SNNs using conventional CMOS and emerging nanoscale hardware platforms. The first half of this dissertation presents an SNN architecture which is trained using a supervised spike-based learning algorithm for the handwritten digit classification problem. This network achieves an accuracy of 98.17% on the MNIST test data-set, with about 4X fewer parameters compared to the state-of-the-art neural networks achieving over 99% accuracy. In addition, a scheme for parallelizing and speeding up the SNN simulation on a GPU platform is presented. The second half of this dissertation presents an optimal hardware design for accelerating SNN inference and training with SRAM (Static Random Access Memory) and nanoscale non-volatile memory (NVM) crossbar arrays. Three prominent NVM devices are studied for realizing hardware accelerators for SNNs: Phase Change Memory (PCM), Spin Transfer Torque RAM (STT-RAM) and Resistive RAM (RRAM). The analysis shows that a spike-based inference engine with crossbar arrays of STT-RAM bit-cells is 2X and 5X more efficient compared to PCM and RRAM memories, respectively. Furthermore, the STT-RAM design has nearly 6X higher throughput per unit Watt per unit area than that of an equivalent SRAM-based (Static Random Access Memory) design. A hardware accelerator with on-chip learning on an STT-RAM memory array is also designed, requiring $16$ bits of floating-point synaptic weight precision to reach the baseline SNN algorithmic performance on the MNIST dataset. The complete design with STT-RAM crossbar array achieves nearly 20X higher throughput per unit Watt per unit mm^2 than an equivalent design with SRAM memory. In summary, this work demonstrates the potential of spike-based neuromorphic computing algorithms and its efficient realization in hardware based on conventional CMOS as well as emerging technologies. The schemes presented here can be further extended to design spike-based systems that can be ubiquitously deployed for energy and memory constrained edge computing applications

Simulation, Application, and Resilience of an Organic Neuromorphic Architecture, Made with Organic Bistable Devices and Organic Field Effect Transistors

This thesis presents work done simulating a type of organic neuromorphic architecture, modeled after Artificial Neural Network, and termed Synthetic Neural Network, or SNN. The first major contribution of this thesis is development of a single-transistor-single-organic-bistable-device-per-input circuit that approximates behavior of an artificial neuron. The efficacy of this design is validated by comparing the behavior of a single synthetic neuron to that of an artificial neuron as well as two examples involving a network of synthetic neurons. The analysis utilizes electrical characteristics of polymer electronic elements, namely Organic Bistable Device and Organic Field Effect Transistor, created in the laboratory at University of Denver. Polymer electronics is a new branch of electronics that is based on conductive and semi-conductive polymers. These new elements hold a great advantage over the inorganic electronics in the form of physical flexibility and low cost of fabrication. However, their device variability between individual devices is also much greater. Therefore the second major contribution of this thesis is the analysis of resilience of neural networks subjected to physical damage and other manufacturing faults