Design of Neuromemristive Systems for Visual Information Processing

Neuromemristive systems (NMSs) are brain-inspired, adaptive computer architectures based on emerging resistive memory technology (memristors). NMSs adopt a mixed-signal design approach with closely-coupled memory and processing, resulting in high area and energy efficiencies. Previous work suggests that NMSs could even supplant conventional architectures in niche application domains such as visual information processing. However, given the infancy of the field, there are still several obstacles impeding the transition of these systems from theory to practice. This dissertation advances the state of NMS research by addressing open design problems spanning circuit, architecture, and system levels. Novel synapse, neuron, and plasticity circuits are designed to reduce NMSs’ area and power consumption by using current-mode design techniques and exploiting device variability. Circuits are designed in a 45 nm CMOS process with memristor models based on multilevel (W/Ag-chalcogenide/W) and bistable (Ag/GeS2/W) device data. Higher-level behavioral, power, area, and variability models are ported into MATLAB to accelerate the overall simulation time. The circuits designed in this work are integrated into neural network architectures for visual information processing tasks, including feature detection, clustering, and classification. Networks in the NMSs are trained with novel stochastic learning algorithms that achieve 3.5 reduction in circuit area, reduced design complexity, and exhibit similar convergence properties compared to the least-mean-squares algorithm. This work also examines the effects of device-level variations on NMS performance, which has received limited attention in previous work. The impact of device variations is reduced with a partial on-chip training methodology that enables NMSs to be configured with relatively sophisticated algorithms (e.g. resilient backpropagation), while maximizing their area-accuracy tradeoff

Deep Liquid State Machines with Neural Plasticity and On-Device Learning

The Liquid State Machine (LSM) is a recurrent spiking neural network designed for efficient processing of spatio-temporal streams of information. LSMs have several inbuilt features such as robustness, fast training and inference speed, generalizability, continual learning (no catastrophic forgetting), and energy efficiency. These features make LSM’s an ideal network for deploying intelligence on-device. In general, single LSMs are unable to solve complex real-world tasks. Recent literature has shown emergence of hierarchical architectures to support temporal information processing over different time scales. However, these approaches do not typically investigate the optimum topology for communication between layers in the hierarchical network, or assume prior knowledge about the target problem and are not generalizable. In this thesis, a deep Liquid State Machine (deep-LSM) network architecture is proposed. The deep-LSM uses staggered reservoirs to process temporal information on multiple timescales. A key feature of this network is that neural plasticity and attention are embedded in the topology to bolster its performance for complex spatio-temporal tasks. An advantage of the deep-LSM is that it exploits the random projection native to the LSM as well as local plasticity mechanisms to optimize the data transfer between sequential layers. Both random projections and local plasticity mechanisms are ideal for on-device learning due to their low computational complexity and the absence of backpropagating error. The deep-LSM is deployed on a custom learning architecture with memristors to study the feasibility of on-device learning. The performance of the deep-LSM is demonstrated on speech recognition and seizure detection applications

Mitigating Differential Power Analysis Attacks on AES using NeuroMemristive Hardware

Cryptographic algorithms such as the Advanced Encryption Standard (AES) are vulnerable to side channel attacks. AES was once thought to be impervious to attacks, but this proved to be true only for a mathematical model of AES, not a physical realization. Hard- ware implementations leak side channel information such as power dissipation. One of the practical SCA attacks is the Differential power analysis (DPA) attack, which statistically analyzes power measurements to find data-dependent correlations. Several countermeasures against DPA have been proposed at the circuit and logic level in conventional technologies. These techniques generally include masking the data inside the algorithm or hiding the power profile. Next generation processors bring in additional challenges to mitigate DPA attacks, by way of heterogeneity of the devices used in the hardware realizations. Neuromemristive systems hold potential in this domain and also bring new challenges to the hardware security of cryptosystems. In this exploratory work, a neuromemristive architecture was designed to compute an AES transformation and mitigate DPA attacks. The random power profile of the neuromemristive architecture reduces the correlations between data and power consumption. Hardware primitives, such as neuron and synapse circuits were developed along with a framework to generate neural networks in hardware. An attack framework was developed to run DPA attacks using different leakage models. A baseline AES cryptoprocessor using only CMOS technology was attacked successfully. The SubBytes transformation was replaced by a neuromemristive architecture, and the proposed designs were more resilient against DPA attacks at the cost of increased power consumption

Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing

Reservoir computing (RC) is gaining traction in several signal processing domains, owing to its nonlinear stateful computation, spatiotemporal encoding, and reduced training complexity over recurrent neural networks (RNNs). Previous studies have shown the effectiveness of software-based RCs for a wide spectrum of applications. A parallel body of work indicates that realizing RNN architectures using custom integrated circuits and reconfigurable hardware platforms yields significant improvements in power and latency. In this research, we propose a neuromemristive RC architecture, with doubly twisted toroidal structure, that is validated for biosignal processing applications. We exploit the device mismatch to implement the random weight distributions within the reservoir and propose mixed-signal subthreshold circuits for energy efficiency. A comprehensive analysis is performed to compare the efficiency of the neuromemristive RC architecture in both digital(reconfigurable) and subthreshold mixed-signal realizations. Both EEG and EMG biosignal benchmarks are used for validating the RC designs. The proposed RC architecture demonstrated an accuracy of 90% and 84% for epileptic seizure detection and EMG prosthetic finger control respectively

Spatio-temporal Learning with Arrays of Analog Nanosynapses

Emerging nanodevices such as resistive memories are being considered for hardware realizations of a variety of artificial neural networks (ANNs), including highly promising online variants of the learning approaches known as reservoir computing (RC) and the extreme learning machine (ELM). We propose an RC/ELM inspired learning system built with nanosynapses that performs both on-chip projection and regression operations. To address time-dynamic tasks, the hidden neurons of our system perform spatio-temporal integration and can be further enhanced with variable sampling or multiple activation windows. We detail the system and show its use in conjunction with a highly analog nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46 battery of spoken digits. The system achieves nearly perfect (99%) accuracy at sufficient hidden layer size, which compares favorably with software results. In addition, the model is extended to a larger dataset, the MNIST database of handwritten digits. By translating the database into the time domain and using variable integration windows, up to 95% classification accuracy is achieved. In addition to an intrinsically low-power programming style, the proposed architecture learns very quickly and can easily be converted into a spiking system with negligible loss in performance- all features that confer significant energy efficiency.Comment: 6 pages, 3 figures. Presented at 2017 IEEE/ACM Symposium on Nanoscale architectures (NANOARCH