Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing

Reservoir computing (RC) is gaining traction in several signal processing domains, owing to its nonlinear stateful computation, spatiotemporal encoding, and reduced training complexity over recurrent neural networks (RNNs). Previous studies have shown the effectiveness of software-based RCs for a wide spectrum of applications. A parallel body of work indicates that realizing RNN architectures using custom integrated circuits and reconfigurable hardware platforms yields significant improvements in power and latency. In this research, we propose a neuromemristive RC architecture, with doubly twisted toroidal structure, that is validated for biosignal processing applications. We exploit the device mismatch to implement the random weight distributions within the reservoir and propose mixed-signal subthreshold circuits for energy efficiency. A comprehensive analysis is performed to compare the efficiency of the neuromemristive RC architecture in both digital(reconfigurable) and subthreshold mixed-signal realizations. Both EEG and EMG biosignal benchmarks are used for validating the RC designs. The proposed RC architecture demonstrated an accuracy of 90% and 84% for epileptic seizure detection and EMG prosthetic finger control respectively

Design of a Neuromemristive Echo State Network Architecture

Echo state neural networks (ESNs) provide an efficient classification technique for spatiotemporal signals. The feedback connections in the ESN enable feature extraction in both spatial and temporal components in time series data. This property has been used in several application domains such as image and video analysis, anomaly detection, and speech recognition. The software implementations of the ESN demonstrated efficiency in processing such applications, and have low design cost and flexibility. However, hardware implementation is necessary for power constrained resources applications such as therapeutic and mobile devices. Moreover, software realization consumes an order or more power compared to the hardware realization. In this work, a hardware ESN architecture with neuromemristive system is proposed. A neuromemristive system is a brain inspired computing system that uses memristive devises for synaptic plasticity. The memristive devices in neuromemristive systems have several interesting properties such as small footprint, simple device structure, and most importantly zero static power dissipation. The proposed architecture is reconfigurable for different ESN topologies. 2-D mesh architecture and toroidal networks are exploited in the reservoir layer. The relation between performance of the proposed reservoir architecture and reservoir metrics are analyzed. The proposed architecture is tested on a suite of medical and human computer interaction applications. The benchmark suite includes epileptic seizure detection, speech emotion recognition, and electromyography (EMG) based finger motion recognition. The proposed ESN architecture demonstrated an accuracy of $90\%$, $96\%$, and $84\%$ for epileptic seizure detection, speech emotion recognition and EMG prosthetic fingers control respectively

Spatio-temporal Learning with Arrays of Analog Nanosynapses

Emerging nanodevices such as resistive memories are being considered for hardware realizations of a variety of artificial neural networks (ANNs), including highly promising online variants of the learning approaches known as reservoir computing (RC) and the extreme learning machine (ELM). We propose an RC/ELM inspired learning system built with nanosynapses that performs both on-chip projection and regression operations. To address time-dynamic tasks, the hidden neurons of our system perform spatio-temporal integration and can be further enhanced with variable sampling or multiple activation windows. We detail the system and show its use in conjunction with a highly analog nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46 battery of spoken digits. The system achieves nearly perfect (99%) accuracy at sufficient hidden layer size, which compares favorably with software results. In addition, the model is extended to a larger dataset, the MNIST database of handwritten digits. By translating the database into the time domain and using variable integration windows, up to 95% classification accuracy is achieved. In addition to an intrinsically low-power programming style, the proposed architecture learns very quickly and can easily be converted into a spiking system with negligible loss in performance- all features that confer significant energy efficiency.Comment: 6 pages, 3 figures. Presented at 2017 IEEE/ACM Symposium on Nanoscale architectures (NANOARCH

In materia implementation strategies of physical reservoir computing with memristive nanonetworks

Physical reservoir computing (RC) represents a computational framework that exploits information-processing capabilities of programmable matter, allowing the realization of energy-efficient neuromorphic hardware with fast learning and low training cost. Despite self-organized memristive networks have been demonstrated as physical reservoir able to extract relevant features from spatiotemporal input signals, multiterminal nanonetworks open the possibility for novel strategies of computing implementation. In this work, we report on implementation strategies of in materia RC with self-assembled memristive networks. Besides showing the spatiotemporal information processing capabilities of self-organized nanowire networks, we show through simulations that the emergent collective dynamics allows unconventional implementations of RC where the same electrodes can be used as both reservoir inputs and outputs. By comparing different implementation strategies on a digit recognition task, simulations show that the unconventional implementation allows a reduction of the hardware complexity without limiting computing capabilities, thus providing new insights for taking full advantage of in materia computing toward a rational design of neuromorphic systems

Design of Robust Memristor-Based Neuromorphic Circuits and Systems with Online Learning

Computing systems that are capable of performing human-like cognitive tasks have been an area of active research in the recent past. However, due to the bottleneck faced by the traditionally adopted von Neumann computing architecture, bio-inspired neural network style computing paradigm has seen a spike in research interest. Physical implementations of this paradigm of computing are known as neuromorphic systems. In the recent years, in the domain of neuromorphic systems, memristor based neuromorphic systems have gained increased attention from the research community due to the advantages offered by memristors such as their nanoscale size, nonvolatile nature and power efficient programming capability. However, these devices also suffer from a variety of non-ideal behaviors such as switching speed and threshold asymmetry, limited resolution and endurance that can have a detrimental impact on the operation of the systems employing these devices. This work aims to develop device-aware circuits that are robust in the face of such non-ideal properties. A bi-memristor synapse is first presented whose spike-timing-dependent plasticity (STDP) behavior can be precisely controlled on-chip and hence is shown to be robust. Later, a mixed-mode neuron is introduced that is amenable for use in conjunction with a range of memristors without needing to custom design it. These circuits are then used together to construct a memristive crossbar based system with supervised STDP learning to perform a pattern recognition application. The learning in the crossbar system is shown to be robust to the device-level issues owing to the robustness of the proposed circuits. Lastly, the proposed circuits are applied to build a liquid state machine based reservoir computing system. The reservoir used here is a spiking recurrent neural network generated using an evolutionary optimization algorithm and the readout layer is built with the crossbar system presented earlier, with STDP based online learning. A generalized framework for the hardware implementation of this system is proposed and it is shown that this liquid state machine is robust against device-level switching issues that would have otherwise impacted learning in the readout layer. Thereby, it is demonstrated that the proposed circuits along with their learning techniques can be used to build robust memristor-based neuromorphic systems with online learning