Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics

The paper considers a neural network with a class of real extended memristors obtained via the parallel connection of an ideal memristor and a nonlinear resistor. The resistor has the same rectifying characteristic for the current as that used in relevant models in the literature to account for diode-like effects at the interface between the memristor metal and insulating material. The paper proves some fundamental results on the trajectory convergence of this class of real memristor neural networks under the assumption that the interconnection matrix satisfies some symmetry conditions. First of all, the paper shows that, while in the case of neural networks with ideal memristors, it is possible to explicitly find functions of the state variables that are invariants of motions, the same functions can be used as Lyapunov functions that decrease along the trajectories in the case of real memristors with rectifying characteristics. This fundamental property is then used to study convergence by means of a reduction-of-order technique in combination with a Lyapunov approach. The theoretical predictions are verified via numerical simulations, and the convergence results are illustrated via the applications of real memristor neural networks to the solution of some image processing tasks in real time

Neuro-memristive Circuits for Edge Computing: A review

The volume, veracity, variability, and velocity of data produced from the ever-increasing network of sensors connected to Internet pose challenges for power management, scalability, and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and show the advantages, drawbacks and open problems in the field of neuro-memristive circuits for edge computing

On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex

In this paper we present a very exciting overlap between emergent nanotechnology and neuroscience, which has been discovered by neuromorphic engineers. Specifically, we are linking one type of memristor nanotechnology devices to the biological synaptic update rule known as spike-time-dependent-plasticity (STDP) found in real biological synapses. Understanding this link allows neuromorphic engineers to develop circuit architectures that use this type of memristors to artificially emulate parts of the visual cortex. We focus on the type of memristors referred to as voltage or flux driven memristors and focus our discussions on a behavioral macro-model for such devices. The implementations result in fully asynchronous architectures with neurons sending their action potentials not only forward but also backward. One critical aspect is to use neurons that generate spikes of specific shapes. We will see how by changing the shapes of the neuron action potential spikes we can tune and manipulate the STDP learning rules for both excitatory and inhibitory synapses. We will see how neurons and memristors can be interconnected to achieve large scale spiking learning systems, that follow a type of multiplicative STDP learning rule. We will briefly extend the architectures to use three-terminal transistors with similar memristive behavior. We will illustrate how a V1 visual cortex layer can assembled and how it is capable of learning to extract orientations from visual data coming from a real artificial CMOS spiking retina observing real life scenes. Finally, we will discuss limitations of currently available memristors. The results presented are based on behavioral simulations and do not take into account non-idealities of devices and interconnects. The aim of this paper is to present, in a tutorial manner, an initial framework for the possible development of fully asynchronous STDP learning neuromorphic architectures exploiting two or three-terminal memristive type devices. All files used for the simulations are made available through the journal web site1

On the Application of PSpice for Localised Cloud Security

The work reported in this thesis commenced with a review of methods for creating random binary sequences for encoding data locally by the client before storing in the Cloud. The first method reviewed investigated evolutionary computing software which generated noise-producing functions from natural noise, a highly-speculative novel idea since noise is stochastic. Nevertheless, a function was created which generated noise to seed chaos oscillators which produced random binary sequences and this research led to a circuit-based one-time pad key chaos encoder for encrypting data. Circuit-based delay chaos oscillators, initialised with sampled electronic noise, were simulated in a linear circuit simulator called PSpice. Many simulation problems were encountered because of the nonlinear nature of chaos but were solved by creating new simulation parts, tools and simulation paradigms. Simulation data from a range of chaos sources was exported and analysed using Lyapunov analysis and identified two sources which produced one-time pad sequences with maximum entropy. This led to an encoding system which generated unlimited, infinitely-long period, unique random one-time pad encryption keys for plaintext data length matching. The keys were studied for maximum entropy and passed a suite of stringent internationally-accepted statistical tests for randomness. A prototype containing two delay chaos sources initialised by electronic noise was produced on a double-sided printed circuit board and produced more than 200 Mbits of OTPs. According to Vladimir Kotelnikov in 1941 and Claude Shannon in 1945, one-time pad sequences are theoretically-perfect and unbreakable, provided specific rules are adhered to. Two other techniques for generating random binary sequences were researched; a new circuit element, memristance was incorporated in a Chua chaos oscillator, and a fractional-order Lorenz chaos system with order less than three. Quantum computing will present many problems to cryptographic system security when existing systems are upgraded in the near future. The only existing encoding system that will resist cryptanalysis by this system is the unconditionally-secure one-time pad encryption

Temporal Data Analysis Using Reservoir Computing and Dynamic Memristors

Temporal data analysis including classification and forecasting is essential in a range of fields from finance to engineering. While static data are largely independent of each other, temporal data have a considerable correlation between the samples, which is important for temporal data analysis. Neural networks thus offer a more general and flexible approach since they do not depend on parameters of specific tasks but are driven only by the data. In particular, recurrent neural networks have gathered much attention since the temporal information captured by the recurrent connections improves the prediction performance. Recently, reservoir computing (RC), which evolves from recurrent neural networks, has been extensively studied for temporal data analysis as it can offer efficient temporal processing of recurrent neural networks with a low training cost. This dissertation presents a hardware implementation of the RC system using an emerging device - memristor, followed by a theoretical study on hierarchical architectures of the RC system. A RC hardware system based on dynamic tungsten oxide (WOx) memristors is first demonstrated. The internal short-term memory effects of the WOx memristors allow the memristor-based reservoir to nonlinearly map temporal inputs into reservoir states, where the projected features can be readily processed by a simple linear readout function. We use the system to experimentally demonstrate two standard benchmarking tasks: isolated spoken digit recognition with partial inputs and chaotic system forecasting. High classification accuracy of 99.2% is obtained for spoken digit recognition and autonomous chaotic time series forecasting has been demonstrated over the long term. We then investigate the influence of the hierarchical reservoir structure on the properties of the reservoir and the performance of the RC system. Analogous to deep neural networks, stacking sub-reservoirs in series is an efficient way to enhance the nonlinearity of data transformation to high-dimensional space and expand the diversity of temporal information captured by the reservoir. These deep reservoir systems offer better performance when compared to simply increasing the size of the reservoir or the number of sub-reservoirs. Low-frequency components are mainly captured by the sub-reservoirs in the later stages of the deep reservoir structure, similar to observations that more abstract information can be extracted by layers in the late stage of deep neural networks. When the total size of the reservoir is fixed, the tradeoff between the number of sub-reservoirs and the size of each sub-reservoir needs to be carefully considered, due to the degraded ability of the individual sub-reservoirs at small sizes. Improved performance of the deep reservoir structure alleviates the difficulty of implementing the RC system on hardware systems. Beyond temporal data classification and prediction, one of the interesting applications of temporal data analysis is inferring the neural connectivity patterns from the high-dimensional neural activity recording data. By computing the temporal correlation between the neural spikes, connections between the neurons can be inferred using statistics-based techniques, but it becomes increasingly computationally expensive for large scale neural systems. We propose a second-order memristor-based hardware system using the natively implemented spike-timing-dependent plasticity learning rule for neural connectivity inference. By incorporating biological features such as transmission delay to the neural networks, the proposed concept not only correctly infers the direct connections but also distinguishes direct connections from indirect connections. Effects of additional biophysical properties not considered in the simulation and challenges of experimental memristor implementation will be also discussed.PHDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/167995/1/moonjohn_1.pd

Advanced CMOS Integrated Circuit Design and Application

The recent development of various application systems and platforms, such as 5G, B5G, 6G, and IoT, is based on the advancement of CMOS integrated circuit (IC) technology that enables them to implement high-performance chipsets. In addition to development in the traditional fields of analog and digital integrated circuits, the development of CMOS IC design and application in high-power and high-frequency operations, which was previously thought to be possible only with compound semiconductor technology, is a core technology that drives rapid industrial development. This book aims to highlight advances in all aspects of CMOS integrated circuit design and applications without discriminating between different operating frequencies, output powers, and the analog/digital domains. Specific topics in the book include: Next-generation CMOS circuit design and application; CMOS RF/microwave/millimeter-wave/terahertz-wave integrated circuits and systems; CMOS integrated circuits specially used for wireless or wired systems and applications such as converters, sensors, interfaces, frequency synthesizers/generators/rectifiers, and so on; Algorithm and signal-processing methods to improve the performance of CMOS circuits and systems

Memristors

This Edited Volume Memristors - Circuits and Applications of Memristor Devices is a collection of reviewed and relevant research chapters, offering a comprehensive overview of recent developments in the field of Engineering. The book comprises single chapters authored by various researchers and edited by an expert active in the physical sciences, engineering, and technology research areas. All chapters are complete in itself but united under a common research study topic. This publication aims at providing a thorough overview of the latest research efforts by international authors on physical sciences, engineering, and technology,and open new possible research paths for further novel developments

Hardware for Memristive Neuromorphic Systems with Reliable Programming and Online Learning

Alternative computing technologies are highly sought after due to limitations on transistor fabrication improvements. Fabricated memristive technology allows for a non-volatile analog memory for neuromorphic computing. In an integrated CMOS process, the synapse circuits designed for a spiking neuromorphic system can use memristors to regulate accumulation in the neuron circuits. Testing the fabricated memristive devices composed of hafnium oxide and developing a model to represent the key device characteristics lead to specific design choices in implementing the analog memory core of the synapse circuit. The circuits I designed for neuromorphic computing in this process take advantage of the unique capabilities of the memristive device to store a programmable analog memory reliably and efficiently. I designed the peripheral circuitry required including the circuits for programming the memristor and for online learning capabilities