Hardware design of LIF with Latency neuron model with memristive STDP synapses

In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode more information than the common Integrate-and-Fire models, typically considered for neuromorphic implementations. In our system LIFL neuron is implemented using CMOS circuits while memristor is used for the implementation of the STDP synapse. A description of the entire circuit is provided. Finally, the capabilities of the proposed architecture have been evaluated by simulating a motif composed of three neurons and two synapses. The simulation results confirm the validity of the proposed system and its suitability for the design of more complex spiking neural network

An Investigation into Neuromorphic ICs using Memristor-CMOS Hybrid Circuits

The memristance of a memristor depends on the amount of charge flowing through it and when current stops flowing through it, it remembers the state. Thus, memristors are extremely suited for implementation of memory units. Memristors find great application in neuromorphic circuits as it is possible to couple memory and processing, compared to traditional Von-Neumann digital architectures where memory and processing are separate. Neural networks have a layered structure where information passes from one layer to another and each of these layers have the possibility of a high degree of parallelism. CMOS-Memristor based neural network accelerators provide a method of speeding up neural networks by making use of this parallelism and analog computation. In this project we have conducted an initial investigation into the current state of the art implementation of memristor based programming circuits. Various memristor programming circuits and basic neuromorphic circuits have been simulated. The next phase of our project revolved around designing basic building blocks which can be used to design neural networks. A memristor bridge based synaptic weighting block, a operational transconductor based summing block were initially designed. We then designed activation function blocks which are used to introduce controlled non-linearity. Blocks for a basic rectified linear unit and a novel implementation for tan-hyperbolic function have been proposed. An artificial neural network has been designed using these blocks to validate and test their performance. We have also used these fundamental blocks to design basic layers of Convolutional Neural Networks. Convolutional Neural Networks are heavily used in image processing applications. The core convolutional block has been designed and it has been used as an image processing kernel to test its performance.Comment: Bachelor's thesi

Memristor based neural networks: Feasibility, theories and approaches

Memristor-based neural networks refer to the utilisation of memristors, the newly emerged nanoscale devices, in building neural networks. The memristor was first postulated by Leon Chua in 1971 as the fourth fundamental passive circuit element and experimentally validated by one of HP labs in 2008. Memristors, short for memory-resistor, have a peculiar memory effect which distinguishes them from resistors. By applying a bias voltage across it, the resistance of a memristor, namely memristance, is changed. In addition, the memristance is retained when the power supply is removed which demonstrates the non-volatility of the memristor. Memristor-based neural networks are currently being researched in order to replace complementary metal-oxide-semiconductor (CMOS) devices in neuromorphic circuits with memristors and to investigate their potential applications. Current research primarily focuses on the utilisation of memristors as synaptic connections between neurons, however in any application it may be possible to allow memristors to perform computation in a natural way which attempts to avoid additional CMOS devices. Examples of such methods utilised in neural networks are presented in this thesis, such as memristor-based cellular neural network (CNN) structures, the memristive spiking-time dependent plasticity (STDP) model and the exploration of their potential applications. This thesis presents manifold studies in the topic of memristor-based neural networks from theories and feasibility to approaches to implementations. Studies are divided into two parts which are the utilisation of memristors in non-spiking neural networks and spiking neural networks (SNNs). At the beginning of the thesis, fundamentals of neural networks and memristors are explored with the analysis of the physical properties and $v-i$ behaviour of memristors. In the studies of memristor-based non-spiking neural networks, a staircase memristor model is presented based on memristors which have multi-level resistive states and the delayed-switching effect. This model is adapted to CNNs and echo state networks (ESNs) as applications that benefit from memristive implementations. In the studies of memristor-based SNNs, a trace-based memristive STDP model is proposed and discussed to overcome the incompatibility issues of the previous model with all-to-all spike interaction. The work also presents applications of the trace-based memristive model in associative learning with retention loss and supervised learning. The computational results of experiments with different applications have shown that memristor-based neural networks will be advantageous in building synchronous or asynchronous parallel neuromorphic systems. The work presents several new findings on memristor modelling, memristor-based neural network structures and memristor-based associative learning. These studies address unexplored research areas in the context of memristor-based neural networks to the best of our knowledge, and therefore form original contributions

The Effects of Radiation on Memristor-Based Electronic Spiking Neural Networks

In this dissertation, memristor-based spiking neural networks (SNNs) are used to analyze the effect of radiation on the spatio-temporal pattern recognition (STPR) capability of the networks. Two-terminal resistive memory devices (memristors) are used as synapses to manipulate conductivity paths in the network. Spike-timing-dependent plasticity (STDP) learning behavior results in pattern learning and is achieved using biphasic shaped pre- and post-synaptic spikes. A TiO2 based non-linear drift memristor model designed in Verilog-A implements synaptic behavior and is modified to include experimentally observed effects of state-altering, ionizing, and off-state degradation radiation on the device. The impact of neuron “death” (disabled neuron circuits) due to radiation is also examined. In general, radiation interaction events distort the STDP learning curve undesirably, favoring synaptic potentiation. At lower short-term flux, the network is able to recover and relearn the pattern with consistent training, although some pixels may be affected due to stability issues. As the radiation flux and duration increases, it can overwhelm the leaky integrate-and-fire (LIF) post-synaptic neuron circuit, and the network does not learn the pattern. On the other hand, in the absence of the pattern, the radiation effects cumulate, and the system never regains stability. Neuron-death simulation results emphasize the importance of non-participating neurons during the learning process, concluding that non-participating afferents contribute to improving the learning ability of the neural network. Instantaneous neuron death proves to be more detrimental for the network compared to when the afferents die over time thus, retaining the network’s pattern learning capability

Analog Axon Hillock Neuron Design for Memristive Neuromorphic Systems

Neuromorphic electronics studies the physical realization of neural networks in discrete circuit components. Hardware implementations of neural networks take advantage of highly parallelized computing power with low energy systems. The hardware designed for these systems functions as a low power, low area alternative to computer simulations. With on-line learning in the system, hardware implementations of neural networks can further improve their solution to a given task.In this work, the analog computational system presented is the computational core for running a spiking neural network model. This component of a neural network, the neuron, is one of the building blocks used to create neural networks. The neuron takes inputs from the connected synapses, which each store a weight value. The inputs are stored in the neuron and checked against a threshold. The neuron activates, causing a firing event, when the neuron’s internal storage crosses its threshold. The neuron designed is an Axon-Hillock neuron utilizing memristive synapses for low area and energy operation

A spintronic Huxley-Hodgkin-analogue neuron implemented with a single magnetic tunnel junction

Spiking neural networks aim to emulate the brain's properties to achieve similar parallelism and high-processing power. A caveat of these neural networks is the high computational cost to emulate, while current proposals for analogue implementations are energy inefficient and not scalable. We propose a device based on a single magnetic tunnel junction to perform neuron firing for spiking neural networks without the need of any resetting procedure. We leverage two physics, magnetism and thermal effects, to obtain a bio-realistic spiking behavior analogous to the Huxley-Hodgkin model of the neuron. The device is also able to emulate the simpler Leaky-Integrate and Fire model. Numerical simulations using experimental-based parameters demonstrate firing frequency in the MHz to GHz range under constant input at room temperature. The compactness, scalability, low cost, CMOS-compatibility, and power efficiency of magnetic tunnel junctions advocate for their broad use in hardware implementations of spiking neural networks.Comment: 23 pages, 6 figures, 2 table

Dynamic Memristors: from Devices to Applications

Memristors have been extensively studied as a promising candidate for next generation non-volatile memory technology. More recently, memristors have also become extremely popular in neuromorphic applications because of their striking resemblance to biological synapses. The memristor was firstly proposed conceptually as the fourth basic electric circuit element whose resistance is dependent on the history of electrical stimulation. Physical implementations of memristors are normally based a solid state, nanoscale metal-insulator-metal (MIM) sandwich structure, and the resistance change is achieved by controlling the ion (either cation or anion) redistribution inside the insulating/switching layer. Specifically, a conductive filament can be formed with a high-concentration of metal cations or oxygen vacancies, leading to an increase in device conductance during set, and a decrease in device conductance during reset associated with the annihilation of the filament. Devices based on such resistive switching mechanisms are often termed resistive random-access memory (RRAM) devices, and offer advantages of simple structure, high density, low power, good endurance, etc. for memory and computing applications. In this dissertation, two kinds of memristor devices will be discussed, using Ag2S and WOx as the switching material, respectively. The WOx device allows incremental modulation of the device conductance, and enables efficient hardware emulation of important synaptic learning functions including paired pulse facilitation, sliding threshold effect, rate dependent plasticity and spike timing dependent plasticity (Chapter 3), showing the resemblance between memristors and biological synapses. Neural networks based on the memristor crossbar array have been used to successfully perform image reconstruction tasks based on the sparse coding algorithm (Chapter 2). A 32×32 WOx memristor crossbar array was used for vector-matrix multiplication acceleration, and the device non-ideality effects in the memristor crossbar array on the image reconstruction performance were examined. Additionally, interesting short-term decay dynamics can be observed in both Ag2S and WOx based devices. Different from the requirements of non-volatile memory which aims for long term memory storage, the volatile nature of these devices can be used to directly encode and process temporal information. Specifically, the Ag2S memristor can encode different neural spiking information in the temporal domain into analog switching probability distributions (Chapter 5). These devices are termed “dynamic memristors” and can be applied in novel computing schemes such as reservoir computing systems for efficient temporal information processing including speech recognition (Chapter 4). Both devices show very promising properties for neuromorphic computing – overcoming the von-Neumann bottleneck by incorporating information processing into memory storage. It is believed in the future, very efficient neuromorphic computing chips can be designed and implemented using these memristors that offer potential advantages in terms of area consumption, computing speed and power consumption.PHDElectrical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144102/1/wenma_1.pd