On the validity of memristor modeling in the neural network literature

An analysis of the literature shows that there are two types of non-memristive models that have been widely used in the modeling of so-called "memristive" neural networks. Here, we demonstrate that such models have nothing in common with the concept of memristive elements: they describe either non-linear resistors or certain bi-state systems, which all are devices without memory. Therefore, the results presented in a significant number of publications are at least questionable, if not completely irrelevant to the actual field of memristive neural networks

Hardware design of LIF with Latency neuron model with memristive STDP synapses

In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode more information than the common Integrate-and-Fire models, typically considered for neuromorphic implementations. In our system LIFL neuron is implemented using CMOS circuits while memristor is used for the implementation of the STDP synapse. A description of the entire circuit is provided. Finally, the capabilities of the proposed architecture have been evaluated by simulating a motif composed of three neurons and two synapses. The simulation results confirm the validity of the proposed system and its suitability for the design of more complex spiking neural network

A neural network approach towards generalized resistive switching modelling

Funding: This research was funded by FEDER funds through the COMPETE 2020 Programme and National Funds through FCT—Portuguese Foundation for Science and Technology under project number DFA/BD/8335/2020. Publisher Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland.Resistive switching behaviour has been demonstrated to be a common characteristic to many materials. In this regard, research teams to date have produced a plethora of different devices exhibiting diverse behaviour, but when system design is considered, finding a ‘one-model-fits-all’ solution can be quite difficult, or even impossible. However, it is in the interest of the community to achieve more general modelling tools for design that allows a quick model update as devices evolve. Laying the grounds with such a principle, this paper presents an artificial neural network learning approach to resistive switching modelling. The efficacy of the method is demonstrated firstly with two simulated devices and secondly with a 4 µm2 amorphous IGZO device. For the amorphous IGZO device, a normalized root-mean-squared error (NRMSE) of 5.66 × 10−3 is achieved with a [2, 50, 50, 1] network structure, representing a good balance between model complexity and accuracy. A brief study on the number of hidden layers and neurons and its effect on network performance is also conducted with the best NRMSE reported at 4.63 × 10−3 . The low error rate achieved in both simulated and real-world devices is a good indicator that the presented approach is flexible and can suit multiple device types.publishersversionpublishe

Convergence of Neural Networks with a Class of Real Memristors with Rectifying Characteristics

The paper considers a neural network with a class of real extended memristors obtained via the parallel connection of an ideal memristor and a nonlinear resistor. The resistor has the same rectifying characteristic for the current as that used in relevant models in the literature to account for diode-like effects at the interface between the memristor metal and insulating material. The paper proves some fundamental results on the trajectory convergence of this class of real memristor neural networks under the assumption that the interconnection matrix satisfies some symmetry conditions. First of all, the paper shows that, while in the case of neural networks with ideal memristors, it is possible to explicitly find functions of the state variables that are invariants of motions, the same functions can be used as Lyapunov functions that decrease along the trajectories in the case of real memristors with rectifying characteristics. This fundamental property is then used to study convergence by means of a reduction-of-order technique in combination with a Lyapunov approach. The theoretical predictions are verified via numerical simulations, and the convergence results are illustrated via the applications of real memristor neural networks to the solution of some image processing tasks in real time

VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.Comment: 11 pages, 12 figure

A switching control for finite-time synchronization of memristor-based BAM neural networks with stochastic disturbances

This paper deals with the finite-time stochastic synchronization for a class of memristorbased bidirectional associative memory neural networks (MBAMNNs) with time-varying delays and stochastic disturbances. Firstly, based on the physical property of memristor and the circuit of MBAMNNs, a MBAMNNs model with more reasonable switching conditions is established. Then, based on the theory of Filippov’s solution, by using Lyapunov–Krasovskii functionals and stochastic analysis technique, a sufficient condition is given to ensure the finite-time stochastic synchronization of MBAMNNs with a certain controller. Next, by a further discussion, an errordependent switching controller is given to shorten the stochastic settling time. Finally, numerical simulations are carried out to illustrate the effectiveness of theoretical results

Device Modeling and Circuit Design of Neuromorphic Memory Structures

The downscaling of CMOS technology and the benefits gleaned thereof have made it the cornerstone of the semiconductor industry for many years. As the technology reaches its fundamental physical limits, however, CMOS is expected to run out of steam instigating the exploration of new nanoelectronic devices. Memristors have emerged as promising candidates for future computing paradigms, specifically, memory arrays and neuromorphic circuits. Towards this end, this dissertation will explore the use of two memristive devices, namely, Transition Metal Oxide (TMO) devices and Insulator Metal Transition (IMT) devices in constructing neuromorphic circuits. A compact model for TMO devices is first proposed and verified against experimental data. The proposed model, unlike most of the other models present in the literature, leverages the instantaneous resistance of the device as the state variable which facilitates parameter extraction. In addition, a model for the forming voltage of TMO devices is developed and verified against experimental data and Monte Carlo simulations. Impact of the device geometry and material characteristics of the TMO device on the forming voltage is investigated and techniques for reducing the forming voltage are proposed. The use of TMOs in syanptic arrays is then explored and a multi-driver write scheme is proposed that improves their performance. The proposed technique enhances voltage delivery across the selected cells via suppressing the effective line resistance and leakage current paths, thus, improving the performance of the crossbar array. An IMT compact model is also developed and verified against experiemntal data and electro-thermal device simulations. The proposed model describes the device as a memristive system with the temperature being the state variable, thus, capturing the temperature dependent resistive switching of the IMT device in a compact form suitable for SPICE implementation. An IMT based Integrate-And-Fire neuron is then proposed. The IMT neuron leverages the temperature dynamics of the device to deliver the functionality of the neuron. The proposed IMT neuron is more compact than its CMOS counterparts as it alleviates the need for complex CMOS circuitry. Impact of the IMT device parameters on the neuron\u27s performance is then studied and design considerations are provided

The Effects of Radiation on Memristor-Based Electronic Spiking Neural Networks

In this dissertation, memristor-based spiking neural networks (SNNs) are used to analyze the effect of radiation on the spatio-temporal pattern recognition (STPR) capability of the networks. Two-terminal resistive memory devices (memristors) are used as synapses to manipulate conductivity paths in the network. Spike-timing-dependent plasticity (STDP) learning behavior results in pattern learning and is achieved using biphasic shaped pre- and post-synaptic spikes. A TiO2 based non-linear drift memristor model designed in Verilog-A implements synaptic behavior and is modified to include experimentally observed effects of state-altering, ionizing, and off-state degradation radiation on the device. The impact of neuron “death” (disabled neuron circuits) due to radiation is also examined. In general, radiation interaction events distort the STDP learning curve undesirably, favoring synaptic potentiation. At lower short-term flux, the network is able to recover and relearn the pattern with consistent training, although some pixels may be affected due to stability issues. As the radiation flux and duration increases, it can overwhelm the leaky integrate-and-fire (LIF) post-synaptic neuron circuit, and the network does not learn the pattern. On the other hand, in the absence of the pattern, the radiation effects cumulate, and the system never regains stability. Neuron-death simulation results emphasize the importance of non-participating neurons during the learning process, concluding that non-participating afferents contribute to improving the learning ability of the neural network. Instantaneous neuron death proves to be more detrimental for the network compared to when the afferents die over time thus, retaining the network’s pattern learning capability