Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective

On metrics of density and power efficiency, neuromorphic technologies have the potential to surpass mainstream computing technologies in tasks where real-time functionality, adaptability, and autonomy are essential. While algorithmic advances in neuromorphic computing are proceeding successfully, the potential of memristors to improve neuromorphic computing have not yet born fruit, primarily because they are often used as a drop-in replacement to conventional memory. However, interdisciplinary approaches anchored in machine learning theory suggest that multifactor plasticity rules matching neural and synaptic dynamics to the device capabilities can take better advantage of memristor dynamics and its stochasticity. Furthermore, such plasticity rules generally show much higher performance than that of classical Spike Time Dependent Plasticity (STDP) rules. This chapter reviews the recent development in learning with spiking neural network models and their possible implementation with memristor-based hardware

Committee Machines—A Universal Method to Deal with Non-Idealities in Memristor-Based Neural Networks

Arti ficial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artifi cial neural networks in hardware by implementing their synaptic weights using memristive devices. However, various device- and system-level non-idealities usually prevent these physical implementations from achieving high inference accuracy. We suggest applying a well-known concept in computer science|committee machines|in the context of memristor-based neural networks. Using simulations and experimental data from three different types of memristive devices, we show that committee machines employing ensemble averaging can successfully increase inference accuracy in physically implemented neural networks that suffer from faulty devices, device-to-device variability, random telegraph noise and line resistance. Importantly, we demonstrate that the accuracy can be improved even without increasing the total number of memristors

Nanoscale Memristor: Great potential for memory and synapse emulator for computing applications

This work reports the fabrication and electrical characterization of Metal-Insulator-Metal (MIM) devices for neuromorphic applications using zinc-tin-oxide (ZTO) and indium-gallium-zinc-oxide (IGZO) as the switching layers and molybdenum (Mo) for the devices ‘contacts. A lithographic mask was used along with physical vapor deposition (PVD) processes for the production of the different samples’ layers. Using ZTO as a switching layer in order to replace other elements that are becoming scarce such as indium or gallium is of relevant importance, therefore it was first attempted a ZTO based MIM device. Upon electrical characterization the ZTO devices show an analog behavior without the need of current compliance (being therefore self-limited), good multilevel storage property, reliability and a stable state retention for long periods of time. It is suspected a 2D type of switching mechanism, based on the tunneling through a Schottky barrier at the interface, however the details of the exact mechanism aren’t yet clear. Furthermore, the device is highly prone to interact with humidity present in the atmosphere and some fabrication steps, which is a possible explanation for the anticlockwise RESET. A second batch of ZTO devices was fabricated in order to remediate the RESET process, using a passivation step, however the RESET direction wasn’t affected although the rectification properties of the devices were enhanced. Since upon pulse testing the ZTO devices behaved erratically, this switching layer was discarded and IGZO used instead. With this alternative amorphous oxide semiconductor material, the symmetry and linearity of the conductance change was evaluated and transition from STP (Short-Term Potentiation) to LTP (Long-Term Potentiation) successfully demonstrated upon pulse repetition, showing similar decay fashion to human memory, following a Kohlrausch-Williams-Watts function (commonly called “stretched-exponential function”)

In-memory computing with emerging memory devices: Status and outlook

Supporting data for "In-memory computing with emerging memory devices: status and outlook", submitted to APL Machine Learning

Cerium oxide based resistive random access memory devices

Resistive Random Access Memory (RRAM) is an emerging technology of non-volatile memory (NVM). Although the observation of metal oxide that can undergo an abrupt insulator-metal transition into a conductive state has been known for over 40 years, researchers started investigating those materials for memory applications in late 1990s. It has been considered as the next generation memory technology to replace current flash memory because RRAM has demonstrated feasible switching characteristics and potential to build high density arrays and also RRAM is also compatible with contemporary CMOS processes, which means RRAM can be integrated into current CMOS chips. While the structure of RRAM is a simple metal-insulator-metal (MIM) device, there are numerous materials that exhibit resistive switching. The switching behavior is not only dependent on the switching layer materials but also dependent on the choice of metal electrodes and their interfacial properties. Many metal oxides such as hafnium oxide, titanium oxide, aluminum oxide, nickel oxide (NiO), tantalum oxide and etc. have been studied in details; however, some materials are unexplored such as cerium oxide. In addition to nonvolatile storage applications, RRAM is considered as one of essential elements for advancing neuromorphic computing because of its analog switching and retention characteristics. This thesis investigated CeO[subscript x]-based RRAMs, from its fundamental device characteristics to neuromorphic applications.Electrical and Computer Engineerin

Assessment and Improvement of the Pattern Recognition Performance of Memdiode-Based Cross-Point Arrays with Randomly Distributed Stuck-at-Faults

In this work, the effect of randomly distributed stuck-at faults (SAFs) in memristive crosspoint array (CPA)-based single and multi-layer perceptrons (SLPs and MLPs, respectively) intended for pattern recognition tasks is investigated by means of realistic SPICE simulations. The quasi-static memdiode model (QMM) is considered here for the modelling of the synaptic weights implemented with memristors. Following the standard memristive approach, the QMM comprises two coupled equations, one for the electron transport based on the double-diode equation with a single series resistance and a second equation for the internal memory state of the device based on the so-called logistic hysteron. By modifying the state parameter in the current-voltage characteristic, SAFs of different severeness are simulated and the final outcome is analysed. Supervised ex-situ training and two well-known image datasets involving hand-written digits and human faces are employed to assess the inference accuracy of the SLP as a function of the faulty device ratio. The roles played by the memristor’s electrical parameters, line resistance, mapping strategy, image pixelation, and fault type (stuck-at-ON or stuck-at-OFF) on the CPA performance are statistically analysed following a Monte-Carlo approach. Three different re-mapping schemes to help mitigate the effect of the SAFs in the SLP inference phase are thoroughly investigated.In this work, the effect of randomly distributed stuck-at faults (SAFs) in memristive cross-point array (CPA)-based single and multi-layer perceptrons (SLPs and MLPs, respectively) intended for pattern recognition tasks is investigated by means of realistic SPICE simulations. The quasi-static memdiode model (QMM) is considered here for the modelling of the synaptic weights implemented with memristors. Following the standard memristive approach, the QMM comprises two coupled equations, one for the electron transport based on the double-diode equation with a single series resistance and a second equation for the internal memory state of the device based on the so-called logistic hysteron. By modifying the state parameter in the current-voltage characteristic, SAFs of different severeness are simulated and the final outcome is analysed. Supervised ex-situ training and two well-known image datasets involving hand-written digits and human faces are employed to assess the inference accuracy of the SLP as a function of the faulty device ratio. The roles played by the memristor?s electrical parameters, line resistance, mapping strategy, image pixelation, and fault type (stuck-at-ON or stuck-at-OFF) on the CPA performance are statistically analysed following a Monte-Carlo approach. Three different re-mapping schemes to help mitigate the effect of the SAFs in the SLP inference phase are thoroughly investigated.Fil: Aguirre, Fernando Leonel. Universidad Tecnológica Nacional. Facultad Regional Buenos Aires. Unidad de Investigación y Desarrollo de las Ingenierías; Argentina. Universitat Autònoma de Barcelona; España. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Pazos, Sebastián Matías. Universidad Tecnológica Nacional. Facultad Regional Buenos Aires. Unidad de Investigación y Desarrollo de las Ingenierías; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Palumbo, Félix Roberto Mario. Universidad Tecnológica Nacional. Facultad Regional Buenos Aires. Unidad de Investigación y Desarrollo de las Ingenierías; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Morell, Antoni. Universitat Autònoma de Barcelona; EspañaFil: Suñé, Jordi. Universitat Autònoma de Barcelona; EspañaFil: Miranda, Enrique. Universitat Autònoma de Barcelona; Españ