Long-Term Memory for Cognitive Architectures: A Hardware Approach Using Resistive Devices

A cognitive agent capable of reliably performing complex tasks over a long time will acquire a large store of knowledge. To interact with changing circumstances, the agent will need to quickly search and retrieve knowledge relevant to its current context. Real time knowledge search and cognitive processing like this is a challenge for conventional computers, which are not optimised for such tasks. This thesis describes a new content-addressable memory, based on resistive devices, that can perform massively parallel knowledge search in the memory array. The fundamental circuit block that supports this capability is a memory cell that closely couples comparison logic with non-volatile storage. By using resistive devices instead of transistors in both the comparison circuit and storage elements, this cell improves area density by over an order of magnitude compared to state of the art CMOS implementations. The resulting memory does not need power to maintain stored information, and is therefore well suited to cognitive agents with large long-term memories. The memory incorporates activation circuits, which bias the knowledge retrieval process according to past memory access patterns. This is achieved by approximating the widely used base-level activation function using resistive devices to store, maintain and compare activation values. By distributing an instance of this circuit to every row in memory, the activation for all memory objects can be updated in parallel. A test using the word sense disambiguation task shows this circuit-based activation model only incurs a small loss in accuracy compared to exact base-level calculations. A variation of spreading activation can also be achieved in-memory. Memory objects are encoded with high-dimensional vectors that create association between correlated representations. By storing these high-dimensional vectors in the new content-addressable memory, activation can be spread to related objects during search operations. The new memory is scalable, power and area efficient, and performs operations in parallel that are infeasible in real-time for a sequential processor with a conventional memory hierarchy.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201

Memcapacitor and Meminductor Circuit Emulators: A Review

This research was funded by the Japanese KAKENHI through Grant Number JP18k04275 and Spanish Ministry of Education, Culture, and Sport (MECD), through Project TEC2017-89955-P and Grant Numbers: FPU16/01451 and FPU16/04043.In 1971, Prof. L. Chua theoretically introduced a new circuit element, which exhibited a different behavior from that displayed by any of the three known passive elements: the resistor, the capacitor or the inductor. This element was called memristor, since its behavior corresponded to a resistor with memory. Four decades later, the concept of mem-elements was extended to the other two circuit elements by the definition of the constitutive equations of both memcapacitors and meminductors. Since then, the non-linear and non-volatile properties of these devices have attracted the interest of many researches trying to develop a wide range of applications. However, the lack of solid-state implementations of memcapacitors and meminductors make it necessary to rely on circuit emulators for the use and investigation of these elements in practical implementations. On this basis, this review gathers the current main alternatives presented in the literature for the emulation of both memcapacitors and meminductors. Different circuit emulators have been thoroughly analyzed and compared in detail, providing a wide range of approaches that could be considered for the implementation of these devices in future designs.Ministry of Education, Culture, Sports, Science and Technology, Japan (MEXT) Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (KAKENHI) JP18k04275Spanish Ministry of Education, Culture, and Sport (MECD) TEC2017-89955-P FPU16/01451 FPU16/0404

Analog Computing using 1T1R Crossbar Arrays

Memristor is a novel passive electronic device and a promising candidate for new generation non-volatile memory and analog computing. Analog computing based on memristors has been explored in this study. Due to the lack of commercial electrical testing instruments for those emerging devices and crossbar arrays, we have designed and built testing circuits to implement analog and parallel computing operations. With the setup developed in this study, we have successfully demonstrated image processing functions utilizing large memristor crossbar arrays. We further designed and experimentally demonstrated the first memristor based field programmable analog array (FPAA), which was successfully configured for audio equalizer and frequency classifier demonstration as exemplary applications of such memristive FPAA (memFPAA)

Towards Data Reliable, Low-Power, and Repairable Resistive Random Access Memories

A series of breakthroughs in memristive devices have demonstrated the potential of memristor arrays to serve as next generation resistive random access memories (ReRAM), which are fast, low-power, ultra-dense, and non-volatile. However, memristors' unique device characteristics also make them prone to several sources of error. Owing to the stochastic filamentary nature of memristive devices, various recoverable errors can affect the data reliability of a ReRAM. Permanent device failures further limit the lifetime of a ReRAM. This dissertation developed low-power solutions for more reliable and longer-enduring ReRAM systems. In this thesis, we first look into a data reliability issue known as write disturbance. Writing into a memristor in a crossbar could disturb the stored values in other memristors that are on the same memory line as the target cell. Such disturbance is accumulative over time which may lead to complete data corruption. To address this problem, we propose the use of two regular memristors on each word to keep track of the disturbance accumulation and trigger a refresh to restore the weakened data, once it becomes necessary. We also investigate the considerable variation in the write-time characteristics of individual memristors. With such variation, conventional fixed-pulse write schemes not only waste significant energy, but also cannot guarantee reliable completion of the write operations. We address such variation by proposing an adaptive write scheme that adjusts the width of the write pulses for each memristor. Our scheme embeds an online monitor to detect the completion of a write operation and takes into account the parasitic effect of line-shared devices in access-transistor-free memristive arrays. We further investigate the use of this method to shorten the test time of memory march algorithms by eliminating the need of a verifying read right after a write, which is commonly employed in the test sequences of march algorithms.Finally, we propose a novel mechanism to extend the lifetime of a ReRAM by protecting it against hard errors through the exploitation of a unique feature of bipolar memristive devices. Our solution proposes an unorthodox use of complementary resistive switches (a particular implementation of memristive devices) to provide an ``in-place spare'' for each memory cell at negligible extra cost. The in-place spares are then utilized by a repair scheme to repair memristive devices that have failed at a stuck-at-ON state at a page-level granularity. Furthermore, we explore the use of in-place spares in lieu of other memory reliability and yield enhancement solutions, such as error correction codes (ECC) and spare rows. We demonstrate that with the in-place spares, we can yield the same lifetime as a baseline ReRAM with either significantly fewer spare rows or a lighter-weight ECC, both of which can save on energy consumption and area

Low Power Memory/Memristor Devices and Systems

This reprint focusses on achieving low-power computation using memristive devices. The topic was designed as a convenient reference point: it contains a mix of techniques starting from the fundamental manufacturing of memristive devices all the way to applications such as physically unclonable functions, and also covers perspectives on, e.g., in-memory computing, which is inextricably linked with emerging memory devices such as memristors. Finally, the reprint contains a few articles representing how other communities (from typical CMOS design to photonics) are fighting on their own fronts in the quest towards low-power computation, as a comparison with the memristor literature. We hope that readers will enjoy discovering the articles within

Searching for the physical nature of intelligence in Neuromorphic Nanowire Networks

The brain’s unique information processing efficiency has inspired the development of neuromorphic, or brain-inspired, hardware in effort to reduce the power consumption of conventional Artificial Intelligence (AI). One example of a neuromorphic system is nanowire networks (NWNs). NWNs have been shown to produce conductance pathways similar to neuro-synaptic pathways in the brain, demonstrating nonlinear dynamics, as well as emergent behaviours such as memory and learning. Their synapse-like electro-chemical junctions are connected by a heterogenous neural network-like structure. This makes NWNs a unique system for realising hardware-based machine intelligence that is potentially more brain-like than existing implementations of AI. Much of the brain’s emergent properties are thought to arise from a unique structure-function relationship. The first part of the thesis establishes structural network characterisation methods in NWNs. Borrowing techniques from neuroscience, a toolkit is introduced for characterising network topology in NWNs. NWNs are found to display a ‘small-world’ structure with highly modular connections, like simple biological systems. Next, investigation of the structure-function link in NWNs occurs via implementation of machine learning benchmark tasks on varying network structures. Highly modular networks exhibit an ability to multitask, while integrated networks suffer from crosstalk interference. Finally, above findings are combined to develop and implement neuroscience-inspired learning methods and tasks in NWNs. Specifically, an adaptation of a cognitive task that tests working memory in humans is implemented. Working memory and memory consolidation are demonstrated and found to be attributable to a process similar to synaptic metaplasticity in the brain. The results of this thesis have created new research directions that warrant further exploration to test the universality of the physical nature of intelligence in inorganic systems beyond NWNs

Evolving Nano-scale Associative Memories with Memristors

Associative Memories (AMs) are essential building blocks for brain-like intelligent computing with applications in artificial vision, speech recognition, artificial intelligence, and robotics. Computations for such applications typically rely on spatial and temporal associations in the input patterns and need to be robust against noise and incomplete patterns. The conventional method for implementing AMs is through Artificial Neural Networks (ANNs). Improving the density of ANN based on conventional circuit elements poses a challenge as devices reach their physical scalability limits. Furthermore, stored information in AMs is vulnerable to destructive input signals. Novel nano-scale components, such as memristors, represent one solution to the density problem. Memristors are non-linear time-dependent circuit elements with an inherently small form factor. However, novel neuromorphic circuits typically use memristors to replace synapses in conventional ANN circuits. This sub-optimal use is primarily because there is no established design methodology to exploit the memristor\u27s non-linear properties in a more encompassing way. The objective of this thesis is to explore denser and more robust AM designs using memristor networks. We hypothesize that such network AMs will be more area-efficient than the traditional ANN designs if we can use the memristor\u27s non-linear property for spatial and time-dependent temporal association. We have built a comprehensive simulation framework that employs Genetic Programming (GP) to evolve AM circuits with memristors. The framework is based on the ParadisEO metaheuristics API and uses ngspice for the circuit evaluation. Our results show that we can evolve efficient memristor-based networks that have the potential to replace conventional ANNs used for AMs. We obtained AMs that a) can learn spatial and temporal correlation in the input patterns; b) optimize the trade-off between the size and the accuracy of the circuits; and c) are robust against destructive noise in the inputs. This robustness was achieved at the expense of additional components in the network. We have shown that automated circuit discovery is a promising tool for memristor-based circuits. Future work will focus on evolving circuits that can be used as a building block for more complicated intelligent computing architectures

An organic memristor as the building block for bio-inspired adaptive networks

This thesis reports the research path I followed during my PhD course, which i followed from January 2008 to December 2010 working at the University of Parma, in the Laboratory of Molecular Nanotechnologies, under the supervision of Prof. Marco P. Fontana and Dr. Victor Erokhin, within the framework of an interdisciplinary, international research project called BION – Biologically inspired Organized Networks. The keystone of my research is an organic memristor, a two terminal polymeric electronic device recently developed in our research group at the university of Parma. A memristor is a passive electronic device in which the electrical resistance depends on the electrical charge that has passed through it, and hence is adjustable by applying the appropriate electric potential or sequence of potentials. As of the beginning of my PhD, the device was in its early characterization stages, but it was already clear that it could be used to mimic the kind of plasticity found in synapses within neuronal circuits. In the thesis I show some further characterization work, used for engineering the device to maximize its more useful characteristics and to deepen our understanding of the functioning of the device, as well as the work done on. The knowledge of computational neuroscience acquired during this side project has proved very useful to better coordinate research in the material science side of the project, whose ultimate goal is the realization of a new, highly innovative technology for the production of functional molecular assemblies that can perform advanced tasks of information processing, involving learning and decision making, and that can be tailored down to the nanoscale.Questa tesi riporta il percorso di ricerca seguito durante il mio dottorato di ricerca, che ho svolto da gennaio 2008 a dicembre 2010 lavorando nel Laboratorio di Nanotecnologie Molecolari, presso l'Università di Parma, , sotto la supervisione del Prof. Marco P. Fontana e del Dott. Victor Erokhin, nel quadro di un approccio interdisciplinare, progetto di ricerca internazionale denominato BION - Biologically ispired Organized Networks . La chiave di svolta della mia ricerca è un memristor organico, un dispositivo a due terminali elettronici polimerici recentemente messo a punto nel nostro gruppo di ricerca presso l'università di Parma. Un memristor è un dispositivo elettronico passivo in cui la resistenza elettrica dipende dalla carica elettrica che è passata attraverso di essa, e quindi è regolabile applicando il potenziale elettrico appropriato o una sequenza di potenziali. A partire dall'inizio del mio dottorato di ricerca, il dispositivo è stato nelle sue fasi di caratterizzazione iniziale, ma era già chiaro che poteva essere usata per simulare il tipo di plasticità trovato in sinapsi all'interno di circuiti neuronali. Nella tesi ho mostrato un ulteriore lavoro di caratterizzazione, utilizzato per l'ingegneria del dispositivo al fine di massimizzare le sue caratteristiche più utili e di approfondire la nostra comprensione del funzionamento del dispositivo, così come il lavoro svolto. La conoscenza delle neuroscienze computazionali acquisite nel corso di questo progetto parallelo si è rivelato molto utile per meglio coordinare la ricerca per quanto riguarda il materiale scientifico del progetto, il cui scopo ultimo è la realizzazione di una nuova tecnologia altamente innovativa per la produzione di composti molecolari funzionali in grado di eseguire attività avanzate di elaborazione delle informazioni, che coinvolgano l'apprendimento e il processo decisionale, e che può essere adattata fino alla scala nanometrica