Algorithm and Hardware Co-design for Learning On-a-chip

abstract: Machine learning technology has made a lot of incredible achievements in recent years. It has rivalled or exceeded human performance in many intellectual tasks including image recognition, face detection and the Go game. Many machine learning algorithms require huge amount of computation such as in multiplication of large matrices. As silicon technology has scaled to sub-14nm regime, simply scaling down the device cannot provide enough speed-up any more. New device technologies and system architectures are needed to improve the computing capacity. Designing specific hardware for machine learning is highly in demand. Efforts need to be made on a joint design and optimization of both hardware and algorithm. For machine learning acceleration, traditional SRAM and DRAM based system suffer from low capacity, high latency, and high standby power. Instead, emerging memories, such as Phase Change Random Access Memory (PRAM), Spin-Transfer Torque Magnetic Random Access Memory (STT-MRAM), and Resistive Random Access Memory (RRAM), are promising candidates providing low standby power, high data density, fast access and excellent scalability. This dissertation proposes a hierarchical memory modeling framework and models PRAM and STT-MRAM in four different levels of abstraction. With the proposed models, various simulations are conducted to investigate the performance, optimization, variability, reliability, and scalability. Emerging memory devices such as RRAM can work as a 2-D crosspoint array to speed up the multiplication and accumulation in machine learning algorithms. This dissertation proposes a new parallel programming scheme to achieve in-memory learning with RRAM crosspoint array. The programming circuitry is designed and simulated in TSMC 65nm technology showing 900X speedup for the dictionary learning task compared to the CPU performance. From the algorithm perspective, inspired by the high accuracy and low power of the brain, this dissertation proposes a bio-plausible feedforward inhibition spiking neural network with Spike-Rate-Dependent-Plasticity (SRDP) learning rule. It achieves more than 95% accuracy on the MNIST dataset, which is comparable to the sparse coding algorithm, but requires far fewer number of computations. The role of inhibition in this network is systematically studied and shown to improve the hardware efficiency in learning.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

A Split-Gate Positive Feedback Device With an Integrate-and-Fire Capability for a High-Density Low-Power Neuron Circuit

Hardware-based spiking neural networks (SNNs) to mimic biological neurons have been reported. However, conventional neuron circuits in SNNs have a large area and high power consumption. In this work, a split-gate floating-body positive feedback (PF) device with a charge trapping capability is proposed as a new neuron device that imitates the integrate-and-fire function. Because of the PF characteristic, the subthreshold swing (SS) of the device is less than 0.04 mV/dec. The super-steep SS of the device leads to a low energy consumption of ∼0.25 pJ/spike for a neuron circuit (PF neuron) with the PF device, which is ∼100 times smaller than that of a conventional neuron circuit. The charge storage properties of the device mimic the integrate function of biological neurons without a large membrane capacitor, reducing the PF neuron area by about 17 times compared to that of a conventional neuron. We demonstrate the successful operation of a dense multiple PF neuron system with reset and lateral inhibition using a common self-controller in a neuron layer through simulation. With the multiple PF neuron system and the synapse array, on-line unsupervised pattern learning and recognition are successfully performed to demonstrate the feasibility of our PF device in a neural network

29th Annual Computational Neuroscience Meeting: CNS*2020

Meeting abstracts This publication was funded by OCNS. The Supplement Editors declare that they have no competing interests. Virtual | 18-22 July 202

Dense implementations of binary cellular nonlinear networks : from CMOS to nanotechnology

This thesis deals with the design and hardware realization of the cellular neural/nonlinear network (CNN)-type processors operating on data in the form of black and white (B/W) images. The ultimate goal is to achieve a very compact yet versatile cell structure that would allow for building a network with a very large spatial resolution. It is very important to be able to implement an array with a great number of cells on a single die. Not only it improves the computational power of the processor, but it might be the enabling factor for new applications as well. Larger resolution can be achieved in two ways. First, the cell functionality and operating principles can be tailored to improve the layout compactness. The other option is to use more advanced fabrication technology – either a newer, further downscaled CMOS process or one of the emerging nanotechnologies. It can be beneficial to realize an array processor as two separate parts – one dedicated for gray-scale and the other for B/W image processing, as their designs can be optimized. For instance, an implementation of a CNN dedicated for B/W image processing can be significantly simplified. When working with binary images only, all coefficients in the template matrix can also be reduced to binary values. In this thesis, such a binary programming scheme is presented as a means to reduce the cell size as well as to provide the circuits composed of emerging nanodevices with an efficient programmability. Digital programming can be very fast and robust, and leads to very compact coefficient circuits. A test structure of a binary-programmable CNN has been designed and implemented with standard 0.18 µm CMOS technology. A single cell occupies only 155 µm2, which corresponds to a cell density of 6451 cells per square millimeter. A variety of templates have been tested and the measured chip performance is discussed. Since the minimum feature size of modern CMOS devices has already entered the nanometer scale, and the limitations of further scaling are projected to be reached within the next decade or so, more and more interest and research activity is attracted by nanotechnology. Investigation of the quantum physics phenomena and development of new devices and circuit concepts, which would allow to overcome the CMOS limitations, is becoming an increasingly important science. A single-electron tunneling (SET) transistor is one of the most attractive nanodevices. While relying on the Coulomb interactions, these devices can be connected directly with a wire or through a coupling capacitance. To develop suitable structures for implementing the binary programming scheme with capacitive couplings, the CNN cell based on the floating gate MOSFET (FG-MOSFET) has been designed. This approach can be considered as a step towards a programmable cell implementation with nanodevices. Capacitively coupled CNN has been simulated and the presented results confirm the proper operation. Therefore, the same circuit strategies have also been applied to the CNN cell designed for SET technology. The cell has been simulated to work well with the binary programming scheme applied. This versatile structure can be implemented either as a pure SET design or as a SET-FET hybrid. In addition to the designs mentioned above, a number of promising nanodevices and emerging circuit architectures are introduced.reviewe

Special Topics in Information Technology

This open access book presents thirteen outstanding doctoral dissertations in Information Technology from the Department of Electronics, Information and Bioengineering, Politecnico di Milano, Italy. Information Technology has always been highly interdisciplinary, as many aspects have to be considered in IT systems. The doctoral studies program in IT at Politecnico di Milano emphasizes this interdisciplinary nature, which is becoming more and more important in recent technological advances, in collaborative projects, and in the education of young researchers. Accordingly, the focus of advanced research is on pursuing a rigorous approach to specific research topics starting from a broad background in various areas of Information Technology, especially Computer Science and Engineering, Electronics, Systems and Control, and Telecommunications. Each year, more than 50 PhDs graduate from the program. This book gathers the outcomes of the thirteen best theses defended in 2019-20 and selected for the IT PhD Award. Each of the authors provides a chapter summarizing his/her findings, including an introduction, description of methods, main achievements and future work on the topic. Hence, the book provides a cutting-edge overview of the latest research trends in Information Technology at Politecnico di Milano, presented in an easy-to-read format that will also appeal to non-specialists

Large scale retinal modeling for the design of new generation retinal prostheses

With the help of modern technology, blindness caused by retinal diseases such as age-related macular degeneration or retinitis pigmentosa is now considered reversible. Scientists from various fields such as Neuroscience, Electrical Engineering, Computer Science, and Bioscience have been collaborating to design and develop retinal prostheses, with the aim of replacing malfunctioning parts of the retina and restoring vision in the blind. Human trials conducted to test retinal prostheses have yielded encouraging results, showing the potential of this approach in vision recovery. However, a retinal prosthesis has several limitations with regard to its hardware and biological functions, and several attempts have been made to overcome these limitations. This thesis focuses on the biological aspects of retinal prostheses: the biological processes occurring inside the retina and the limitations of retinal prostheses corresponding to those processes have been analysed. Based on these analyses, three major findings regarding information processing inside the retina have been presented and these findings have been used to conceptualise retinal prostheses that have the characteristics of asymmetrical and separate pathway stimulations. In the future, when nanotechnology gains more popularity and is completely integrated inside the prosthesis, this concept can be utilized to restore useful visual information such as colour, depth, and contrast to achieve high-quality vision in the blind

Neuromorphic Engineering Editors' Pick 2021

This collection showcases well-received spontaneous articles from the past couple of years, which have been specially handpicked by our Chief Editors, Profs. André van Schaik and Bernabé Linares-Barranco. The work presented here highlights the broad diversity of research performed across the section and aims to put a spotlight on the main areas of interest. All research presented here displays strong advances in theory, experiment, and methodology with applications to compelling problems. This collection aims to further support Frontiers’ strong community by recognizing highly deserving authors

Towards Oxide Electronics:a Roadmap

At the end of a rush lasting over half a century, in which CMOS technology has been experiencing a constant and breathtaking increase of device speed and density, Moore's law is approaching the insurmountable barrier given by the ultimate atomic nature of matter. A major challenge for 21st century scientists is finding novel strategies, concepts and materials for replacing silicon-based CMOS semiconductor technologies and guaranteeing a continued and steady technological progress in next decades. Among the materials classes candidate to contribute to this momentous challenge, oxide films and heterostructures are a particularly appealing hunting ground. The vastity, intended in pure chemical terms, of this class of compounds, the complexity of their correlated behaviour, and the wealth of functional properties they display, has already made these systems the subject of choice, worldwide, of a strongly networked, dynamic and interdisciplinary research community. Oxide science and technology has been the target of a wide four-year project, named Towards Oxide-Based Electronics (TO-BE), that has been recently running in Europe and has involved as participants several hundred scientists from 29 EU countries. In this review and perspective paper, published as a final deliverable of the TO-BE Action, the opportunities of oxides as future electronic materials for Information and Communication Technologies ICT and Energy are discussed. The paper is organized as a set of contributions, all selected and ordered as individual building blocks of a wider general scheme. After a brief preface by the editors and an introductory contribution, two sections follow. The first is mainly devoted to providing a perspective on the latest theoretical and experimental methods that are employed to investigate oxides and to produce oxide-based films, heterostructures and devices. In the second, all contributions are dedicated to different specific fields of applications of oxide thin films and heterostructures, in sectors as data storage and computing, optics and plasmonics, magnonics, energy conversion and harvesting, and power electronics