Neuromorphic-P2M: Processing-in-Pixel-in-Memory Paradigm for Neuromorphic Image Sensors

Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources. Hence, researchers have been exploring different energy-efficient solutions such as near-sensor processing, in-sensor processing, and in-pixel processing, bringing the computation closer to the sensor. In particular, in-pixel processing embeds the computation capabilities inside the pixel array and achieves high energy efficiency by generating low-level features instead of the raw data stream from CMOS image sensors. Many different in-pixel processing techniques and approaches have been demonstrated on conventional frame-based CMOS imagers, however, the processing-in-pixel approach for neuromorphic vision sensors has not been explored so far. In this work, we for the first time, propose an asynchronous non-von-Neumann analog processing-in-pixel paradigm to perform convolution operations by integrating in-situ multi-bit multi-channel convolution inside the pixel array performing analog multiply and accumulate (MAC) operations that consume significantly less energy than their digital MAC alternative. To make this approach viable, we incorporate the circuit's non-ideality, leakage, and process variations into a novel hardware-algorithm co-design framework that leverages extensive HSpice simulations of our proposed circuit using the GF22nm FD-SOI technology node. We verified our framework on state-of-the-art neuromorphic vision sensor datasets and show that our solution consumes ~2x lower backend-processor energy while maintaining almost similar front-end (sensor) energy on the IBM DVS128-Gesture dataset than the state-of-the-art while maintaining a high test accuracy of 88.36%.Comment: 17 pages, 11 figures, 2 table

Image compression and energy harvesting for energy constrained sensors

Title from PDF of title page, viewed on June 21, 2013Dissertation advisor: Walter D. Leon-SalasVitaIncludes bibliographic references (pages 176-[187])Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2013The advances in complementary metal-oxide-semiconductor (CMOS) technology have led to the integration of all components of electronic system into a single integrated circuit. Ultra-low power circuit techniques have reduced the power consumption of circuits. Moreover, solar cells with improved efficiency can be integrated on chip to harvest energy from sunlight. As a result of all the above, a new class of miniaturized electronic systems known as self-powered system on a chip has emerged. There is an increasing research interest in the area of self-powered devices which provide cost-effective solutions especially when these devices are used in the areas that changing or replacing batteries is too costly. Therefore, image compression and energy harvesting are studied in this dissertation. The integration of energy harvesting, image compression, and an image sensor on the same chip provides the energy source to charge a battery, reduces the data rate, and improves the performance of wireless image sensors. Integrated circuits of image compression, solar energy harvesting, and image sensors are studied, designed, and analyzed in this work. In this dissertation, a hybrid image sensor that can perform the tasks of sensing and energy harvesting is presented. Photodiodes of hybrid image sensor can be programmed as image sensors or energy harvesting cells. The hybrid image sensor can harvest energy in between frames, in sleep mode, and even when it is taking images. When sensing images and harvesting energy are both needed at the same time, some pixels have to work as sensing pixels, and the others have to work as solar cells. Since some pixels are devoted to harvest energy, the resolution of the image will be reduced. To preserve the resolution or to keep the fair resolution when a lot of energy collection is needed, image reconstruction algorithms and compressive sensing theory provide solutions to achieve a good image quality. On the other hand, when the battery has enough charge, image compression comes into the picture. Multiresolution decomposition image compression provides a way to compress image data in order to reduce the energy need from data transmission. The solution provided in this dissertation not only harvests energy but also saves energy resulting long lasting wireless sensors. The problem was first studied at the system level to identify the best system-level configuration which was then implemented on silicon. As a proof of concept, a 32 x 32 array of hybrid image sensor, a 32 x 32 array of image sensor with multiresolution decomposition compression, and a compressive sensing converter have been designed and fabricated in a standard 0.5 [micrometer] CMOS process. Printed circuit broads also have been designed to test and verify the proposed and fabricated chips. VHDL and Matlab codes were written to generate the proper signals to control, and read out data from chips. Image processing and recovery were carried out in Matlab. DC-DC converters were designed to boost the inherently low voltage output of the photodiodes. The DC-DC converter has also been improved to increase the efficiency of power transformation.Introduction -- Hybrid imager system and circuit design -- Hybrid imager energy harvesting and image acquisition results and discussion -- Detailed description and mathematical analysis for a circuit of energy harvesting using on-chip solar cells -- Multiresolution decomposition for lossless and near-lossless compression -- An incremental [sigma-delta] converter for compressive sensing -- Detailed description of a sigma-delta random demodulator converter architecture for compressive sensing applications -- Conclusion -- Appendix A. Chip pin-out -- Appendix B. Schematics -- Appendix C. Pictures of custom PC

A Bio-Inspired Vision Sensor With Dual Operation and Readout Modes

This paper presents a novel event-based vision sensor with two operation modes: intensity mode and spatial contrast detection. They can be combined with two different readout approaches: pulse density modulation and time-to-first spike. The sensor is conceived to be a node of an smart camera network made up of several independent an autonomous nodes that send information to a central one. The user can toggle the operation and the readout modes with two control bits. The sensor has low latency (below 1 ms under average illumination conditions), low power consumption (19 mA), and reduced data flow, when detecting spatial contrast. A new approach to compute the spatial contrast based on inter-pixel event communication less prone to mismatch effects than diffusive networks is proposed. The sensor was fabricated in the standard AMS4M2P 0.35-um process. A detailed system-level description and experimental results are provided.Office of Naval Research (USA) N00014-14-1-0355Ministerio de Economía y Competitividad TEC2012- 38921-C02-02, P12-TIC-2338, IPT-2011-1625-43000

Efficient Neuromorphic Computing Enabled by Spin-Transfer Torque: Devices, Circuits and Systems

Present day computers expend orders of magnitude more computational resources to perform various cognitive and perception related tasks that humans routinely perform everyday. This has recently resulted in a seismic shift in the field of computation where research efforts are being directed to develop a neurocomputer that attempts to mimic the human brain by nanoelectronic components and thereby harness its efficiency in recognition problems. Bridging the gap between neuroscience and nanoelectronics, this thesis demonstrates the encoding of biological neural and synaptic functionalities in the underlying physics of electron spin. Description of various spin-transfer torque mechanisms that can be potentially utilized for realizing neuro-mimetic device structures is provided. A cross-layer perspective extending from the device to the circuit and system level is presented to envision the design of an All-Spin neuromorphic processor enabled with on-chip learning functionalities. Device-circuit-algorithm co-simulation framework calibrated to experimental results suggest that such All-Spin neuromorphic systems can potentially achieve almost two orders of magnitude energy improvement in comparison to state-of-the-art CMOS implementations

Design of CMOS Digital Silicon Photomultipliers with ToF for Positron Emission Tomography

This thesis presents a contribution to the design of single-photon detectors for medical imaging. Specifically, the focus has been on the development of a pixel capable of single-photon counting in CMOS technology, and the associated sensor thereof. These sensors can work under low light conditions and provide timing information to determine the time-stamp of the incoming photons. For instance, this is particularly attractive for applications that rely either on time-of-flight measurements or on exponential decay determination of the light source, like positron emission tomography or fluorescence-lifetime imaging, respectively. This thesis proposes the study of the pixel architecture to optimize its performance in terms of sensitivity, linearity and signal to noise ratio. The design of the pixel has followed a bottom-up approach, taking care of the smallest building block and studying how the different architecture choices affect performance. Among the various building blocks needed, special emphasis has been placed on the following: • the Single-Photon Avalanche Diode (SPAD), a photodiode able to detect photons one by one; • the front-end circuitry of this diode, commonly called quenching and recharge circuit; • the Time-to-Digital Converter (TDC), which determines the timing performance of the pixel. The proposed architectural exploration provides a comprehensive insight into the design space of the pixel, allowing to determine the optimum design points in terms of sensor sensitivity, linearity or signal to noise ratio, thus helping designers to navigate through non-straightforward trade-offs. The proposed TDC is based on a voltage-controlled ring oscillator, since this architecture provides moderate time resolutions while keeping the footprint, the power, and conversion time relatively small. Two pseudo-differential delay stages have been studied, one with cross-coupled PMOS transistors and the other with cross-coupled inverters. Analytical studies and simulations have shown that cross-coupled inverters are the most appropriate to implement the TDC because they achieve better time resolution with smaller energy per conversion than cross-coupled PMOS transistor stages. A 1.3×1.3 mm2 pixel has been implemented in an 110 nm CMOS image sensor technology, to have the benefits of sub-micron technologies along with the cleanliness of CMOS image sensor technologies. The fabricated chips have been used to characterize the single-photon avalanche diodes. The results agree with expectations: a maximum photon detection probability of 46 % and a median dark count rate of 0.4 Hz/µm2 with an excess voltage of 3 V. Furthermore, the characterization of the TDC shows that the time resolution is below 100 ps, which agrees with post-layout simulations. The differential non-linearity is ±0.4LSB, and the integral non-linearity is ±6.1LSB. Photoemission occurs during characterization - an indication that the avalanches are not quenched properly. The cause of this has been identified to be in the design of the SPAD and the quenching circuit. SPADs are sensitive devices which maximum reverse current must be well defined and limited by the quenching circuit, otherwise unwanted effects like excessive cross-talk, noise, and power consumption may happen. Although this issue limits the operation of the implemented pixel, the information obtained during the characterization will help to avoid mistakes in future implementations