A 10 000 fps CMOS Sensor With Massively Parallel Image Processing

International audienceA high-speed analog VLSI image acquisition and preprocessing system has been designed and fabricated in a 0.35 standard CMOS process. The chip features a massively parallel architecture enabling the computation of programmable low-level image processing in each pixel. Extraction of spatial gradients and convolutions such as Sobel or Laplacian filters are implemented on the circuit. Measured results show that the proposed sensor successfully captures raw images up to 10 000 frames per second and runs low-level image processing at a frame rate of 2000 to 5000 frames per second

A Programmable Vision Chip with High Speed Image Processing

International audienceA high speed Analog VLSI Image acquisition and pre-processing system is described in this paper. A 64×64 pixel retina is used to extract the magnitude and direction of spatial gradients from images. So, the sensor implements some low-level image processing in a massively parallel strategy in each pixel of the sensor. Spatial gradients, various convolutions as Sobel filter or Laplacian are described and implemented on the circuit. The retina implements in a massively parallel way, at pixel level, some various treatments based on a four-quadrants multipliers architecture. Each pixel includes a photodiode, an amplifier, two storage capacitors and an analog arithmetic unit. A maximal output frame rate of about 10000 frames per second with only image acquisition and 2000 to 5000 frames per second with image processing is achieved in a 0.35 μm standard CMOS process. The retina provides address-event coded output on three asynchronous buses, one output is dedicated to the gradient and both other to the pixel values. A prototype based on this principle, has been designed. Simulation results from Mentor GraphicsTMsoftware and AustriaMicrosystem Design kit are presented

ACE16K: The Third Generation of Mixed-Signal SIMD-CNN ACE Chips Toward VSoCs

Today, with 0.18-μm technologies mature and stable enough for mixed-signal design with a large variety of CMOS compatible optical sensors available and with 0.09-μm technologies knocking at the door of designers, we can face the design of integrated systems, instead of just integrated circuits. In fact, significant progress has been made in the last few years toward the realization of vision systems on chips (VSoCs). Such VSoCs are eventually targeted to integrate within a semiconductor substrate the functions of optical sensing, image processing in space and time, high-level processing, and the control of actuators. The consecutive generations of ACE chips define a roadmap toward flexible VSoCs. These chips consist of arrays of mixed-signal processing elements (PEs) which operate in accordance with single instruction multiple data (SIMD) computing architectures and exhibit the functional features of CNN Universal Machines. They have been conceived to cover the early stages of the visual processing path in a fully-parallel manner, and hence more efficiently than DSP-based systems. Across the different generations, different improvements and modifications have been made looking to converge with the newest discoveries of neurobiologists regarding the behavior of natural retinas. This paper presents considerations pertaining to the design of a member of the third generation of ACE chips, namely to the so-called ACE16k chip. This chip, designed in a 0.35-μm standard CMOS technology, contains about 3.75 million transistors and exhibits peak computing figures of 330 GOPS, 3.6 GOPS/mm2 and 82.5 GOPS/W. Each PE in the array contains a reconfigurable computing kernel capable of calculating linear convolutions on 3×3 neighborhoods in less than 1.5 μs, imagewise Boolean combinations in less than 200 ns, imagewise arithmetic operations in about 5 μs, and CNN-like temporal evolutions with a time constant of about 0.5 μs. Unfortunately, the many ideas underlying the design of this chip cannot be covered in a single paper; hence, this paper is focused on, first, placing the ACE16k in the ACE chip roadmap and, then, discussing the most significant modifications of ACE16K versus its predecessors in the family.LOCUST IST2001—38 097VISTA TIC2003—09 817 - C02—01Office of Naval Research N000 140 210 88

A 2-D PROCESSOR ARRAY FOR MASSIVELY PARALLEL IMAGE PROCESSING

The concept of introducing image processing logic within the spatial gaps of an array of photodiodes is the key factor behind the presented work. A two-dimensional massively parallel image processing paradigm based on 8X8 pixel neighborhood digital processors has been designed. A low complexity processor array architecture along with its instruction set has been designed and fully verified on a FPGA platform. Various image processing tests have been run on the FPGA platform to demonstrate the functionality of a design that uses 12 parallel processors. The test results indicate that the architecture is scalable to support high frame rates while allowing for flexible processing due to inherent programmability at a high level. The gate level logic synthesis results of the processor targeting a 0.13 μm CMOS technology indicates a low silicon area complexity, allowing for image sensor integration. Adviser: Dr. Sina Balkir, Co-Adviser: Dr. Michael W. Hoffma

A 1000 FPS at 128×128 vision processor with 8-bit digitized I/O

This paper presents a mixed-signal programmable chip for high-speed vision applications. It consists of an array of processing elements, arranged to operate in accordance with the principles of single instruction multiple data (SIMD) computing architectures. This chip, implemented in a 0.35-μm fully digital CMOS technology, contains ∼ 3.75 M transistors and exhibits peak performance figures of 330 GOPS (8-bit equivalent giga-operations per second), 3.6 GOPS/mm2 and 82.5 GOPS/W. It includes structures for image acquisition and for image processing, meaning that it does not require a separate imager for operation. At the sensory side, integration and log-compression sensing circuits are embedded, thus allowing the chip to handle a large variety of illumination conditions. At the processing plane, analog and digital circuits are employed whose parameters can be programmed and their architecture reconfigured for the realization of software-coded processing algorithms. The chip provides, and accepts, 8-bit digitized data through a 32-bit bidirectional data bus which operates at 120 MB/s. Experimental results show that frame rates of 1000 frames per second (FPS) can be achieved under room illumination conditions; applications using exposures of about 50 μs have been recently reached by using special illumination setups. The chip can capture an image, run approximately 150 two-dimensional linear convolutions, and download the result in 8-bit digital format, in less than 1 ms. This feature, together with the possibility of executing sequences of user-definable instructions (stored on a full-custom 32-kb on-chip memory), and storing intermediate results (up to 8 grayscale images) makes the chip a true general-purpose sensory/processing device.European Commission IST-2001-38097Office of Naval Research (USA) N00014021088

Digital implementation of the cellular sensor-computers

Two different kinds of cellular sensor-processor architectures are used nowadays in various applications. The first is the traditional sensor-processor architecture, where the sensor and the processor arrays are mapped into each other. The second is the foveal architecture, in which a small active fovea is navigating in a large sensor array. This second architecture is introduced and compared here. Both of these architectures can be implemented with analog and digital processor arrays. The efficiency of the different implementation types, depending on the used CMOS technology, is analyzed. It turned out, that the finer the technology is, the better to use digital implementation rather than analog

A Future for Integrated Diagnostic Helping

International audienceMedical systems used for exploration or diagnostic helping impose high applicative constraints such as real time image acquisition and displaying. A large part of computing requirement of these systems is devoted to image processing. This chapter provides clues to transfer consumers computing architecture approaches to the benefit of medical applications. The goal is to obtain fully integrated devices from diagnostic helping to autonomous lab on chip while taking into account medical domain specific constraints.This expertise is structured as follows: the first part analyzes vision based medical applications in order to extract essentials processing blocks and to show the similarities between consumer’s and medical vision based applications. The second part is devoted to the determination of elementary operators which are mostly needed in both domains. Computing capacities that are required by these operators and applications are compared to the state-of-the-art architectures in order to define an efficient algorithm-architecture adequation. Finally this part demonstrates that it's possible to use highly constrained computing architectures designed for consumers handled devices in application to medical domain. This is based on the example of a high definition (HD) video processing architecture designed to be integrated into smart phone or highly embedded components. This expertise paves the way for the industrialisation of intergraded autonomous diagnostichelping devices, by showing the feasibility of such systems. Their future use would also free the medical staff from many logistical constraints due the deployment of today’s cumbersome systems

Low power CMOS vision sensor for foreground segmentation

This thesis focuses on the design of a top-ranked algorithm for background subtraction, the Pixel Adaptive Based Segmenter (PBAS), for its mapping onto a CMOS vision sensor on the focal plane processing. The redesign of PBAS into its hardware oriented version, HO-PBAS, has led to a less number of memories per pixel, along with a simpler overall model, yet, resulting in an acceptable loss of accuracy with respect to its counterpart on CPU. This thesis features two CMOS vision sensors. The first one, HOPBAS1K, has laid out a 24 x 56 pixel array onto a miniasic chip in standard 180 nm CMOS technology. The second one, HOPBAS10K, features an array of 98 x 98 pixels in standard 180 nm CMOS technology too. The second chip fixes some issues found in the first chip, and provides good hardware and background performance metrics

Analysis and design of a wide dynamic range pulse-frequency modulation CMOS image sensor

Complementary Metal-Oxide Semiconductor (CMOS) image sensor is the dominant electronic imaging device in many application fields, including the mobile or portable devices, teleconference cameras, surveillance and medical imaging sensors. Wide dynamic range (WDR) imaging is of interest particular, demonstrating a large-contrast imaging range of the sensor. As of today, different approaches have been presented to provide solutions for this purpose, but there exists various trade-offs among these designs, which limit the number of applications. A pulse-frequency modulation (PFM) pixel offers the possibility to outperform existing designs in WDR imaging applications, however issues such as uniformity and cost have to be carefully handled to make it practical for different purposes. In addition, a complete evaluation of the sensor performance has to be executed prior to fabrication in silicon technology. A thorough investigation of WDR image sensor based on the PFM pixel is performed in this thesis. Starting with the analysis, modeling, and measurements of a PFM pixel, the details of every particular circuit operation are presented. The causes of dynamic range (DR) limitations and signal nonlinearity are identified, and noise measurement is also performed, to guide future design strategies. We present the design of an innovative double-delta compensating (DDC) technique which increases the sensor uniformity as well as DR. This technique achieves performance optimization of the PFM pixel with a minimal cost an improved linearity, and is carefully simulated to demonstrate its feasibility. A quad-sampling technique is also presented with the cooperation of pixel and column circuits to generate a WDR image sensor with a reduced cost for the pixel. This method, which is verified through the field-programmable gate array (FPGA) implementation, saves considerable area in the pixel and employs the maximal DR that a PFM pixel provides. A complete WDR image sensor structure is proposed to evaluate the performance and feasibility of fabrication in silicon technology. The plans of future work and possible improvements are also presented