CMOS Vision Sensors: Embedding Computer Vision at Imaging Front-Ends

CMOS Image Sensors (CIS) are key for imaging technol-ogies. These chips are conceived for capturing opticalscenes focused on their surface, and for delivering elec-trical images, commonly in digital format. CISs may incor-porate intelligence; however, their smartness basicallyconcerns calibration, error correction and other similartasks. The term CVISs (CMOS VIsion Sensors) definesother class of sensor front-ends which are aimed at per-forming vision tasks right at the focal plane. They havebeen running under names such as computational imagesensors, vision sensors and silicon retinas, among others. CVIS and CISs are similar regarding physical imple-mentation. However, while inputs of both CIS and CVISare images captured by photo-sensors placed at thefocal-plane, CVISs primary outputs may not be imagesbut either image features or even decisions based on thespatial-temporal analysis of the scenes. We may hencestate that CVISs are more “intelligent” than CISs as theyfocus on information instead of on raw data. Actually,CVIS architectures capable of extracting and interpretingthe information contained in images, and prompting reac-tion commands thereof, have been explored for years inacademia, and industrial applications are recently ramp-ing up.One of the challenges of CVISs architects is incorporat-ing computer vision concepts into the design flow. Theendeavor is ambitious because imaging and computervision communities are rather disjoint groups talking dif-ferent languages. The Cellular Nonlinear Network Univer-sal Machine (CNNUM) paradigm, proposed by Profs.Chua and Roska, defined an adequate framework forsuch conciliation as it is particularly well suited for hard-ware-software co-design [1]-[4]. This paper overviewsCVISs chips that were conceived and prototyped at IMSEVision Lab over the past twenty years. Some of them fitthe CNNUM paradigm while others are tangential to it. Allthem employ per-pixel mixed-signal processing circuitryto achieve sensor-processing concurrency in the quest offast operation with reduced energy budget.Junta de Andalucía TIC 2012-2338Ministerio de Economía y Competitividad TEC 2015-66878-C3-1-R y TEC 2015-66878-C3-3-

Trends in Pixel Detectors: Tracking and Imaging

For large scale applications, hybrid pixel detectors, in which sensor and read-out IC are separate entities, constitute the state of the art in pixel detector technology to date. They have been developed and start to be used as tracking detectors and also imaging devices in radiography, autoradiography, protein crystallography and in X-ray astronomy. A number of trends and possibilities for future applications in these fields with improved performance, less material, high read-out speed, large radiation tolerance, and potential off-the-shelf availability have appeared and are momentarily matured. Among them are monolithic or semi-monolithic approaches which do not require complicated hybridization but come as single sensor/IC entities. Most of these are presently still in the development phase waiting to be used as detectors in experiments. The present state in pixel detector development including hybrid and (semi-)monolithic pixel techniques and their suitability for particle detection and for imaging, is reviewed.Comment: 10 pages, 15 figures, Invited Review given at IEEE2003, Portland, Oct, 200

Pixel Detectors

Pixel detectors for precise particle tracking in high energy physics have been developed to a level of maturity during the past decade. Three of the LHC detectors will use vertex detectors close to the interaction point based on the hybrid pixel technology which can be considered the state of the art in this field of instrumentation. A development period of almost 10 years has resulted in pixel detector modules which can stand the extreme rate and timing requirements as well as the very harsh radiation environment at the LHC without severe compromises in performance. From these developments a number of different applications have spun off, most notably for biomedical imaging. Beyond hybrid pixels, a number of monolithic or semi-monolithic developments, which do not require complicated hybridization but come as single sensor/IC entities, have appeared and are currently developed to greater maturity. Most advanced in terms of maturity are so called CMOS active pixels and DEPFET pixels. The present state in the construction of the hybrid pixel detectors for the LHC experiments together with some hybrid pixel detector spin-off is reviewed. In addition, new developments in monolithic or semi-monolithic pixel devices are summarized.Comment: 14 pages, 38 drawings/photographs in 21 figure

Design Of Neural Network Circuit Inside High Speed Camera Using Analog CMOS 0.35 ¼m Technology

Analog VLSI on-chip learning Neural Networks represent a mature technology for a large number of applications involving industrial as well as consumer appliances. This is particularly the case when low power consumption, small size and/or very high speed are required. This approach exploits the computational features of Neural Networks, the implementation efficiency of analog VLSI circuits and the adaptation capabilities of the on-chip learning feedback schema. High-speed video cameras are powerful tools for investigating for instance the biomechanics analysis or the movements of mechanical parts in manufacturing processes. In the past years, the use of CMOS sensors instead of CCDs has enabled the development of high-speed video cameras offering digital outputs , readout flexibility, and lower manufacturing costs. In this paper, we propose a high-speed smart camera based on a CMOS sensor with embedded Analog Neural Network

2D Detectors for Particle Physics and for Imaging Applications

The demands on detectors for particle detection as well as for medical and astronomical X-ray imaging are continuously pushing the development of novel pixel detectors. The state of the art in pixel detector technology to date are hybrid pixel detectors in which sensor and read-out integrated circuits are processed on different substrates and connected via high density interconnect structures. While these detectors are technologically mastered such that large scale particle detectors can be and are being built, the demands for improved performance for the next generation particle detectors ask for the development of monolithic or semi-monolithic approaches. Given the fact that the demands for medical imaging are different in some key aspects, developments for these applications, which started as particle physics spin-off, are becomming rather independent. New approaches are leading to novel signal processing concepts and interconnect technologies to satisfy the need for very high dynamic range and large area detectors. The present state in hybrid and (semi-)monolithic pixel detector development and their different approaches for particle physics and imaging application is reviewed

A fast lightstripe rangefinding system with smart VLSI sensor

The focus of the research is to build a compact, high performance lightstripe rangefinder using a Very Large Scale Integration (VLSI) smart photosensor array. Rangefinding, the measurement of the three-dimensional profile of an object or scene, is a critical component for many robotic applications, and therefore many techniques were developed. Of these, lightstripe rangefinding is one of the most widely used and reliable techniques available. Though practical, the speed of sampling range data by the conventional light stripe technique is severely limited. A conventional light stripe rangefinder operates in a step-and-repeat manner. A stripe source is projected on an object, a video image is acquired, range data is extracted from the image, the stripe is stepped, and the process repeats. Range acquisition is limited by the time needed to grab the video images, increasing linearly with the desired horizontal resolution. During the acquisition of a range image, the objects in the scene being scanned must be stationary. Thus, the long scene sampling time of step-and-repeat rangefinders limits their application. The fast range sensor proposed is based on the modification of this basic lightstripe ranging technique in a manner described by Sato and Kida. This technique does not require a sampling of images at various stripe positions to build a range map. Rather, an entire range image is acquired in parallel while the stripe source is swept continuously across the scene. Total time to acquire the range image data is independent of the range map resolution. The target rangefinding system will acquire 1,000 100 x 100 point range images per second with 0.5 percent range accuracy. It will be compact and rugged enough to be mounted on the end effector of a robot arm to aid in object manipulation and assembly tasks

Neuromorphic analogue VLSI

Neuromorphic systems emulate the organization and function of nervous systems. They are usually composed of analogue electronic circuits that are fabricated in the complementary metal-oxide-semiconductor (CMOS) medium using very large-scale integration (VLSI) technology. However, these neuromorphic systems are not another kind of digital computer in which abstract neural networks are simulated symbolically in terms of their mathematical behavior. Instead, they directly embody, in the physics of their CMOS circuits, analogues of the physical processes that underlie the computations of neural systems. The significance of neuromorphic systems is that they offer a method of exploring neural computation in a medium whose physical behavior is analogous to that of biological nervous systems and that operates in real time irrespective of size. The implications of this approach are both scientific and practical. The study of neuromorphic systems provides a bridge between levels of understanding. For example, it provides a link between the physical processes of neurons and their computational significance. In addition, the synthesis of neuromorphic systems transposes our knowledge of neuroscience into practical devices that can interact directly with the real world in the same way that biological nervous systems do

Image Sensors in Security and Medical Applications

This paper briefly reviews CMOS image sensor technology and its utilization in security and medical applications. The role and future trends of image sensors in each of the applications are discussed. To provide the reader deeper understanding of the technology aspects the paper concentrates on the selected applications such as surveillance, biometrics, capsule endoscopy and artificial retina. The reasons for concentrating on these applications are due to their importance in our daily life and because they present leading-edge applications for imaging systems research and development. In addition, review of image sensors implementation in these applications allows the reader to investigate image sensor technology from the technical and from other views as well