Advances on CMOS image sensors

This paper offers an introduction to the technological advances of image sensors designed using complementary metal–oxide–semiconductor (CMOS) processes along the last decades. We review some of those technological advances and examine potential disruptive growth directions for CMOS image sensors and proposed ways to achieve them. Those advances include breakthroughs on image quality such as resolution, capture speed, light sensitivity and color detection and advances on the computational imaging. The current trend is to push the innovation efforts even further as the market requires higher resolution, higher speed, lower power consumption and, mainly, lower cost sensors. Although CMOS image sensors are currently used in several different applications from consumer to defense to medical diagnosis, product differentiation is becoming both a requirement and a difficult goal for any image sensor manufacturer. The unique properties of CMOS process allows the integration of several signal processing techniques and are driving the impressive advancement of the computational imaging. With this paper, we offer a very comprehensive review of methods, techniques, designs and fabrication of CMOS image sensors that have impacted or might will impact the images sensor applications and markets

Single-Slope ADC with Embedded Convolution Filter for Global-Shutter CMOS Image Sensors

© 2023 IEEE. This is the accepted manuscript version of an article which has been published in final form at https://doi.org/10.1109/TCSII.2023.3266714This paper presents an analog-to-digital converter (ADC) suitable for acquisition and processing of images in the global-shutter mode at the pixel level. The ADC consists of an analog comparator, a multi-directional shift register for the comparator states, and a 16-bit reversible binary counter with programmable step size. It works in the traditional single-slope mode. The novelty is that during each step of the reference ramp, neighboring pixels can exchange status information. During the conversion, the direction and step size of the counter are set globally to realize the corresponding coefficient of a convolution kernel. This technique does not slow down the conversion when used for small kernels (3W3) and does not significantly increase sensor noise. Convolution windows of arbitrary size can be implemented. The concept was verified in an experimental 64W64 imaging array implemented in 180 nm CMOS technology.Peer reviewe

Optical Camera Communications: Principles, Modulations, Potential and Challenges

Optical wireless communications (OWC) are emerging as cost-effective and practical solutions to the congested radio frequency-based wireless technologies. As part of OWC, optical camera communications (OCC) have become very attractive, considering recent developments in cameras and the use of fitted cameras in smart devices. OCC together with visible light communications (VLC) is considered within the framework of the IEEE 802.15.7m standardization. OCCs based on both organic and inorganic light sources as well as cameras are being considered for low-rate transmissions and localization in indoor as well as outdoor short-range applications and within the framework of the IEEE 802.15.7m standardization together with VLC. This paper introduces the underlying principles of OCC and gives a comprehensive overview of this emerging technology with recent standardization activities in OCC. It also outlines the key technical issues such as mobility, coverage, interference, performance enhancement, etc. Future research directions and open issues are also presented

DESIGN OF A BURST MODE ULTRA HIGH-SPEED LOW-NOISE CMOS IMAGE SENSOR

Ultra-high-speed (UHS) image sensors are of interest for studying fast scientific phenomena and may also be useful in medicine. Several published studies have recently achieved frame rates of up to millions of frames per second (Mfps) using advanced processes and/or customized processes. This thesis presents a burst-mode (108 frames) UHS low-noise CMOS image sensor (CIS) based on charge-sweep transfer gates in an unmodified, standard 180 nm front-side-illuminated CIS process. By optimizing the photodiode geometry, the 52.8 μm pitch pixels with 20x20 μm^2 of active area, achieve a charge-transfer time of less than 10 ns. A proof-of-concept CIS was designed and fabricated. Through characterization, it is shown that the designed CIS has the potential to achieve 20 Mfps with an input-referred noise of 5.1 e− rms

A high speed Tri-Vision system for automotive applications

Purpose: Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important for transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. Methods: An experimental, high-speed tri-vision camera system intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotive-grade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the European FP6 project – sensation (advanced sensor development for attention stress, vigilance and sleep/wakefulness monitoring). Results : The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280 × 480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maximum global shutter speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link® cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both electrically and optically. Synchronisation is automatically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensors exceeds 123 dB with a spectral sensitivity extending well into the infra-red range. Conclusion: The system was subjected to a comprehensive testing protocol, which confirms that the salient requirements for the driver monitoring application are adequately met and in some respects, exceeded. The synchronisation technique presented may also benefit several other automotive stereovision applications including near and far-field obstacle detection and collision avoidance, road condition monitoring and others.Partially funded by the EU FP6 through the IST-507231 SENSATION project.peer-reviewe

High-speed imaging in fluids

High-speed imaging is in popular demand for a broad range of experiments in fluids. It allows for a detailed visualization of the event under study by acquiring a series of image frames captured at high temporal and spatial resolution. This review covers high-speed imaging basics, by defining criteria for high-speed imaging experiments in fluids and to give rule-of-thumbs for a series of cases. It also considers stroboscopic imaging, triggering and illumination, and scaling issues. It provides guidelines for testing and calibration. Ultra high-speed imaging at frame rates exceeding 1 million frames per second is reviewed, and the combination of conventional experiments in fluids techniques with high-speed imaging techniques are discussed. The review is concluded with a high-speed imaging chart, which summarizes criteria for temporal scale and spatial scale and which facilitates the selection of a high-speed imaging system for the applicatio

Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary

Cameras face a fundamental tradeoff between the spatial and temporal resolution - digital still cameras can capture images with high spatial resolution, but most high-speed video cameras suffer from low spatial resolution. It is hard to overcome this tradeoff without incurring a significant increase in hardware costs. In this paper, we propose techniques for sampling, representing and reconstructing the space-time volume in order to overcome this tradeoff. Our approach has two important distinctions compared to previous works: (1) we achieve sparse representation of videos by learning an over-complete dictionary on video patches, and (2) we adhere to practical constraints on sampling scheme which is imposed by architectures of present image sensor devices. Consequently, our sampling scheme can be implemented on image sensors by making a straightforward modification to the control unit. To demonstrate the power of our approach, we have implemented a prototype imaging system with per-pixel coded exposure control using a liquid crystal on silicon (LCoS) device. Using both simulations and experiments on a wide range of scenes, we show that our method can effectively reconstruct a video from a single image maintaining high spatial resolution

In-ADC, Rank-Order Filter for Digital Pixel Sensors

© 2023 The Author(s). Licensee MDPI, Basel, Switzerland. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY), https://creativecommons.org/licenses/by/4.0/This paper presents a new implementation of the rank-order filter which is established on 10 a parallel-operated array of single-slope (SS) analogue-to-digital converters (ADCs). The SS ADCs 11 use “on-the-ramp processing” technique i.e. filtration is performed along with analogue-to-digital 12 conversion, so the final states of the converters represent a filtered image. The proof-of-concept 13 64×64 array of SS ADCs, integrated with MOS photogates, was fabricated in a standard 180-nm 14 CMOS process. The measurement results demonstrate the full functionality of the novel filter 15 concept, with image acquisition in both single-sampling and correlated-double-sampling (CDS) 16 modes (the CDS is performed digitally by ADCs). The experimental, massively-parallel rank-order 17 filter can process 650 frames per second with a power consumption of 4.81 mW.Peer reviewe

The Quanta Image Sensor: Every Photon Counts

The Quanta Image Sensor (QIS) was conceived when contemplating shrinking pixel sizes and storage capacities, and the steady increase in digital processing power. In the single-bit QIS, the output of each field is a binary bit plane, where each bit represents the presence or absence of at least one photoelectron in a photodetector. A series of bit planes is generated through high-speed readout, and a kernel or “cubicle” of bits (x, y, t) is used to create a single output image pixel. The size of the cubicle can be adjusted post-acquisition to optimize image quality. The specialized sub-diffraction-limit photodetectors in the QIS are referred to as “jots” and a QIS may have a gigajot or more, read out at 1000 fps, for a data rate exceeding 1 Tb/s. Basically, we are trying to count photons as they arrive at the sensor. This paper reviews the QIS concept and its imaging characteristics. Recent progress towards realizing the QIS for commercial and scientific purposes is discussed. This includes implementation of a pump-gate jot device in a 65 nm CIS BSI process yielding read noise as low as 0.22 e− r.m.s. and conversion gain as high as 420 µV/e−, power efficient readout electronics, currently as low as 0.4 pJ/b in the same process, creating high dynamic range images from jot data, and understanding the imaging characteristics of single-bit and multi-bit QIS devices. The QIS represents a possible major paradigm shift in image capture