Advances on CMOS image sensors

This paper offers an introduction to the technological advances of image sensors designed using complementary metal–oxide–semiconductor (CMOS) processes along the last decades. We review some of those technological advances and examine potential disruptive growth directions for CMOS image sensors and proposed ways to achieve them. Those advances include breakthroughs on image quality such as resolution, capture speed, light sensitivity and color detection and advances on the computational imaging. The current trend is to push the innovation efforts even further as the market requires higher resolution, higher speed, lower power consumption and, mainly, lower cost sensors. Although CMOS image sensors are currently used in several different applications from consumer to defense to medical diagnosis, product differentiation is becoming both a requirement and a difficult goal for any image sensor manufacturer. The unique properties of CMOS process allows the integration of several signal processing techniques and are driving the impressive advancement of the computational imaging. With this paper, we offer a very comprehensive review of methods, techniques, designs and fabrication of CMOS image sensors that have impacted or might will impact the images sensor applications and markets

On evolution of CMOS image sensors

CMOS Image Sensors have become the principal technology in majority of digital cameras. They started replacing the film and Charge Coupled Devices in the last decade with the promise of lower cost, lower power requirement, higher integration and the potential of focal plane processing. However, the principal factor behind their success has been the ability to utilise the shrinkage in CMOS technology to make smaller pixels, and thereby have more resolution without increasing the cost. With the market of image sensors exploding courtesy their inte- gration with communication and computation devices, technology developers improved the CMOS processes to have better optical performance. Nevertheless, the promises of focal plane processing as well as on-chip integration have not been fulfilled. The market is still being pushed by the desire of having higher number of pixels and better image quality, however, differentiation is being difficult for any image sensor manufacturer. In the paper, we will explore potential disruptive growth directions for CMOS Image sensors and ways to achieve the same

Development of a real-time full-field range imaging system

This article describes the development of a full-field range imaging system employing a high frequency amplitude modulated light source and image sensor. Depth images are produced at video frame rates in which each pixel in the image represents distance from the sensor to objects in the scene. The various hardware subsystems are described as are the details about the firmware and software implementation for processing the images in real-time. The system is flexible in that precision can be traded off for decreased acquisition time. Results are reported to illustrate this versatility for both high-speed (reduced precision) and high-precision operating modes

Speckle pattern interferometry : vibration measurement based on a novel CMOS camera

A digital speckle pattern interferometer based on a novel custom complementary metaloxide- semiconductor (CMOS) array detector is described. The temporal evolution of the dynamic deformation of a test object is measured using inter-frame phase stepping. The flexibility of the CMOS detector is used to identify regions of interest with full-field time averaged measurements and then to interrogate those regions with time-resolved measurements sampled at up to 7 kHz. The maximum surface velocity that can be measured and the number of measurement points are limited by the frame rate and the data transfer rate of the detector. The custom sensor used in this work is a modulated light camera (MLC), whose pixel design is still based on the standard four transistor active pixel sensor (APS), but each pixel has four large independently shuttered capacitors that drastically boost the well capacity from that of the diode alone. Each capacitor represents a channel which has its own shutter switch and can either be operated independently or in tandem with others. The particular APS of this camera enables a novel approach in how the data are acquired and then processed. In this Thesis we demonstrate how, at a given frame rate and at a given number of measurement points, the data transfer rate of our system is increased if compared to the data transfer rate of a system using a standard approach. Moreover, under some assumptions, the gain in system bandwidth doesn’t entail any reduction in the maximum surface velocity that can be reliably measured with inter-frame phase stepping

Photographic Noise Performance Measures Based on RAW Files Analysis of Consumer Cameras

[EN] Photography is being benefited from the huge improvement in CMOS image sensors. New cameras extend the dynamic range allowing photographers to take photos with a higher quality than they could imagine one decade ago. However, the existence of different technologies make more complicated the photographic analysis of how to determine the optimal camera exposure settings. In this paper, we analyze how the different noise models are translated to different signal to noise SNR curve patterns and which factors are relevant. In particular, we discuss profoundly the relationships between exposure settings (shutter speed, aperture and ISO). Since a fair comparison between cameras can be tricky because of different pixel size, sensor format or ISO scale definition, we explain how the pixel analysis of a camera can be translated to a more helpful universal photographic noise measure based on human perception and common photography rules. We analyze the RAW files of different camera models and show how the noise performance analysis (SNR and dynamic range) interact with photographer's requirements.Igual García, J. (2019). Photographic Noise Performance Measures Based on RAW Files Analysis of Consumer Cameras. Electronics. 8(11):1-30. https://doi.org/10.3390/electronics8111284S130811Camera Imaging Products Association: Digital Cameras Reporthttp://cipa.jp/stats/dc_e.htmlGye, L. (2007). Picture This: the Impact of Mobile Camera Phones on Personal Photographic Practices. Continuum, 21(2), 279-288. doi:10.1080/10304310701269107Bhandari, A., & Raskar, R. (2016). Signal Processing for Time-of-Flight Imaging Sensors: An introduction to inverse problems in computational 3-D imaging. IEEE Signal Processing Magazine, 33(5), 45-58. doi:10.1109/msp.2016.2582218Wang, J., Zhang, C., & Hao, P. (2011). New color filter arrays of high light sensitivity and high demosaicking performance. 2011 18th IEEE International Conference on Image Processing. doi:10.1109/icip.2011.6116336Chan, C.-C., & Chen, H. H. (2018). Improving the Reliability of Phase Detection Autofocus. Electronic Imaging, 2018(5), 241-1-241-5. doi:10.2352/issn.2470-1173.2018.05.pmii-241Kirkpatrick, K. (2019). The edge of computational photography. Communications of the ACM, 62(7), 14-16. doi:10.1145/3329721Koppal, S. J. (2016). A Survey of Computational Photography in the Small: Creating intelligent cameras for the next wave of miniature devices. IEEE Signal Processing Magazine, 33(5), 16-22. doi:10.1109/msp.2016.2581418CMOS Image Sensor Market: Forecasts from 2019 to 2024https://www.knowledge-sourcing.com/report/cmos-Image-sensor-marketPhotonstophotos.nethttp://photonstophotos.netDxomarkhttp://dxomark.comBoukhayma, A., Peizerat, A., & Enz, C. (2016). Temporal Readout Noise Analysis and Reduction Techniques for Low-Light CMOS Image Sensors. IEEE Transactions on Electron Devices, 63(1), 72-78. doi:10.1109/ted.2015.2434799Vargas-Sierra, S., Linán-Cembrano, G., & Rodríguez-Vázquez, A. (2015). A 151 dB High Dynamic Range CMOS Image Sensor Chip Architecture With Tone Mapping Compression Embedded In-Pixel. IEEE Sensors Journal, 15(1), 180-195. doi:10.1109/jsen.2014.2340875Hassan, N. B., Huang, Y., Shou, Z., Ghassemlooy, Z., Sturniolo, A., Zvanovec, S., … Le-Minh, H. (2018). Impact of Camera Lens Aperture and the Light Source Size on Optical Camera Communications. 2018 11th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP). doi:10.1109/csndsp.2018.8471766Hirsch, J., & Curcio, C. A. (1989). The spatial resolution capacity of human foveal retina. Vision Research, 29(9), 1095-1101. doi:10.1016/0042-6989(89)90058-8ColorChecker Classic Charthttps://xritephoto.com/colorchecker-classicWang, F., & Theuwissen, A. (2017). Linearity analysis of a CMOS image sensor. Electronic Imaging, 2017(11), 84-90. doi:10.2352/issn.2470-1173.2017.11.imse-191Wakashima, S., Kusuhara, F., Kuroda, R., & Sugawa, S. (2015). Analysis of pixel gain and linearity of CMOS image sensor using floating capacitor load readout operation. Image Sensors and Imaging Systems 2015. doi:10.1117/12.2083111Wang, F., Han, L., & Theuwissen, A. J. P. (2018). Development and Evaluation of a Highly Linear CMOS Image Sensor With a Digitally Assisted Linearity Calibration. IEEE Journal of Solid-State Circuits, 53(10), 2970-2981. doi:10.1109/jssc.2018.285625

A Dual Sensor Computational Camera for High Quality Dark Videography

Videos captured under low light conditions suffer from severe noise. A variety of efforts have been devoted to image/video noise suppression and made large progress. However, in extremely dark scenarios, extensive photon starvation would hamper precise noise modeling. Instead, developing an imaging system collecting more photons is a more effective way for high-quality video capture under low illuminations. In this paper, we propose to build a dual-sensor camera to additionally collect the photons in NIR wavelength, and make use of the correlation between RGB and near-infrared (NIR) spectrum to perform high-quality reconstruction from noisy dark video pairs. In hardware, we build a compact dual-sensor camera capturing RGB and NIR videos simultaneously. Computationally, we propose a dual-channel multi-frame attention network (DCMAN) utilizing spatial-temporal-spectral priors to reconstruct the low-light RGB and NIR videos. In addition, we build a high-quality paired RGB and NIR video dataset, based on which the approach can be applied to different sensors easily by training the DCMAN model with simulated noisy input following a physical-process-based CMOS noise model. Both experiments on synthetic and real videos validate the performance of this compact dual-sensor camera design and the corresponding reconstruction algorithm in dark videography