Compressive image sensor architecture with on-chip measurement matrix generation

A CMOS image sensor architecture that uses a cellular automaton for the pseudo-random compressive sampling matrix generation is presented. The image sensor employs in-pixel pulse-frequency modulation and column wise pulse counters to produce compressed samples. A common problem of compressive sampling applied to image sensors is that the size of a full-frame compressive strategy is too large to be stored in an on-chip memory. Since this matrix has to be transmitted to or from the reconstruction system its size would also prevent practical applications. A full-frame compressive strategy generated using a 1-D cellular automaton showing a class III behavior neither needs a storage memory nor needs to be continuously transmitted. In-pixel pulse frequency modulation and up-down counters allow the generation of differential compressed samples directly in the digital domain where it is easier to improve the required dynamic range. These solutions combined together improve the accuracy of the compressed samples thus improving the performance of any generic reconstruction algorithm.Ministerio de Economía y Competitividad TEC2015-66878-C3-1-RJunta de Andalucía TIC 2338-2013Office of Naval Research (USA) N00014141035

Concurrent focal-plane generation of compressed samples fromtime-encoded pixel values

Compressive sampling allows wrapping the relevant content of an image in a reduced set of data. It exploits the sparsity of natural images. This principle can be employed to deliver images over a network under a restricted data rate and still receive enough meaningful information. An efficient implementation of this principle lies in the generation of the compressed samples right at the imager. Otherwise, i. e. digitizing the complete image and then composing the compressed samples in the digital plane, the required memory and processing resources can seriously compromise the budget of an autonomous camera node. In this paper we present the design of a pixel architecture that encodes light intensity into time, followed by a global strategy to pseudo-randomly combine pixel values and generate, on-chip and on-line, the compressed samples.Ministerio de Economía y Competitividad TEC 2015-66878-C3-1-RJunta de Andalucía TIC 2338-2013Office of Naval Research (USA) N000141410355CONACYT (Mexico) MZO-2017-29106

DESIGN OF CMOS COMPRESSIVE SENSING IMAGE SENSORS

This work investigates the optimal measurement matrices that can be used in compressive sensing (CS) image sensors. It also optimizes CMOS current-model pixel cell circuits for CS image sensors. Based on the outcomes from these optimization studies, three CS image senor circuits with compression ratios of 4, 6, and 8 are designed with using a 130 nm CMOS technology. The pixel arrays used in the image sensors has a size of 256X256. Circuit simulations with benchmark image Lenna show that the three images sensors can achieve peak signal to noise ratio (PSNR) values of 37.64, 33.29, and 32.44 dB respectively

Compressive Image Acquisition in Modern CMOS IC Design

Compressive Sampling (CS) offers bandwidth, power and memory size reduction compared to conventional (Nyquist) sampling. However, very few Integrated Circuit (IC) designs based on CS exist due to the missing link between the well-established CS theory on one side, and the practical aspects/effects related to physical IC design on the other side. This paper focuses on the application of compressed image acquisition in CMOS image sensor integrated circuit design. A new CS scheme is proposed which is suited for hardware implementation in CMOS IC design. All the main physical non-idealities are explained and carefully modeled. Their influences on the acquired image quality are analyzed in the general case and quantifed for the case of the proposed CS scheme. The presented methodology can also be used for different CS schemes and as a general guideline in future CS based CMOS image sensor designs

Exploring information retrieval using image sparse representations:from circuit designs and acquisition processes to specific reconstruction algorithms

New advances in the field of image sensors (especially in CMOS technology) tend to question the conventional methods used to acquire the image. Compressive Sensing (CS) plays a major role in this, especially to unclog the Analog to Digital Converters which are generally representing the bottleneck of this type of sensors. In addition, CS eliminates traditional compression processing stages that are performed by embedded digital signal processors dedicated to this purpose. The interest is twofold because it allows both to consistently reduce the amount of data to be converted but also to suppress digital processing performed out of the sensor chip. For the moment, regarding the use of CS in image sensors, the main route of exploration as well as the intended applications aims at reducing power consumption related to these components (i.e. ADC & DSP represent 99% of the total power consumption). More broadly, the paradigm of CS allows to question or at least to extend the Nyquist-Shannon sampling theory. This thesis shows developments in the field of image sensors demonstrating that is possible to consider alternative applications linked to CS. Indeed, advances are presented in the fields of hyperspectral imaging, super-resolution, high dynamic range, high speed and non-uniform sampling. In particular, three research axes have been deepened, aiming to design proper architectures and acquisition processes with their associated reconstruction techniques taking advantage of image sparse representations. How the on-chip implementation of Compressed Sensing can relax sensor constraints, improving the acquisition characteristics (speed, dynamic range, power consumption) ? How CS can be combined with simple analysis to provide useful image features for high level applications (adding semantic information) and improve the reconstructed image quality at a certain compression ratio ? Finally, how CS can improve physical limitations (i.e. spectral sensitivity and pixel pitch) of imaging systems without a major impact neither on the sensing strategy nor on the optical elements involved ? A CMOS image sensor has been developed and manufactured during this Ph.D. to validate concepts such as the High Dynamic Range - CS. A new design approach was employed resulting in innovative solutions for pixels addressing and conversion to perform specific acquisition in a compressed mode. On the other hand, the principle of adaptive CS combined with the non-uniform sampling has been developed. Possible implementations of this type of acquisition are proposed. Finally, preliminary works are exhibited on the use of Liquid Crystal Devices to allow hyperspectral imaging combined with spatial super-resolution. The conclusion of this study can be summarized as follows: CS must now be considered as a toolbox for defining more easily compromises between the different characteristics of the sensors: integration time, converters speed, dynamic range, resolution and digital processing resources. However, if CS relaxes some material constraints at the sensor level, it is possible that the collected data are difficult to interpret and process at the decoder side, involving massive computational resources compared to so-called conventional techniques. The application field is wide, implying that for a targeted application, an accurate characterization of the constraints concerning both the sensor (encoder), but also the decoder need to be defined

A (256*256) pixel 76.7mW CMOS imager/ compressor based on real-time in-pixel compressive sensing

A CMOS imager is presented which has the ability to perform localized compressive sensing on-chip. In-pixel convolutions of the sensed image with measurement matrices are computed in real time, and a proposed programmable two-dimensional scrambling technique guarantees the randomness of the coefficients used in successive observation. A power and area-efficient implementation architecture is presented making use of a single ADC. A 256*256 imager has been developed as a test vehicle in a 0.18 mu m CIS technology. Using an 11-bit ADC, a SNR of 18.6dB with a compression factor of 3.3 is achieved after reconstruction. The total power consumption of the imager is simulated at 76.7mW from a 1.8V supply voltage.Anglai