Rethinking the Pipeline of Demosaicing, Denoising and Super-Resolution

Incomplete color sampling, noise degradation, and limited resolution are the three key problems that are unavoidable in modern camera systems. Demosaicing (DM), denoising (DN), and super-resolution (SR) are core components in a digital image processing pipeline to overcome the three problems above, respectively. Although each of these problems has been studied actively, the mixture problem of DM, DN, and SR, which is a higher practical value, lacks enough attention. Such a mixture problem is usually solved by a sequential solution (applying each method independently in a fixed order: DM $\to$ DN $\to$ SR), or is simply tackled by an end-to-end network without enough analysis into interactions among tasks, resulting in an undesired performance drop in the final image quality. In this paper, we rethink the mixture problem from a holistic perspective and propose a new image processing pipeline: DN $\to$ SR $\to$ DM. Extensive experiments show that simply modifying the usual sequential solution by leveraging our proposed pipeline could enhance the image quality by a large margin. We further adopt the proposed pipeline into an end-to-end network, and present Trinity Enhancement Network (TENet). Quantitative and qualitative experiments demonstrate the superiority of our TENet to the state-of-the-art. Besides, we notice the literature lacks a full color sampled dataset. To this end, we contribute a new high-quality full color sampled real-world dataset, namely PixelShift200. Our experiments show the benefit of the proposed PixelShift200 dataset for raw image processing.Comment: Code is available at: https://github.com/guochengqian/TENe

Simultaneous real-time visible and infrared video with single-pixel detectors

Conventional cameras rely upon a pixelated sensor to provide spatial resolution. An alternative approach replaces the sensor with a pixelated transmission mask encoded with a series of binary patterns. Combining knowledge of the series of patterns and the associated filtered intensities, measured by single-pixel detectors, allows an image to be deduced through data inversion. In this work we extend the concept of a ‘single-pixel camera’ to provide continuous real-time video at 10 Hz , simultaneously in the visible and short-wave infrared, using an efficient computer algorithm. We demonstrate our camera for imaging through smoke, through a tinted screen, whilst performing compressive sampling and recovering high-resolution detail by arbitrarily controlling the pixel-binning of the masks. We anticipate real-time single-pixel video cameras to have considerable importance where pixelated sensors are limited, allowing for low-cost, non-visible imaging systems in applications such as night-vision, gas sensing and medical diagnostics

High-speed global shutter CMOS machine vision sensor with high dynamic range image acquisition and embedded intelligence

High-speed imagers are required for industrial applications, traffic monitoring, robotics and unmanned vehicles, moviemaking, etc. Many of these applications call also for large spatial resolution, high sensitivity and the ability to detect images with large intra-frame dynamic range. This paper reports a CIS intelligent digital image sensor with 5.2Mpixels which delivers 12-bit fully-corrected images at 250Fps. The new sensor embeds on-chip digital processing circuitry for a large variety of functions including: windowing; pixel binning; sub-sampling; combined windowing-binning-subsampling modes; fixed-pattern noise correction; fine gain and offset control; color processing, etc. These and other CIS functions are programmable through a simple four-wire serial port interface.Ministerio de Ciencia e Innovación IPT-2011-1625-43000