The effect of the color filter array layout choice on state-of-the-art demosaicing

Interpolation from a Color Filter Array (CFA) is the most common method for obtaining full color image data. Its success relies on the smart combination of a CFA and a demosaicing algorithm. Demosaicing on the one hand has been extensively studied. Algorithmic development in the past 20 years ranges from simple linear interpolation to modern neural-network-based (NN) approaches that encode the prior knowledge of millions of training images to fill in missing data in an inconspicious way. CFA design, on the other hand, is less well studied, although still recognized to strongly impact demosaicing performance. This is because demosaicing algorithms are typically limited to one particular CFA pattern, impeding straightforward CFA comparison. This is starting to change with newer classes of demosaicing that may be considered generic or CFA-agnostic. In this study, by comparing performance of two state-of-the-art generic algorithms, we evaluate the potential of modern CFA-demosaicing. We test the hypothesis that, with the increasing power of NN-based demosaicing, the influence of optimal CFA design on system performance decreases. This hypothesis is supported with the experimental results. Such a finding would herald the possibility of relaxing CFA requirements, providing more freedom in the CFA design choice and producing high-quality cameras

The PLATO Dome A Site-Testing Observatory : instrumentation and first results

The PLATeau Observatory (PLATO) is an automated self-powered astrophysical observatory that was deployed to Dome A, the highest point on the Antarctic plateau, in 2008 January. PLATO consists of a suite of site-testing instruments designed to quantify the benefits of the Dome A site for astronomy, and science instruments designed to take advantage of the unique observing conditions. Instruments include CSTAR, an array of optical telescopes for transient astronomy; Gattini, an instrument to measure the optical sky brightness and cloud cover statistics; DASLE, an experiment to measure the statistics of the meteorological conditions within the near-surface layer; Pre-HEAT, a submillimeter tipping radiometer measuring the atmospheric transmission and water vapor content and performing spectral line imaging of the Galactic plane; and Snodar, an acoustic radar designed to measure turbulence within the near-surface layer. PLATO has run completely unattended and collected data throughout the winter 2008 season. Here we present a detailed description of the PLATO instrument suite and preliminary results obtained from the first season of operation

Deep Bilateral Learning for Real-Time Image Enhancement

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.Comment: 12 pages, 14 figures, Siggraph 201

Hybridization of optical plasmonics with terahertz metamaterials to create multi-spectral filters

Multi-spectral imaging systems typically require the cumbersome integration of disparate filtering materials in order to work simultaneously in multiple spectral regions. We show for the first time how a single nano-patterned metal film can be used to filter multi-spectral content from the visible, near infrared and terahertz bands by hybridizing plasmonics and metamaterials. Plasmonic structures are well-suited to the visible band owing to the resonant dielectric properties of metals, whereas metamaterials are preferable at terahertz frequencies where metal conductivity is high. We present the simulated and experimental characteristics of our new hybrid synthetic multi-spectral material filters and demonstrate the independence of the metamaterial and plasmonic responses with respect to each other

Advances on CMOS image sensors

This paper offers an introduction to the technological advances of image sensors designed using complementary metal–oxide–semiconductor (CMOS) processes along the last decades. We review some of those technological advances and examine potential disruptive growth directions for CMOS image sensors and proposed ways to achieve them. Those advances include breakthroughs on image quality such as resolution, capture speed, light sensitivity and color detection and advances on the computational imaging. The current trend is to push the innovation efforts even further as the market requires higher resolution, higher speed, lower power consumption and, mainly, lower cost sensors. Although CMOS image sensors are currently used in several different applications from consumer to defense to medical diagnosis, product differentiation is becoming both a requirement and a difficult goal for any image sensor manufacturer. The unique properties of CMOS process allows the integration of several signal processing techniques and are driving the impressive advancement of the computational imaging. With this paper, we offer a very comprehensive review of methods, techniques, designs and fabrication of CMOS image sensors that have impacted or might will impact the images sensor applications and markets