Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition

International audienceMultispectral acquisition improves machine vision since it permits capturing more information on object surface properties than color imaging. The concept of spectral filter arrays has been developed recently and allows multispectral single shot acquisition with a compact camera design. Due to filter manufacturing difficulties, there was, up to recently, no system available for a large span of spectrum, i.e., visible and Near Infra-Red acquisition. This article presents the achievement of a prototype of camera that captures seven visible and one near infra-red bands on the same sensor chip. A calibration is proposed to characterize the sensor, and images are captured. Data are provided as supplementary material for further analysis and simulations. This opens a new range of applications in security, robotics, automotive and medical fields

High Dynamic Range Spectral Imaging Pipeline For Multispectral Filter Array Cameras

Spectral filter arrays imaging exhibits a strong similarity with color filter arrays. This permits us to embed this technology in practical vision systems with little adaptation of the existing solutions. In this communication, we define an imaging pipeline that permits high dynamic range (HDR)-spectral imaging, which is extended from color filter arrays. We propose an implementation of this pipeline on a prototype sensor and evaluate the quality of our implementation results on real data with objective metrics and visual examples. We demonstrate that we reduce noise, and, in particular we solve the problem of noise generated by the lack o

Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition

Multispectral acquisition improves machine vision since it permits capturing more information on object surface properties than color imaging. The concept of spectral filter arrays has been developed recently and allows multispectral single shot acquisition with a compact camera design. Due to filter manufacturing difficulties, there was, up to recently, no system available for a large span of spectrum, i.e., visible and Near Infra-Red acquisition. This article presents the achievement of a prototype of camera that captures seven visible and one near infra-red bands on the same sensor chip. A calibration is proposed to characterize the sensor, and images are captured. Data are provided as supplementary material for further analysis and simulations. This opens a new range of applications in security, robotics, automotive and medical fields

Expanding Dimensionality in Cinema Color: Impacting Observer Metamerism through Multiprimary Display

Television and cinema display are both trending towards greater ranges and saturation of reproduced colors made possible by near-monochromatic RGB illumination technologies. Through current broadcast and digital cinema standards work, system designs employing laser light sources, narrow-band LED, quantum dots and others are being actively endorsed in promotion of Wide Color Gamut (WCG). Despite artistic benefits brought to creative content producers, spectrally selective excitations of naturally different human color response functions exacerbate variability of observer experience. An exaggerated variation in color-sensing is explicitly counter to the exhaustive controls and calibrations employed in modern motion picture pipelines. Further, singular standard observer summaries of human color vision such as found in the CIE’s 1931 and 1964 color matching functions and used extensively in motion picture color management are deficient in recognizing expected human vision variability. Many researchers have confirmed the magnitude of observer metamerism in color matching in both uniform colors and imagery but few have shown explicit color management with an aim of minimized difference in observer perception variability. This research shows that not only can observer metamerism influences be quantitatively predicted and confirmed psychophysically but that intentionally engineered multiprimary displays employing more than three primaries can offer increased color gamut with drastically improved consistency of experience. To this end, a seven-channel prototype display has been constructed based on observer metamerism models and color difference indices derived from the latest color vision demographic research. This display has been further proven in forced-choice paired comparison tests to deliver superior color matching to reference stimuli versus both contemporary standard RGB cinema projection and recently ratified standard laser projection across a large population of color-normal observers

Nanophotonic filters for digital imaging

There has been an increasing demand for low cost, portable CMOS image sensors because of increased integration, and new applications in the automotive, mobile communication and medical industries, amongst others. Colour reproduction remains imperfect in conventional digital image sensors, due to the limitations of the dye-based filters. Further improvement is required if the full potential of digital imaging is to be realised. In alternative systems, where accurate colour reproduction is a priority, existing equipment is too bulky for anything but specialist use. In this work both these issues are addressed by exploiting nanophotonic techniques to create enhanced trichromatic filters, and multispectral filters, all of which can be fabricated on-chip, i.e. integrated into a conventional digital image sensor, to create compact, low cost, mass produceable imaging systems with accurate colour reproduction. The trichromatic filters are based on plasmonic structures. They exploit the excitation of surface plasmon resonances in arrays of subwavelength holes in metal films to filter light. The currently-known analytical expressions are inadequate for optimising all relevant parameters of a plasmonic structure. In order to obtain arbitrary filter characteristics, an automated design procedure was developed that integrated a genetic algorithm and 3D finite-difference time-domain tool. The optimisation procedure's efficacy is demonstrated by designing a set of plasmonic filters that replicate the CIE (1931) colour matching functions, which themselves mimic the human eye's daytime colour response. The best designs were fabricated and demonstrated a least-mean-square error, in comparison to the desired colour matching functions, of 6.37*10^3, 2.34*10^3 and 11.10*10^3 for the red, green, and blue filters respectively. Notably the spectrum for the red filter contained a double peak, as present in the corresponding colour matching function. Such dual peak behaviour cannot be achieved using a single current dye-based filter. The filters retain the same layer thickness for all structures so they can be defined in a single lithography step. A new approach to enable the fabrication of a multispectral filter array on a CMOS imager is also presented. This combines a Fabry-Perot filter with effective medium theory (EMT) to enable the fabrication of multiple filters in a single cavity length via lithographic tuning of the filter passband. Two approaches are proposed; air-filled nanostructures and dielectric backfilled nanostructures. The air-filled approach is demonstrated experimentally producing three filters with FWHM of 60 - 64 nm. Using the backfilled design, and incorporating a highindex cavity material, a set of twenty three narrowband filters, with a FWHM of 22 - 46nm is demonstrated. A virtual image reproduction process was developed to quantify the image reproduction performance of both the plasmonic and Fabry-Perot filter sets. A typical rgb dye-based filter set used in conventional imagers achieves a mean colour error of 2.711, whereas the experimental data from the plasmonic filters achieves an error of 2.222 which demonstrated a slight improvement in colour reproduction. The multispectral filter set developed in this work performed even better, with 4 filters giving an error of 0.906, 10 filters an error of 0.072 and continued improvement in the colour error reaching 0.047 for 23 filters. All the filter sets proposed are fully compatible with the CMOS process so as to enable direct integration onto CMOS image sensors in industrial foundries in future. The performance of the presented filters also suggest new compact applications in art reproduction, agricultural monitoring and medical imaging

Evaluation and optimal design of spectral sensitivities for digital color imaging

The quality of an image captured by color imaging system primarily depends on three factors: sensor spectral sensitivity, illumination and scene. While illumination is very important to be known, the sensitivity characteristics is critical to the success of imaging applications, and is necessary to be optimally designed under practical constraints. The ultimate image quality is judged subjectively by human visual system. This dissertation addresses the evaluation and optimal design of spectral sensitivity functions for digital color imaging devices. Color imaging fundamentals and device characterization are discussed in the first place. For the evaluation of spectral sensitivity functions, this dissertation concentrates on the consideration of imaging noise characteristics. Both signal-independent and signal-dependent noises form an imaging noise model and noises will be propagated while signal is processed. A new colorimetric quality metric, unified measure of goodness (UMG), which addresses color accuracy and noise performance simultaneously, is introduced and compared with other available quality metrics. Through comparison, UMG is designated as a primary evaluation metric. On the optimal design of spectral sensitivity functions, three generic approaches, optimization through enumeration evaluation, optimization of parameterized functions, and optimization of additional channel, are analyzed in the case of the filter fabrication process is unknown. Otherwise a hierarchical design approach is introduced, which emphasizes the use of the primary metric but the initial optimization results are refined through the application of multiple secondary metrics. Finally the validity of UMG as a primary metric and the hierarchical approach are experimentally tested and verified

An FPGA-based pipeline for micropolarizer array imaging

The enhancement of current camera performances, in terms of framerate, image resolution and pixel width, has direct consequences on the amount of resources needed to process video data. Stokes imaging permits to estimate polarization of light and create multiple polarization descriptors of the scene. Therefore, such video cameras need fast processing for critical applications like overseeing, defect detection or surface characterization. An FPGA hardware implementation of Stokes processing is presented here that embeds dedicated pipeline for micropolarizer array sensors. An optimized fixed-point pipeline is used to compute polarimetric images, i.e. Stokes vector, degree of polarization and angle of polarization. Simulation and experimental studies are done. The hardware design contains parallel processing, low latency and low power and could meet actual real-time and embeddable requirements for smart camera systems