The effect of the color filter array layout choice on state-of-the-art demosaicing

Interpolation from a Color Filter Array (CFA) is the most common method for obtaining full color image data. Its success relies on the smart combination of a CFA and a demosaicing algorithm. Demosaicing on the one hand has been extensively studied. Algorithmic development in the past 20 years ranges from simple linear interpolation to modern neural-network-based (NN) approaches that encode the prior knowledge of millions of training images to fill in missing data in an inconspicious way. CFA design, on the other hand, is less well studied, although still recognized to strongly impact demosaicing performance. This is because demosaicing algorithms are typically limited to one particular CFA pattern, impeding straightforward CFA comparison. This is starting to change with newer classes of demosaicing that may be considered generic or CFA-agnostic. In this study, by comparing performance of two state-of-the-art generic algorithms, we evaluate the potential of modern CFA-demosaicing. We test the hypothesis that, with the increasing power of NN-based demosaicing, the influence of optimal CFA design on system performance decreases. This hypothesis is supported with the experimental results. Such a finding would herald the possibility of relaxing CFA requirements, providing more freedom in the CFA design choice and producing high-quality cameras

Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

Multispectral Color Constancy: Real Image Tests

Experiments using real images are conducted on a variety of color constancy algorithms (Chromagenic, Greyworld, Max RGB, and a Maloney-Wandell extension called Subspace Testing) in order to determine whether or not extending the number of channels from 3 to 6 to 9 would enhance the accuracy with which they estimate the scene illuminant color. To create the 6 and 9 channel images, filters where placed over a standard 3-channel color camera. Although some improvement is found with 6 channels, the results indicate that essentially the extra channels do not help as much as might be expected

Spectral image analysis for measuring ripeness of tomatoes

In this study, spectral images of five ripeness stages of tomatoes have been recorded and analyzed. The electromagnetic spectrum between 396 and 736 nm was recorded in 257 bands (every 1.3 nm). Results show that spectral images offer more discriminating power than standard RGB images for measuring ripeness stages of tomatoes. The classification error of individual pixels was reduced from 51% to 19%. Using a gray reference, the reflectance can be made invariant to the light source and even object geometry, which makes it possible to have comparable classification results over a large range of illumination conditions. Experimental results show that, although the error rate increases from 19% to 35% when using different light sources, it is still considerably below the 51% for RGB under a single light sourc

Evaluating color texture descriptors under large variations of controlled lighting conditions

The recognition of color texture under varying lighting conditions is still an open issue. Several features have been proposed for this purpose, ranging from traditional statistical descriptors to features extracted with neural networks. Still, it is not completely clear under what circumstances a feature performs better than the others. In this paper we report an extensive comparison of old and new texture features, with and without a color normalization step, with a particular focus on how they are affected by small and large variation in the lighting conditions. The evaluation is performed on a new texture database including 68 samples of raw food acquired under 46 conditions that present single and combined variations of light color, direction and intensity. The database allows to systematically investigate the robustness of texture descriptors across a large range of variations of imaging conditions.Comment: Submitted to the Journal of the Optical Society of America

Metamer Mismatching in Practice versus Theory

Metamer mismatching (the phenomenon that two objects matching in color under one illuminant may not match under a different illuminant) potentially has important consequences for color perception. Logvinenko et al. [PLoS ONE 10, e0135029 (2015)] show that in theory the extent of metamer mismatching can be very significant. This paper examines metamer mismatching in practice by computing the volumes of the empirical metamer mismatch bodies and comparing them to the volumes of the theoretical mismatch bodies. A set of more than 25 million unique reflectance spectra is assembled using datasets from several sources. For a given color signal (e.g., CIE XYZ) recorded under a given first illuminant, its empirical metamer mismatch body for a change to a second illuminant is computed as follows: the reflectances having the same color signal when lit by the first illuminant (i.e., reflect metameric light) are computationally relit by the second illuminant, and the convex hull of the resulting color signals then defines the empirical metamer mismatch body. The volume of these bodies is shown to vary systematically with Munsell value and chroma. The empirical mismatch bodies are compared to the theoretical mismatch bodies computed using the algorithm of Logvinenko et al. [IEEE Trans. Image Process. 23, 34 (2014)]. There are three key findings: (1) the empirical bodies are found to be substantially smaller than the theoretical ones; (2) the sizes of both the empirical and theoretical bodies show a systematic variation with Munsell value and chroma; and (3) applied to the problem of color-signal prediction, the centroid of the empirical metamer mismatch body is shown to be a better predictor of what a given color signal might become under a specified illuminant than state-of-the-art methods

Review: computer vision applied to the inspection and quality control of fruits and vegetables

This is a review of the current existing literature concerning the inspection of fruits and vegetables with the application of computer vision, where the techniques most used to estimate various properties related to quality are analyzed. The objectives of the typical applications of such systems include the classification, quality estimation according to the internal and external characteristics, supervision of fruit processes during storage or the evaluation of experimental treatments. In general, computer vision systems do not only replace manual inspection, but can also improve their skills. In conclusion, computer vision systems are powerful tools for the automatic inspection of fruits and vegetables. In addition, the development of such systems adapted to the food industry is fundamental to achieve competitive advantages

Fusion of hyperspectral, multispectral, color and 3D point cloud information for the semantic interpretation of urban environments

In this paper, we address the semantic interpretation of urban environments on the basis of multi-modal data in the form of RGB color imagery, hyperspectral data and LiDAR data acquired from aerial sensor platforms. We extract radiometric features based on the given RGB color imagery and the given hyperspectral data, and we also consider different transformations to potentially better data representations. For the RGB color imagery, these are achieved via color invariants, normalization procedures or specific assumptions about the scene. For the hyperspectral data, we involve techniques for dimensionality reduction and feature selection as well as a transformation to multispectral Sentinel-2-like data of the same spatial resolution. Furthermore, we extract geometric features describing the local 3D structure from the given LiDAR data. The defined feature sets are provided separately and in different combinations as input to a Random Forest classifier. To assess the potential of the different feature sets and their combination, we present results achieved for the MUUFL Gulfport Hyperspectral and LiDAR Airborne Data Set