Estimation of illuminants from color signals of illuminated objects

Color constancy is the ability of the human visual systems to discount the effect of the illumination and to assign approximate constant color descriptions to objects. This ability has long been studied and widely applied to many areas such as color reproduction and machine vision, especially with the development of digital color processing. This thesis work makes some improvements in illuminant estimation and computational color constancy based on the study and testing of existing algorithms. During recent years, it has been noticed that illuminant estimation based on gamut comparison is efficient and simple to implement. Although numerous investigations have been done in this field, there are still some deficiencies. A large part of this thesis has been work in the area of illuminant estimation through gamut comparison. Noting the importance of color lightness in gamut comparison, and also in order to simplify three-dimensional gamut calculation, a new illuminant estimation method is proposed through gamut comparison at separated lightness levels. Maximum color separation is a color constancy method which is based on the assumption that colors in a scene will obtain the largest gamut area under white illumination. The method was further derived and improved in this thesis to make it applicable and efficient. In addition, some intrinsic questions in gamut comparison methods, for example the relationship between the color space and the application of gamut or probability distribution, were investigated. Color constancy methods through spectral recovery have the limitation that there is no effective way to confine the range of object spectral reflectance. In this thesis, a new constraint on spectral reflectance based on the relative ratios of the parameters from principal component analysis (PCA) decomposition is proposed. The proposed constraint was applied to illuminant detection methods as a metric on the recovered spectral reflectance. Because of the importance of the sensor sensitivities and their wide variation, the influence from the sensor sensitivities on different kinds of illuminant estimation methods was also studied. Estimation method stability to wrong sensor information was tested, suggesting the possible solution to illuminant estimation on images with unknown sources. In addition, with the development of multi-channel imaging, some research on illuminant estimation for multi-channel images both on the correlated color temperature (CCT) estimation and the illuminant spectral recovery was performed in this thesis. All the improvement and new proposed methods in this thesis are tested and compared with those existing methods with best performance, both on synthetic data and real images. The comparison verified the high efficiency and implementation simplicity of the proposed methods

Motion of glossy objects does not promote separation of lighting and surface colour

The surface properties of an object, such as texture, glossiness or colour, provide important cues to its identity. However, the actual visual stimulus received by the eye is determined by both the properties of the object and the illumination. We tested whether operational colour constancy for glossy objects (the ability to distinguish changes in spectral reflectance of the object, from changes in the spectrum of the illumination) was affected by rotational motion of either the object or the light source. The different chromatic and geometric properties of the specular and diffuse reflections provide the basis for this discrimination, and we systematically varied specularity to control the available information. Observers viewed animations of isolated objects undergoing either lighting or surface-based spectral transformations accompanied by motion. By varying the axis of rotation, and surface patterning or geometry, we manipulated: (i) motion-related information about the scene, (ii) relative motion between the surface patterning and the specular reflection of the lighting, and (iii) image disruption caused by this motion. Despite large individual differences in performance with static stimuli, motion manipulations neither improved nor degraded performance. As motion significantly disrupts frameby-frame low-level image statistics, we infer that operational constancy depends on a high-level scene interpretation, which is maintained in all condition

Extending minkowski norm illuminant estimation

The ability to obtain colour images invariant to changes of illumination is called colour constancy. An algorithm for colour constancy takes sensor responses - digital images - as input, estimates the ambient light and returns a corrected image in which the illuminant influence over the colours has been removed. In this thesis we investigate the step of illuminant estimation for colour constancy and aim to extend the state of the art in this field. We first revisit the Minkowski Family Norm framework for illuminant estimation. Because, of all the simple statistical approaches, it is the most general formulation and, crucially, delivers the best results. This thesis makes four technical contributions. First, we reformulate the Minkowski approach to provide better estimation when a constraint on illumination is employed. Second, we show how the method can (by orders of magnitude) be implemented to run much faster than previous algorithms. Third, we show how a simple edge based variant delivers improved estimation compared with the state of the art across many datasets. In contradistinction to the prior state of the art our definition of edges is fixed (a simple combination of first and second derivatives) i.e. we do not tune our algorithm to particular image datasets. This performance is further improved by incorporating a gamut constraint on surface colour -our 4th contribution. The thesis finishes by considering our approach in the context of a recent OSA competition run to benchmark computational algorithms operating on physiologically relevant cone based input data. Here we find that Constrained Minkowski Norms operi ii ating on spectrally sharpened cone sensors (linear combinations of the cones that behave more like camera sensors) supports competition leading illuminant estimation

Chromatic scene statistics as cues for the perception of surface and illumination colour

Diese Arbeit widmet sich einer der meistuntersuchten Fragen der Wahrnehmungspsychologie: Wie löst das menschliche Wahrnehmungssystem das sogenannte Farbkonstanzproblem, um die Farbe von Oberflächen zu bestimmen? Der lokale proximale Input, das Licht, das von einer Oberfläche zum Auge reflektiert wird, hängt nämlich nicht nur von der spektralen Reflektanzeigenschaft der Oberfläche, sondern auch von der spektralen Energieverteilung der Beleuchtung ab und variiert daher unter Veränderungen der Beleuchtung. Wie kann das Wahrnehmungssystem dann die Farbe einer Oberfläche auch unter wechselnden Beleuchtungen als konstant wahrnehmen? Auch nach mehr als einhundert Jahren Forschung herrscht Einigkeit nur darüber, dass das Wahrnehmungssystem für diese Leistung globalere Informationen des gesamten proximalen Bildes verwenden muss, nicht jedoch darüber, welche Informationen es verwendet. Diese Arbeit beinhaltet eine Analyse eines Modells der spektralen Eigenschaften natürlicher Oberflächen sowie eine Analyse chromatischer Statistiken natürlicher Szenen, um aus den beleuchtungsabhängigen Regularitäten der chromatischen Eigenschaften der proximalen Bilder jene globalen statistischen Informationen abzuleiten, die das Wahrnehmungssystem als Hinweisreiz für die Beleuchtung verwenden kann. In psychophysikalischen Experimenten wird dann untersucht, ob das Wahrnehmungssystem diese potentiellen Hinweisreize für die Wahrnehmung von Oberflächenfarben berücksichtigt. Die Ergebnisse für einen dieser Hinweisreize, die Korrelation zwischen Luminanz und Rötlichkeit im proximalen Bild, legen den Schluss nahe, dass das Wahrnehmungssystem die diesem Hinweisreiz zugrundeliegende Regularität unserer chromatischen Außenwelt internalisiert hat, um Oberflächenfarben konstant wahrzunehmen

The role of chromatic texture and 3D shape in colour discrimination, memory colour, and colour constancy of natural objects

The primary goal of this work was to investigate colour perception in a natural environment and to contribute to the understanding of how cues to familiar object identity influence colour appearance. A large number of studies on colour appearance employ 2D uniformly coloured patches, discarding perceptual cues such as binocular disparity, 3D luminance shading, mutual reflection, and glossy highlights are integral part of a natural scene. Moreover, natural objects possess specific cues that help our recognition (shape, surface texture or colour distribution). The aim of the first main experiment presented in this thesis was to understand the effect of shape on (1) memory colour under constant and varying illumination and on (2) colour constancy for uniformly coloured stimuli. The results demonstrated the existence of a range of memory colours associated with a familiar object, the size of which was strongly object-shape-dependent. For all objects, memory retrieval was significantly faster for object-diagnostic shape relative to generic shapes. Based on two successive controls, the author suggests that shape cues to the object identity affect the range of memory colour proportionally to the original object chromatic distribution. The second experiment examined the subject’s accuracy and precision in adjusting a stimulus colour to its typical appearance. Independently on the illuminant, results showed that memory colour accuracy and precision were enhanced by the presence of chromatic textures, diagnostic shapes, or 3D configurations with a strong interaction between diagnosticity and dimensionality of the shape. Hence, more cues to the object identity and more natural stimuli facilitate the observers in accessing their colour information from memory. A direct relationship was demonstrated between chromatic surface representation, object’s physical properties, and identificability and dimensionality of shape on memory colour accuracy, suggesting high-level mechanisms. Chromatic textures facilitated colour constancy. The third and fourth experiments tested the subject’s ability to discriminate between two chromatic stimuli in a simultaneous and successive 2AFC task, respectively. Simultaneous discrimination threshold performances for polychromatic surfaces were only due to low-level mechanism of the stimulus, whereas in the successive discrimination, i.e. when memory is involved, high-level mechanisms were established. The effect of shape was strongly task- dependent and was modulate by the object memory colour. These findings together with the strong interaction between chromatic cues and shape cues to the object identity lead to the conclusion that high level mechanisms linked to object recognition facilitated both tasks. Hence, the current thesis presents new findings on memory colour and colour constancy presented in a natural context and demonstrates the effect of high-level mechanisms in chromatic discrimination as a function of cues to the object identity such as shape and texture. This work contributes to a deeper understanding of colour perception and object recognition in the natural world.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

The time-course of colour vision

Four experiments are presented, each investigating temporal properties of colour vision processing in human observers. The first experiment replicates and extends an experiment by Stromeyer et al. (1991). We look for a phase difference between combined temporal modulations in orthogonal directions in colour space, which might null the often-claimed latency of signals originating from the short-wavelength sensitive cones (S-cones). We provide another estimate of the magnitude of this latency, and give evidence to suggest that it originates early in the chromatic pathway, before signals from S-cones are combined with those that receive opposed L- and M-cone input. In the second experiment we adapt observers to two stimuli that are matched in the mean and amplitude of modulation they offer to the cone classes and to the cardinal opponent mechanisms, but that differ in chromatic appearance, and hence their modulation of later colour mechanisms. Chromatic discrimination thresholds after adaptation to these two stimuli differ along intermediate directions in colour space, and we argue that these differences reveal the adaptation response of central colour mechanisms. In the third experiment we demonstrate similar adaptation using the same stimuli, measured with reaction times rather than thresholds. In the final experiment, we measure the degree to which colour constancy is achieved as a function of time in a simulated stimulus environment in which the illuminant changes periodically. We find that perfect constancy is not achieved instantaneously after an illuminant chromaticity shift and that constancy of colour appearance judgements increases over several seconds

Illuminant Estimation By Deep Learning

Computational color constancy refers to the problem of estimating the color of the scene illumination in a color image, followed by color correction of the image through a white balancing process so that the colors of the image will be viewed as if the image was captured under a neutral white light source, and hence producing a plausible natural looking image. The illuminant estimation part is still a challenging task due to the ill-posed nature of the problem, and many methods have been proposed in the literature while each follows a certain approach in an attempt to improve the performance of the Auto-white balancing system for accurately estimating the illumination color for better image correction. These methods can typically be categorized into static-based and learning-based methods. Most of the proposed methods follow the learning-based approach because of its higher estimation accuracy compared to the former which relies on simple assumptions. While many of those learning-based methods show a satisfactory performance in general, they are built upon extracting handcrafted features which require a deep knowledge of the color image processing. More recent learning-based methods have shown higher improvements in illuminant estimation through using Deep Learning (DL) systems presented by the Convolutional Neural Networks (CNNs) that automatically learned to extract useful features from the given image dataset. In this thesis, we present a highly effective Deep Learning approach which treats the illuminant estimation problem as an illuminant classification task by learning a Convolutional Neural Network to classify input images belonging to certain pre-defined illuminant classes. Then, the output of the CNN which is in the form of class probabilities is used for computing the illuminant color estimate. Since training a deep CNN requires large number of training examples to avoid the “overfitting” problem, most of the recent CNN-based illuminant estimation methods attempted to overcome the limited number of images in the benchmark illuminant estimation dataset by sampling input images to multiple smaller patches as a way of data augmentation, but this can adversely affect the CNN training performance because some of these patches may not contain any semantic information and therefore, can be considered as noisy examples for the CNN that can lead to estimation ambiguity. However, in this thesis, we propose a novel approach for dataset augmentation through synthesizing images with different illuminations using the ground-truth illuminant color of other training images, which enhanced the performance of the CNN training compared to similar previous methods. Experimental results on the standard illuminant estimation benchmark dataset show that the proposed solution outperforms most of the previous illuminant estimation methods and show a competitive performance to the state-of-the-art methods

Colour constancy in simple and complex scenes

PhD ThesisColour constancy is defined as the ability to perceive the surface colours of objects within scenes as approximately constant through changes in scene illumination. Colour constancy in real life functions so seamlessly that most people do not realise that the colour of the light emanating from an object can change markedly throughout the day. Constancy measurements made in simple scenes constructed from flat coloured patches do not produce constancy of this high degree. The question that must be asked is: what are the features of everyday scenes that improve constancy? A novel technique is presented for testing colour constancy. Results are presented showing measurements of constancy in simple and complex scenes. More specifically, matching experiments are performed for patches against uniform and multi-patch backgrounds, the latter of which provide colour contrast. Objects created by the addition of shape and 3-D shading information are also matched against backgrounds consisting of matte reflecting patches. In the final set of experiments observers match detailed depictions of objects - rich in chromatic contrast, shading, mutual illumination and other real life features - within depictions of real life scenes. The results show similar performance across the conditions that contain chromatic contrast, although some uncertainty still remains as to whether the results are indicative of human colour constancy performance or to sensory match capabilities. An interesting division exists between patch matches performed against uniform and multi-patch backgrounds that is manifested as a shift in CIE xy space. A simple model of early chromatic processes is proposed and examined in the context of the results

The Effects of Multi-channel Visible Spectrum Imaging on Perceived Spatial Image Quality and Color Reproduction Accuracy

Two paired-comparison psychophysical experiments were performed. The stimuli consisted of six image types resultingfrom several multispectral image-capture and reconstruction techniques. A seventh image type, color-managed images from a high-end consumer camera, was also included in thefirst experiment to compare the accuracy of commercial RGB imaging. The images were evaluated under simulated daylight (6800K) and incandescent (2700K) illumination. The first experiment evaluated color reproduction accuracy. Under simulated daylight, the subjects judged all of the images to have the same color accuracy, except the consumer camera image which was significantly worse. Under incandescent illumination, all the images, including the consumer camera, had equivalent performance. The second experiment evaluated image quality. The results of this experiment were highly target dependent. A subsequent image registration experiment showed that the results of the image quality experiment were affected by image registration to some degree. An analysis of the color reproduction accuracy and image quality experiments combined showed that the consumer camera image type was preferred the least over all. The most preferred image types were the thirty-one-channel image type and both six-channel image types created using RGB filters along with a Wratten filter, with eigenvector analysis and pseudo-inverse transformations