Estimation of illuminants from color signals of illuminated objects

Color constancy is the ability of the human visual systems to discount the effect of the illumination and to assign approximate constant color descriptions to objects. This ability has long been studied and widely applied to many areas such as color reproduction and machine vision, especially with the development of digital color processing. This thesis work makes some improvements in illuminant estimation and computational color constancy based on the study and testing of existing algorithms. During recent years, it has been noticed that illuminant estimation based on gamut comparison is efficient and simple to implement. Although numerous investigations have been done in this field, there are still some deficiencies. A large part of this thesis has been work in the area of illuminant estimation through gamut comparison. Noting the importance of color lightness in gamut comparison, and also in order to simplify three-dimensional gamut calculation, a new illuminant estimation method is proposed through gamut comparison at separated lightness levels. Maximum color separation is a color constancy method which is based on the assumption that colors in a scene will obtain the largest gamut area under white illumination. The method was further derived and improved in this thesis to make it applicable and efficient. In addition, some intrinsic questions in gamut comparison methods, for example the relationship between the color space and the application of gamut or probability distribution, were investigated. Color constancy methods through spectral recovery have the limitation that there is no effective way to confine the range of object spectral reflectance. In this thesis, a new constraint on spectral reflectance based on the relative ratios of the parameters from principal component analysis (PCA) decomposition is proposed. The proposed constraint was applied to illuminant detection methods as a metric on the recovered spectral reflectance. Because of the importance of the sensor sensitivities and their wide variation, the influence from the sensor sensitivities on different kinds of illuminant estimation methods was also studied. Estimation method stability to wrong sensor information was tested, suggesting the possible solution to illuminant estimation on images with unknown sources. In addition, with the development of multi-channel imaging, some research on illuminant estimation for multi-channel images both on the correlated color temperature (CCT) estimation and the illuminant spectral recovery was performed in this thesis. All the improvement and new proposed methods in this thesis are tested and compared with those existing methods with best performance, both on synthetic data and real images. The comparison verified the high efficiency and implementation simplicity of the proposed methods

Investigations into colour constancy by bridging human and computer colour vision

PhD ThesisThe mechanism of colour constancy within the human visual system has long been of great interest to researchers within the psychophysical and image processing communities. With the maturation of colour imaging techniques for both scientific and artistic applications the importance of colour capture accuracy has consistently increased. Colour offers a great deal more information for the viewer than grayscale imagery, ranging from object detection to food ripeness and health estimation amongst many others. However these tasks rely upon the colour constancy process in order to discount scene illumination to allow these tasks to be carried out. Psychophysical studies have attempted to uncover the inner workings of this mechanism, which would allow it to be reproduced algorithmically. This would allow the development of devices which can eventually capture and perceive colour in the same manner as a human viewer. These two communities have approached this challenge from opposite ends, and as such very different and largely unconnected approaches. This thesis investigates the development of studies and algorithms which bridge the two communities. Utilising findings from psychophysical studies as inspiration to firstly improve an existing image enhancement algorithm. Results are then compared to state of the art methods. Then, using further knowledge, and inspiration, of the human visual system to develop a novel colour constancy approach. This approach attempts to mimic and replicate the mechanism of colour constancy by investigating the use of a physiological colour space and specific scene contents to estimate illumination. Performance of the colour constancy mechanism within the visual system is then also investigated. The performance of the mechanism across different scenes and commonly and uncommonly encountered illuminations is tested. The importance of being able to bridge these two communities, with a successful colour constancy method, is then further illustrated with a case study investigating the human visual perception of the agricultural produce of tomatoes.EPSRC DTA: Institute of Neuroscience, Newcastle University

Estimating varying illuminant colours in images

Colour Constancy is the ability to perceive colours independently of varying illumi-nation colour. A human could tell that a white t-shirt was indeed white, even under the presence of blue or red illumination. These illuminant colours would actually make the reflectance colour of the t-shirt bluish or reddish. Humans can, to a good extent, see colours constantly. Getting a computer to achieve the same goal, with a high level of accuracy has proven problematic. Particularly if we wanted to use colour as a main cue in object recognition. If we trained a system on object colours under one illuminant and then tried to recognise the objects under another illuminant, the system would likely fail. Early colour constancy algorithms assumed that an image contains a single uniform illuminant. They would then attempt to estimate the colour of the illuminant to apply a single correction to the entire image. It’s not hard to imagine a scenario where a scene is lit by more than one illuminant. If we take the case of an outdoors scene on a typical summers day, we would see objects brightly lit by sunlight and others that are in shadow. The ambient light in shadows is known to be a different colour to that of direct sunlight (bluish and yellowish respectively). This means that there are at least two illuminant colours to be recovered in this scene. This thesis focuses on the harder case of recovering the illuminant colours when more than one are present in a scene. Early work on this subject made the empirical observation that illuminant colours are actually very predictable compared to surface colours. Real-world illuminants tend not to be greens or purples, but rather blues, yellows and reds. We can think of an illuminant mapping as the function which takes a scene from some unknown illuminant to a known illuminant. We model this mapping as a simple multiplication of the Red, Green and Blue channels of a pixel. It turns out that the set of realistic mappings approximately lies on a line segment in chromaticity space. We propose an algorithm that uses this knowledge and only requires two pixels of the same surface under two illuminants as input. We can then recover an estimate for the surface reflectance colour, and subsequently the two illuminants. Additionally in this thesis, we propose a more robust algorithm that can use vary-ing surface reflectance data in a scene. One of the most successful colour constancy algorithms, known Gamut Mappping, was developed by Forsyth (1990). He argued that the illuminant colour of a scene naturally constrains the surfaces colours that are possible to perceive. We couldn’t perceive a very chromatic red under a deep blue illuminant. We introduce our multiple illuminant constraint in a Gamut Mapping context and are able to further improve it’s performance. The final piece of work proposes a method for detecting shadow-edges, so that we can automatically recover estimates for the illuminant colours in and out of shadow. We also formulate our illuminant estimation algorithm in a voting scheme, that probabilistically chooses an illuminant estimate on both sides of the shadow edge. We test the performance of all our algorithms experimentally on well known datasets, as well as our new proposed shadow datasets

Neurophysiology

Contains research objectives and summary of research on seventeen research projects and reports on four research projects.National Institutes of Health (Grant 5 TOl EY00090-02)Bell Telephone Laboratories, Inc. (Grant)National Institutes of Health (Grant 5 ROI EY01149-03)National Institutes of Health (Grant NS 12307-01)National Institutes of Health (Grant 1 K04 NS00010

Modelling Colour Appearance: Applications in Skin Image Perception

Humans are trichromatic, and yet their perception of colours is rich and complex. The research presented in this thesis explores the process of colour appearance of uniform patches and natural polychromatic stimuli. This is done through the measurement and analysis of the achromatic locus (Chapter 2), modelling of chromatic adaptation in a large dataset of unique hues settings (Chapter 3), and measurement of thresholds for uniform and polychromatic stimuli derived from simulated skin images (Chapter 4). Chapter 2 proposes a novel navigation scheme based on unique hues for traversing colour space. The results show that when colour adjustments are made using this novel scheme, the variability of achromatic settings made by observers is reduced compared to the classical method of making colour adjustments along the cardinal axes of the CIELUV colour space. This result holds across the tested luminance levels (5,20,50 cd/m^2) in each of the three tested ambient illumination conditions – dark, simulated daylight and cool white fluorescent lighting. The analysis also shows that the direction of maximum variance of the achromatic settings lies along the daylight locus. Chapter 3 evaluates models of chromatic adaptation by using unique hues settings measured under different ambient illumination conditions. It is shown that a simple diagonal model in cone excitation space is the most efficient in terms of the trade-off between accuracy and degrees of freedom. It is also found that diagonal and linear models show similar performances, reiterating their theoretical equivalence. Performances of these diagonalisable models are found to be worse for UR and UG unique hue planes compared to UY and UB planes. Chapter 4 presents a set of three experiments reporting estimations of perceptual thresholds for polychromatic and uniform stimuli in a 3-D chromaticity-luminance colour space. The first experiment reports thresholds for simulated skin images and uniform stimuli of the corresponding mean CIELAB colour. The second and third experiments investigate the effect of ambient illumination and the location of the stimuli in colour space. The thresholds for the polychromatic stimuli are found to be consistently higher than those for the uniform patches, for both the chromatic, and the luminance projections. The area of the chromaticity ellipses shows a gradual increase with distance from the illuminant chromaticity. The orientations of these ellipses for simulated skin are found to align with the vector joining the mean patch chromaticity and the illuminant chromaticity

Optical MIMO communication systems under illumination constraints

Technology for wireless information access has enabled innovation of 'smart' portable consumer devices. These have been widely adopted and have become an integral part of our daily lives. They need ubiquitous connectivity to the internet to provide value added services, maximize their functionality and create a smarter world to live in. Cisco's visual networking index currently predicts wireless data consumption to increase by 61% per year. This will put additional stress on the already stressed wireless access network infrastructure creating a phenomenon called 'spectrum crunch'. At the same time, the solid state devices industry has made remarkable advances in energy efficient light-emitting-diodes (LED). The lighting industry is rapidly adopting LEDs to provide illumination in indoor spaces. Lighting fixtures are positioned to support human activities and thus are well located to act as wireless access points. The visible spectrum (380 nm - 780 nm) is yet unregulated and untapped for wireless access. This provides unique opportunity to upgrade existing lighting infrastructure and create a dense grid of small cells by using this additional 'optical' wireless bandwidth. Under the above model, lighting fixtures will service dual missions of illumination and access points for optical wireless communication (OWC). This dissertation investigates multiple-input multiple-output (MIMO) optical wireless broadcast system under unique constraints imposed by the optical channel and illumination requirements. Sample indexed spatial orthogonal frequency division multiplexing (SIS-OFDM) and metameric modulation (MM) are proposed to achieve higher spectral efficiency by exploiting dimensions of space and color respectively in addition to time and frequency. SIS-OFDM can provide significant additional spectral efficiency of up to (Nsc/2 - 1) x k bits/sym where Nsc is total number of subcarriers and k is number of bits per underlying spatial modulation symbol. MM always generates the true requested illumination color and has the potential to provide better color rendering by incorporating multiple LEDs. A normalization framework is then developed to analyze performance of optical MIMO imaging systems. Performance improvements of up to 45 dB for optical systems have been achieved by decorrelating spatially separate links by incorporating an imaging receiver. The dissertation also studies the impact of visual perception on performance of color shift keying as specified in IEEE 802.15.7 standard. It shows that non-linearity for a practical system can have a performance penalty of up to 15 dB when compared to the simplified linear system abstraction as proposed in the standard. Luminous-signal-to-noise ratio, a novel metric is introduced to compare performance of optical modulation techniques operating at same illumination intensity. The dissertation then introduces singular value decomposition based OWC system architecture to incorporate illumination constraints independent of communication constraints in a MIMO system. It then studies design paradigm for a multi-colored wavelength division multiplexed indoor OWC system

Semantik renk değişmezliği

Color constancy aims to perceive the actual color of an object, disregarding the effectof the light source. Recent works showed that utilizing the semantic information inan image enhances the performance of the computational color constancy methods.Considering the recent success of the segmentation methods and the increased numberof labeled images, we propose a color constancy method that combines individualilluminant estimations of detected objects which are computed using the classes of theobjects and their associated colors. Then we introduce a weighting system that valuesthe applicability of the object classes to the color constancy problem. Lastly, weintroduce another metric expressing the detected object and how well it fits the learnedmodel of its class. Finally, we evaluate our proposed method on a popular colorconstancy dataset, confirming that each weight addition enhances the performanceof the global illuminant estimation. Experimental results show promising results,outperforming the conventional methods while competing with the state of the artmethods.--M.S. - Master of Scienc

Practical Measurement and Reconstruction of Spectral Skin Reflectance

We present two practical methods for measurement of spectral skin reflectance suited for live subjects, and drive a spectral BSSRDF model with appropriate complexity to match skin appearance in photographs, including human faces. Our primary measurement method employs illuminating a subject with two complementary uniform spectral illumination conditions using a multispectral LED sphere to estimate spatially varying parameters of chromophore concentrations including melanin and hemoglobin concentration, melanin blend-type fraction, and epidermal hemoglobin fraction. We demonstrate that our proposed complementary measurements enable higher-quality estimate of chromophores than those obtained using standard broadband illumination, while being suitable for integration with multiview facial capture using regular color cameras. Besides novel optimal measurements under controlled illumination, we also demonstrate how to adapt practical skin patch measurements using a hand-held dermatological skin measurement device, a Miravex Antera 3D camera, for skin appearance reconstruction and rendering. Furthermore, we introduce a novel approach for parameter estimation given the measurements using neural networks which is significantly faster than a lookup table search and avoids parameter quantization. We demonstrate high quality matches of skin appearance with photographs for a variety of skin types with our proposed practical measurement procedures, including photorealistic spectral reproduction and renderings of facial appearance

The asymmetries of colour constancy as determined through illumination discrimination using tuneable LED light sources

PhD ThesisThe light reflected from object surfaces changes with the spectral content of the illumination. Despite these changes, the human visual system tends to keep the colours of surfaces constant, a phenomenon known as colour constancy. Colour constancy is known to be imperfect under many conditions; however, it is unknown whether the underlying mechanisms present in the retina and the cortex are optimised for the illuminations under which they have evolved, namely, natural daylights, or for particular objects. A novel method of measuring colour constancy, by illumination discrimination, is presented and explored. This method, unlike previous methods of measuring colour constancy, allows the testing of multiple, real, illuminations with arbitrary spectral content, through the application of tuneable, multi-channel LED light sources. Data from both real scenes, under real illuminations, and computer simulations are presented which support the hypothesis that the visual system maintains higher levels of colour constancy for daylight illumination changes, and in particular in the “bluer” direction, which are also the changes most frequent in nature. The low-level cone inputs for various experimental scenes are examined which challenge all traditional theories of colour constancy supporting the conclusions that higher-level mechanisms of colour constancy are biased for particular illuminations. Furthermore, real and simulated neutral (grey) surfaces are shown to affect levels of colour constancy. Moreover, the conceptual framework for discussing colour constancy with respect to emergent LED light sources is discussed.EPSR

The role of chromatic texture and 3D shape in colour discrimination, memory colour, and colour constancy of natural objects

The primary goal of this work was to investigate colour perception in a natural environment and to contribute to the understanding of how cues to familiar object identity influence colour appearance. A large number of studies on colour appearance employ 2D uniformly coloured patches, discarding perceptual cues such as binocular disparity, 3D luminance shading, mutual reflection, and glossy highlights are integral part of a natural scene. Moreover, natural objects possess specific cues that help our recognition (shape, surface texture or colour distribution). The aim of the first main experiment presented in this thesis was to understand the effect of shape on (1) memory colour under constant and varying illumination and on (2) colour constancy for uniformly coloured stimuli. The results demonstrated the existence of a range of memory colours associated with a familiar object, the size of which was strongly object-shape-dependent. For all objects, memory retrieval was significantly faster for object-diagnostic shape relative to generic shapes. Based on two successive controls, the author suggests that shape cues to the object identity affect the range of memory colour proportionally to the original object chromatic distribution. The second experiment examined the subject’s accuracy and precision in adjusting a stimulus colour to its typical appearance. Independently on the illuminant, results showed that memory colour accuracy and precision were enhanced by the presence of chromatic textures, diagnostic shapes, or 3D configurations with a strong interaction between diagnosticity and dimensionality of the shape. Hence, more cues to the object identity and more natural stimuli facilitate the observers in accessing their colour information from memory. A direct relationship was demonstrated between chromatic surface representation, object’s physical properties, and identificability and dimensionality of shape on memory colour accuracy, suggesting high-level mechanisms. Chromatic textures facilitated colour constancy. The third and fourth experiments tested the subject’s ability to discriminate between two chromatic stimuli in a simultaneous and successive 2AFC task, respectively. Simultaneous discrimination threshold performances for polychromatic surfaces were only due to low-level mechanism of the stimulus, whereas in the successive discrimination, i.e. when memory is involved, high-level mechanisms were established. The effect of shape was strongly task- dependent and was modulate by the object memory colour. These findings together with the strong interaction between chromatic cues and shape cues to the object identity lead to the conclusion that high level mechanisms linked to object recognition facilitated both tasks. Hence, the current thesis presents new findings on memory colour and colour constancy presented in a natural context and demonstrates the effect of high-level mechanisms in chromatic discrimination as a function of cues to the object identity such as shape and texture. This work contributes to a deeper understanding of colour perception and object recognition in the natural world.EThOS - Electronic Theses Online ServiceGBUnited Kingdo