Chromatic Illumination Discrimination Ability Reveals that Human Colour Constancy Is Optimised for Blue Daylight Illuminations

The phenomenon of colour constancy in human visual perception keeps surface colours constant, despite changes in their reflected light due to changing illumination. Although colour constancy has evolved under a constrained subset of illuminations, it is unknown whether its underlying mechanisms, thought to involve multiple components from retina to cortex, are optimised for particular environmental variations. Here we demonstrate a new method for investigating colour constancy using illumination matching in real scenes which, unlike previous methods using surface matching and simulated scenes, allows testing of multiple, real illuminations. We use real scenes consisting of solid familiar or unfamiliar objects against uniform or variegated backgrounds and compare discrimination performance for typical illuminations from the daylight chromaticity locus (approximately blue-yellow) and atypical spectra from an orthogonal locus (approximately red-green, at correlated colour temperature 6700 K), all produced in real time by a 10-channel LED illuminator. We find that discrimination of illumination changes is poorer along the daylight locus than the atypical locus, and is poorest particularly for bluer illumination changes, demonstrating conversely that surface colour constancy is best for blue daylight illuminations. Illumination discrimination is also enhanced, and therefore colour constancy diminished, for uniform backgrounds, irrespective of the object type. These results are not explained by statistical properties of the scene signal changes at the retinal level. We conclude that high-level mechanisms of colour constancy are biased for the blue daylight illuminations and variegated backgrounds to which the human visual system has typically been exposed

Investigations into colour constancy by bridging human and computer colour vision

PhD ThesisThe mechanism of colour constancy within the human visual system has long been of great interest to researchers within the psychophysical and image processing communities. With the maturation of colour imaging techniques for both scientific and artistic applications the importance of colour capture accuracy has consistently increased. Colour offers a great deal more information for the viewer than grayscale imagery, ranging from object detection to food ripeness and health estimation amongst many others. However these tasks rely upon the colour constancy process in order to discount scene illumination to allow these tasks to be carried out. Psychophysical studies have attempted to uncover the inner workings of this mechanism, which would allow it to be reproduced algorithmically. This would allow the development of devices which can eventually capture and perceive colour in the same manner as a human viewer. These two communities have approached this challenge from opposite ends, and as such very different and largely unconnected approaches. This thesis investigates the development of studies and algorithms which bridge the two communities. Utilising findings from psychophysical studies as inspiration to firstly improve an existing image enhancement algorithm. Results are then compared to state of the art methods. Then, using further knowledge, and inspiration, of the human visual system to develop a novel colour constancy approach. This approach attempts to mimic and replicate the mechanism of colour constancy by investigating the use of a physiological colour space and specific scene contents to estimate illumination. Performance of the colour constancy mechanism within the visual system is then also investigated. The performance of the mechanism across different scenes and commonly and uncommonly encountered illuminations is tested. The importance of being able to bridge these two communities, with a successful colour constancy method, is then further illustrated with a case study investigating the human visual perception of the agricultural produce of tomatoes.EPSRC DTA: Institute of Neuroscience, Newcastle University

Learning to Recover Spectral Reflectance from RGB Images

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL

Reflectance, illumination, and appearance in color constancy

We studied color constancy using a pair of identical 3-D Color Mondrian displays. We viewed one 3-D Mondrian in nearly uniform illumination, and the other in directional, nonuniform illumination. We used the three dimensional structures to modulate the light falling on the painted surfaces. The 3-D structures in the displays were a matching set of wooden blocks. Across Mondrian displays, each corresponding facet had the same paint on its surface. We used only 6 chromatic, and 5 achromatic paints applied to 104 block facets. The 3-D blocks add shadows and multiple reflections not found in flat Mondrians. Both 3-D Mondrians were viewed simultaneously, side-by-side. We used two techniques to measure correlation of appearance with surface reflectance. First, observers made magnitude estimates of changes in the appearances of identical reflectances. Second, an author painted a watercolor of the 3-D Mondrians. The watercolor's reflectances quantified the changes in appearances. While constancy generalizations about illumination and reflectance hold for flat Mondrians, they do not for 3-D Mondrians. A constant paint does not exhibit perfect color constancy, but rather shows significant shifts in lightness, hue and chroma in response to the structure in the nonuniform illumination. Color appearance depends on the spatial information in both the illumination and the reflectances of objects. The spatial information of the quanta catch from the array of retinal receptors generates sensations that have variable correlation with surface reflectance. Models of appearance in humans need to calculate the departures from perfect constancy measured here. This article provides a dataset of measurements of color appearances for computational models of sensation. © 2014 McCann, Parraman and Rizzi

Determinants of colour constancy

Colour constancy describes the ability of our visual system to keep colour percepts stable through illumination changes. This is an outstanding feat given that in the retinal image surface and illuminant properties are conflated. Still, in our everyday lives we are able attribute stable colour-labels to objects to make communication economic and efficient. Past research shows colour constancy to be imperfect, compensating for 40% and 80% of the illumination change. While different constancy determinants are suggested, no carefully controlled study shows perfect constancy. The first study presented here addresses the issue of imperfect constancy by investigating colour constancy in a cue rich environment, using a task that resembles our everyday experience with colours. Participants were asked to recall the colour of unique personal objects in natural environment under four chromatic illuminations. This approach yielded perfect colour constancy. The second study investigated the relation between illumination discrimination and chromatic detection. Recent studies using an illumination discrimination paradigm suggest that colour constancy is optimized for bluish daylight illuminations. Because it is not clear if illumination discrimination is directly related to colour constancy or is instead explained by sensitivity to changes in chromaticity of different hues, thresholds for illumination discrimination and chromatic detection for the same 12 illumination hues were compared. While the reported blue bias could be replicated, thresholds for illumination discrimination and chromatic detection were highly related, indicating that lower sensibility towards bluish hues is not exclusive to illumination discrimination. Accompanying the second study, the third study investigated the distribution of colour constancy for 40 chromatic illuminations of different hue using achromatic adjustments and colour naming. These measurements were compared to several determinants of colour constancy, including the daylight locus, colour categories, illumination discrimination, chromatic detection, relational colour constancy and metameric mismatching. In accordance with the observations in study 2, achromatic adjustments revealed a bias towards bluish daylight illumination. This blue bias and naming consistency explained most of the variance in achromatic adjustments, while illumination discrimination was not directly related to colour constancy. The fourth study examined colour memory biases. Past research shows that colours of objects are remembered as being more saturated than they are perceived. These works often used natural objects that exist in a variety of colour and hue, such as grass or bananas. The approach presented here directly compared perceived and memorized colours for unique objects, used also in the first study, and confirmed the previous findings that on average, objects were remembered more saturated than they were perceived.Farbkonstanz beschreibt die Fähigkeit unseres visuellen Systems Farbeindrücke unter Beleuchtungsänderungen beständig zu halten. Dies ist eine außergewöhnliche Leistung, wenn man in Betracht zieht, dass in dem Lichtsignal welches das Auge erreicht Eigenschaften der Beleuchtung und der Oberflächen konfundiert sind. Trotz dieser Problematik sind wir in unserem alltäglichen Leben in der Lage Objekten stabile Farbnamen zuzuordnen, und damit unsere Kommunikation effizient und ökonomisch zu gestalten. Bisherige Studien zur Farbkonstanz berichten jedoch, dass Farbkonstanz nicht perfekt ist, Beleuchtungswechsel wurden nur zwischen 40-80% kompensiert. Während unterschiedliche Determinanten der Farbkonstanz vorgeschlagen wurden, konnte bisher keine sorgfältig kontrollierte Studie perfekte Farbkonstanz zeigen. In der ersten Studie dieser Arbeit wurde dieser Aspekt untersucht, indem Farbkonstanz in einer hinweisreichen Umgebung unter Verwendung einer Aufgabe, die möglichst präzise unserer alltäglichen Erfahrung im Umgang mit Farben wiederspiegelt, gemessen wurde. Die Versuchsteilnehmer wurden aufgefordert die Farbe eines spezifischen persönlichen Gegenstandes unter vier farbigen Beleuchtungen aus dem Gedächtnis abzurufen. Unter Verwendung dieses Ansatzes konnte perfekte Farbkonstanz erreicht werden. Die zweite Studie untersuchte die Beziehung zwischen Beleuchtungs-Diskrimination und chromatischer Detektion. Die Ergebnisse von kürzlich veröffentlichten Forschungsarbeiten, welche ein Beleuchtungs-Diskriminations-Paradigma verwendeten, zeigen das diese Diskrimination in Richtung bläulicher Beleuchtung verzerrt ist. Daraus wurde geschlossen, das Farbkonstanz für bläuliche Tageslicht-Beleuchtungen optimiert ist . Da es aber nicht klar ist, ob Beleuchtungs-Diskrimination in direkter Beziehung zur Farbkonstanz steht, oder aber vielmehr auf die Sensitivität für chromatische Veränderungen zurückführen ist, wurden Wahrnehmungsschwellen für Beleuchtungs-Diskrimination und chromatische Detektion für die selben 12 Beleuchtungsfarben gemessen und verglichen. Während die bereits berichtete Verzerrung in Richtung der bläulichen Tageslichtbeleuchtung repliziert werden konnte, wurde ebenfalls eine hoher Zusammenhang zwischen chromatischer Detektion und Beleuchtungs-Diskrimination gefunden, welcher darauf hinweist, dass die Verzerrung in Richtung bläulicher Farben keine exklusive Eigenschaft der Beleuchtung-Diskrimination ist. Anknüpfend an die zweite Studie wurde in der dritten Studie die Verteilung von Farbkonstanz über 40 chromatische Beleuchtungen anhand von achromatischen Einstellungen und Farbbenennung untersucht. Farbkonstanz wurde auf ihren Zusammenhang zu mehreren Determinanten der Farbkonstanz überprüft, unter anderem mit Tageslichtvariationen, Farbkategorien, Beleuchtungs-Diskrimination, relationaler Farbkonstanz und metameric mismatching. In Übereinstimmung mit der zweiten Studie wurde auch für achromatische Einstellungen eine Verzerrung in Richtung bläulicher Tageslichtbeleuchtungen gefunden. Diese Verzerrung und der Konsensus der Beleuchtungsbenennung erklärten den Großteil der Varianz der achromatischen Einstellungen, während Beleuchtungs-Diskrimination nicht in direkter Verbindung zur Farbkonstanz stand. In der vierten Studie wurden Verzerrungen des Farbgedächtnisses untersucht. Frühere Studien berichten, dass Objektfarben häufig gesättigter erinnert werden als sie tatsächlich wahrgenommen werden. In diesen Studien wurden häufig natürliche Objekte verwendet, die in einer Vielzahl an Farbtönen und Sättigungen existieren, wie beispielsweise Gras oder Bananen. In dem hier präsentierten Ansatz wurden Farbwahlen aus dem Gedächtnis mit Farbwahlen der direkten Objektwahrnehmung für persönliche, spezifische Objekte, die auch schon in der ersten Studie verwendet wurden, verglichen. Die Ergebnisse der vorherigen Studien konnten für diese Objekte repliziert werden: Im Durchschnitt wurden Objektfarben gesättigter erinnert als das Objekt im direkten Vergleich wahrgenommen wurde

A Dataset of Multi-Illumination Images in the Wild

Collections of images under a single, uncontrolled illumination have enabled the rapid advancement of core computer vision tasks like classification, detection, and segmentation. But even with modern learning techniques, many inverse problems involving lighting and material understanding remain too severely ill-posed to be solved with single-illumination datasets. To fill this gap, we introduce a new multi-illumination dataset of more than 1000 real scenes, each captured under 25 lighting conditions. We demonstrate the richness of this dataset by training state-of-the-art models for three challenging applications: single-image illumination estimation, image relighting, and mixed-illuminant white balance.Comment: ICCV 201