Investigations into colour constancy by bridging human and computer colour vision

PhD ThesisThe mechanism of colour constancy within the human visual system has long been of great interest to researchers within the psychophysical and image processing communities. With the maturation of colour imaging techniques for both scientific and artistic applications the importance of colour capture accuracy has consistently increased. Colour offers a great deal more information for the viewer than grayscale imagery, ranging from object detection to food ripeness and health estimation amongst many others. However these tasks rely upon the colour constancy process in order to discount scene illumination to allow these tasks to be carried out. Psychophysical studies have attempted to uncover the inner workings of this mechanism, which would allow it to be reproduced algorithmically. This would allow the development of devices which can eventually capture and perceive colour in the same manner as a human viewer. These two communities have approached this challenge from opposite ends, and as such very different and largely unconnected approaches. This thesis investigates the development of studies and algorithms which bridge the two communities. Utilising findings from psychophysical studies as inspiration to firstly improve an existing image enhancement algorithm. Results are then compared to state of the art methods. Then, using further knowledge, and inspiration, of the human visual system to develop a novel colour constancy approach. This approach attempts to mimic and replicate the mechanism of colour constancy by investigating the use of a physiological colour space and specific scene contents to estimate illumination. Performance of the colour constancy mechanism within the visual system is then also investigated. The performance of the mechanism across different scenes and commonly and uncommonly encountered illuminations is tested. The importance of being able to bridge these two communities, with a successful colour constancy method, is then further illustrated with a case study investigating the human visual perception of the agricultural produce of tomatoes.EPSRC DTA: Institute of Neuroscience, Newcastle University

Semantik renk değişmezliği

Color constancy aims to perceive the actual color of an object, disregarding the effectof the light source. Recent works showed that utilizing the semantic information inan image enhances the performance of the computational color constancy methods.Considering the recent success of the segmentation methods and the increased numberof labeled images, we propose a color constancy method that combines individualilluminant estimations of detected objects which are computed using the classes of theobjects and their associated colors. Then we introduce a weighting system that valuesthe applicability of the object classes to the color constancy problem. Lastly, weintroduce another metric expressing the detected object and how well it fits the learnedmodel of its class. Finally, we evaluate our proposed method on a popular colorconstancy dataset, confirming that each weight addition enhances the performanceof the global illuminant estimation. Experimental results show promising results,outperforming the conventional methods while competing with the state of the artmethods.--M.S. - Master of Scienc

Multispectral Imaging For Face Recognition Over Varying Illumination

This dissertation addresses the advantage of using multispectral narrow-band images over conventional broad-band images for improved face recognition under varying illumination. To verify the effectiveness of multispectral images for improving face recognition performance, three sequential procedures are taken into action: multispectral face image acquisition, image fusion for multispectral and spectral band selection to remove information redundancy. Several efficient image fusion algorithms are proposed and conducted on spectral narrow-band face images in comparison to conventional images. Physics-based weighted fusion and illumination adjustment fusion make good use of spectral information in multispectral imaging process. The results demonstrate that fused narrow-band images outperform the conventional broad-band images under varying illuminations. In the case where multispectral images are acquired over severe changes in daylight, the fused images outperform conventional broad-band images by up to 78%. The success of fusing multispectral images lies in the fact that multispectral images can separate the illumination information from the reflectance of objects which is impossible for conventional broad-band images. To reduce the information redundancy among multispectral images and simplify the imaging system, distance-based band selection is proposed where a quantitative evaluation metric is defined to evaluate and differentiate the performance of multispectral narrow-band images. This method is proved to be exceptionally robust to parameter changes. Furthermore, complexity-guided distance-based band selection is proposed using model selection criterion for an automatic selection. The performance of selected bands outperforms the conventional images by up to 15%. From the significant performance improvement via distance-based band selection and complexity-guided distance-based band selection, we prove that specific facial information carried in certain narrow-band spectral images can enhance face recognition performance compared to broad-band images. In addition, both algorithms are proved to be independent to recognition engines. Significant performance improvement is achieved by proposed image fusion and band selection algorithms under varying illumination including outdoor daylight conditions. Our proposed imaging system and image processing algorithms lead to a new avenue of automatic face recognition system towards a better recognition performance than the conventional peer system over varying illuminations

Accurate Colour Reproduction of Human Face using 3D Printing Technology

The colour of the face is one of the most significant factors in appearance and perception of an individual. With the rapid development of colour 3D printing technology and 3D imaging acquisition techniques, it is possible to achieve skin colour reproduction with the application of colour management. However, due to the complicated skin structure with uneven and non-uniform surface, it is challenging to obtain accurate skin colour appearance and reproduce it faithfully using 3D colour printers. The aim of this study was to improve the colour reproduction accuracy of the human face using 3D printing technology. A workflow of 3D colour image reproduction was developed, including 3D colour image acquisition, 3D model manipulation, colour management, colour 3D printing, postprocessing and colour reproduction evaluation. Most importantly, the colour characterisation methods for the 3D imaging system and the colour 3D printer were comprehensively investigated for achieving higher accuracy

Algorithms for the enhancement of dynamic range and colour constancy of digital images & video

One of the main objectives in digital imaging is to mimic the capabilities of the human eye, and perhaps, go beyond in certain aspects. However, the human visual system is so versatile, complex, and only partially understood that no up-to-date imaging technology has been able to accurately reproduce the capabilities of the it. The extraordinary capabilities of the human eye have become a crucial shortcoming in digital imaging, since digital photography, video recording, and computer vision applications have continued to demand more realistic and accurate imaging reproduction and analytic capabilities. Over decades, researchers have tried to solve the colour constancy problem, as well as extending the dynamic range of digital imaging devices by proposing a number of algorithms and instrumentation approaches. Nevertheless, no unique solution has been identified; this is partially due to the wide range of computer vision applications that require colour constancy and high dynamic range imaging, and the complexity of the human visual system to achieve effective colour constancy and dynamic range capabilities. The aim of the research presented in this thesis is to enhance the overall image quality within an image signal processor of digital cameras by achieving colour constancy and extending dynamic range capabilities. This is achieved by developing a set of advanced image-processing algorithms that are robust to a number of practical challenges and feasible to be implemented within an image signal processor used in consumer electronics imaging devises. The experiments conducted in this research show that the proposed algorithms supersede state-of-the-art methods in the fields of dynamic range and colour constancy. Moreover, this unique set of image processing algorithms show that if they are used within an image signal processor, they enable digital camera devices to mimic the human visual system s dynamic range and colour constancy capabilities; the ultimate goal of any state-of-the-art technique, or commercial imaging device

Evaluation and optimal design of spectral sensitivities for digital color imaging

The quality of an image captured by color imaging system primarily depends on three factors: sensor spectral sensitivity, illumination and scene. While illumination is very important to be known, the sensitivity characteristics is critical to the success of imaging applications, and is necessary to be optimally designed under practical constraints. The ultimate image quality is judged subjectively by human visual system. This dissertation addresses the evaluation and optimal design of spectral sensitivity functions for digital color imaging devices. Color imaging fundamentals and device characterization are discussed in the first place. For the evaluation of spectral sensitivity functions, this dissertation concentrates on the consideration of imaging noise characteristics. Both signal-independent and signal-dependent noises form an imaging noise model and noises will be propagated while signal is processed. A new colorimetric quality metric, unified measure of goodness (UMG), which addresses color accuracy and noise performance simultaneously, is introduced and compared with other available quality metrics. Through comparison, UMG is designated as a primary evaluation metric. On the optimal design of spectral sensitivity functions, three generic approaches, optimization through enumeration evaluation, optimization of parameterized functions, and optimization of additional channel, are analyzed in the case of the filter fabrication process is unknown. Otherwise a hierarchical design approach is introduced, which emphasizes the use of the primary metric but the initial optimization results are refined through the application of multiple secondary metrics. Finally the validity of UMG as a primary metric and the hierarchical approach are experimentally tested and verified

Screening for Neonatal Jaundice by Smartphone Sclera Imaging

Jaundice is observed in over 60% of neonates and must be carefully monitored. Ifsevere cases go unnoticed, death or permanent disability can result. Neonatal jaun-dice causes 100,000 deaths yearly, with low-income countries in Africa and SouthAsia particularly affected. There is an unmet need for an accessible and objectivescreening method. This thesis proposes a smartphone camera-based method forscreening based on quantification of yellow discolouration in the sclera.The primary aim is to develop and test an app to screen for neonatal jaundicethat requires only the smartphone itself. To this end, a novel ambient subtractionmethod is proposed and validated, with less dependence on external hardware orcolour cards than previous app-based methods. Another aim is to investigate thebenefits of screening via the sclera. An existing dataset of newborn sclera images(n=87) is used to show that sclera chromaticity can predict jaundice severity.The neoSCB app is developed to predict total serum bilirubin (TSB) fromambient-subtracted sclera chromaticity via a flash/ no-flash image pair. A studyis conducted in Accra, Ghana to evaluate the app. With 847 capture sessions, thisis the largest study on image-based jaundice detection to date. A model trained onsclera chromaticity is found to be more accurate than one based on skin. The modelis validated on an independent dataset collected at UCLH (n=38).The neoSCB app has a sensitivity of 100% and a specificity of 76% in iden-tifying neonates with TSB≥250μmol/L (n=179). This is equivalent to the TcB(JM-105) data collected concurrently, and as good as the best-performing app in theliterature (BiliCam). Following a one-time calibration, neoSCB works without spe-cialist equipment, which could help widen access to effective jaundice screening

Developing a home monitoring system for patients with chronic liver disease using a smartphone

Liver disease is a growing problem in the UK, and one of the major causes of working-age premature death. Patients with advanced liver disease are typically admitted to hospital on multiple occasions, where they are stabilised before discharge. At home, there is little or no monitoring of their condition available, making it difficult to time additional treatment. Here, a system for non-invasive assessment of serum bilirubin level is proposed, based on imaging the white of the eye (sclera) using a smartphone. Elevated bilirubin level manifests as jaundice, and is a key indicator of overall liver function. Smartphone imaging makes the system low cost, portable and non-contact. An ambient subtraction technique based on subtracting data from flash/ no-flash image pairs is leveraged to account for variations in ambient light. The subtracted signal to noise ratio (SSNR) metric has been developed to ensure good image quality. Values falling below the experimentally-determined threshold of 3.4 trigger a warning to re-capture. To produce device-independent results, mapping approaches based on image metadata and colour chart images were compared. It was found that introducing a one-time calibration step of imaging a colour chart for each device leads to the best compatibility of results from different phones. In a clinical study at the Royal Free Hospital, London, over 100 sets of patient scleral images were captured with two different smartphones and paired clinical information was recorded. A filtering algorithm was developed to tackle the high density of blood vessels and specular reflection observed in the images, yielding a 94% success rate. Strong cross-sectional and longitudinal correlations of scleral yellowness and serum bilirubin level were found of 0.89 and 0.72 respectively (both p<0.001). When the proposed processing was applied, results from the two phones were demonstrated to be compatible. These results demonstrate the strong potential for the system as a monitoring tool

The role of chromatic texture and 3D shape in colour discrimination, memory colour, and colour constancy of natural objects

The primary goal of this work was to investigate colour perception in a natural environment and to contribute to the understanding of how cues to familiar object identity influence colour appearance. A large number of studies on colour appearance employ 2D uniformly coloured patches, discarding perceptual cues such as binocular disparity, 3D luminance shading, mutual reflection, and glossy highlights are integral part of a natural scene. Moreover, natural objects possess specific cues that help our recognition (shape, surface texture or colour distribution). The aim of the first main experiment presented in this thesis was to understand the effect of shape on (1) memory colour under constant and varying illumination and on (2) colour constancy for uniformly coloured stimuli. The results demonstrated the existence of a range of memory colours associated with a familiar object, the size of which was strongly object-shape-dependent. For all objects, memory retrieval was significantly faster for object-diagnostic shape relative to generic shapes. Based on two successive controls, the author suggests that shape cues to the object identity affect the range of memory colour proportionally to the original object chromatic distribution. The second experiment examined the subject’s accuracy and precision in adjusting a stimulus colour to its typical appearance. Independently on the illuminant, results showed that memory colour accuracy and precision were enhanced by the presence of chromatic textures, diagnostic shapes, or 3D configurations with a strong interaction between diagnosticity and dimensionality of the shape. Hence, more cues to the object identity and more natural stimuli facilitate the observers in accessing their colour information from memory. A direct relationship was demonstrated between chromatic surface representation, object’s physical properties, and identificability and dimensionality of shape on memory colour accuracy, suggesting high-level mechanisms. Chromatic textures facilitated colour constancy. The third and fourth experiments tested the subject’s ability to discriminate between two chromatic stimuli in a simultaneous and successive 2AFC task, respectively. Simultaneous discrimination threshold performances for polychromatic surfaces were only due to low-level mechanism of the stimulus, whereas in the successive discrimination, i.e. when memory is involved, high-level mechanisms were established. The effect of shape was strongly task- dependent and was modulate by the object memory colour. These findings together with the strong interaction between chromatic cues and shape cues to the object identity lead to the conclusion that high level mechanisms linked to object recognition facilitated both tasks. Hence, the current thesis presents new findings on memory colour and colour constancy presented in a natural context and demonstrates the effect of high-level mechanisms in chromatic discrimination as a function of cues to the object identity such as shape and texture. This work contributes to a deeper understanding of colour perception and object recognition in the natural world.EThOS - Electronic Theses Online ServiceGBUnited Kingdo