Appearance-based image splitting for HDR display systems

High dynamic range displays that incorporate two optically-coupled image planes have recently been developed. This dual image plane design requires that a given HDR input image be split into two complementary standard dynamic range components that drive the coupled systems, therefore there existing image splitting issue. In this research, two types of HDR display systems (hardcopy and softcopy HDR display) are constructed to facilitate the study of HDR image splitting algorithm for building HDR displays. A new HDR image splitting algorithm which incorporates iCAM06 image appearance model is proposed, seeking to create displayed HDR images that can provide better image quality. The new algorithm has potential to improve image details perception, colorfulness and better gamut utilization. Finally, the performance of the new iCAM06-based HDR image splitting algorithm is evaluated and compared with widely spread luminance square root algorithm through psychophysical studies

TESTING COLOR APPEARANCE MODELS IN COMPLEX SCENE

The sensation of sight is our primary mechanism to perceive the world around us. However it is not yet perfectly clear how the human visual system works. The images of the world are formed on the retina, captured by sensors and converted in signals sent to the brain. Here the signals are processed and somehow interpreted, thus we are able to see. A lot of information, hypothesis, hints come from a field of the optical (or visual) illusions. These illusions have led many scientists and researchers to ask themselves why we are not able to interpret in a correct way some particular scenes. The word \u201cinterpret\u201d underlines the fact that the brain, and not only the eye, is involved in the process of vision. If our sight worked as a measurement tool, similar to a spectrophotometer, we would not perceive, for example, the simultaneous contrast phenomenon, in which a grey patch placed on a black background appears lighter than an identical coloured patch on a white background. So, why do we perceive the patches as different, while the light that reaches the eyes is the same? In the same way we would not be able to distinguish a white paper seen in a room lit with a red light from a red paper seen under a white light, however humans can do this. These phenomena are called colour appearance phenomena. Simulating the appearance is the objective of a range of computational models called colour appearance models. In this dissertation themes about colour appearance models are addressed. Specific experiments, performed by human observers, aim to evaluate and measure the appearance. Different algorithms are tested in order to compare the results of the computational model with the human sensations about colours. From these data, a new printing pipeline is developed, able to simulate the appearance of advertising billboard in different context

Perceptual image quality assessment for various viewing conditions and display systems

From complete darkness to direct sunlight, real-world dis- plays operate in various viewing conditions often resulting in a non-optimal viewing experience. Most existing Image Quality Assessment (IQA) methods, however, assume ideal environments and displays, and thus cannot be used when viewing conditions differ from the standard. In this paper, we investigate the influence of ambient illumination level and display luminance on human perception of image quality. We conduct a psychophysical study to collect a novel dataset of over 10000 image quality preference judgments performed in illumination conditions ranging from 0 lux to 20000 lux. We also propose a perceptual IQA framework that allows most existing image quality metrics (IQM) to accurately predict image quality for a wide range of illumination conditions and display parameters 1 . Our analysis demonstrates strong cor- relation between human IQA and the predictions of our proposed framework combined with multiple prominent IQMs and across a wide range of luminance values

High dynamic range display systems

High contrast ratio (CR) enables a display system to faithfully reproduce the real objects. However, achieving high contrast, especially high ambient contrast (ACR), is a challenging task. In this dissertation, two display systems with high CR are discussed: high ACR augmented reality (AR) display and high dynamic range (HDR) display. For an AR display, we improved its ACR by incorporating a tunable transmittance liquid crystal (LC) film. The film has high tunable transmittance range, fast response time, and is fail-safe. To reduce the weight and size of a display system, we proposed a functional reflective polarizer, which can also help people with color vision deficiency. As for the HDR display, we improved all three aspects of the hardware requirements: contrast ratio, color gamut and bit-depth. By stacking two liquid crystal display (LCD) panels together, we have achieved CR over one million to one, 14-bit depth with 5V operation voltage, and pixel-by-pixel local dimming. To widen color gamut, both photoluminescent and electroluminescent quantum dots (QDs) have been investigated. Our analysis shows that with QD approach, it is possible to achieve over 90% of the Rec. 2020 color gamut for a HDR display. Another goal of an HDR display is to achieve the 12-bit perceptual quantizer (PQ) curve covering from 0 to 10,000 nits. Our experimental results indicate that this is difficult with a single LCD panel because of the sluggish response time. To overcome this challenge, we proposed a method to drive the light emitting diode (LED) backlight and the LCD panel simultaneously. Besides relatively fast response time, this approach can also mitigate the imaging noise. Finally yet importantly, we improved the display pipeline by using a HDR gamut mapping approach to display HDR contents adaptively based on display specifications. A psychophysical experiment was conducted to determine the display requirements

Color Appearance Study under Two Lightings Having Different Illuminance Levels

Department of Human Factors EngineeringColor appearances of the objects are changing, depending on light sources. In everyday life, it is common to see a scene having two or more light sources together, or to look at an object that is shadowed by other objects. However, these situations might not be interpreted by current color appearance models, which are based on a single light source. With increasing interest in color reproduction of high dynamic scenes, color appearance research that can explain these multi illumination situations is necessary. In this research, it was intended to explain color appearance phenomena in the context where observers alternately saw two light sources having largely different illuminance levels (7005 lux and 376 lux, respectively) being present at the same time. This study also attempted to identify the observer's state of adaptation in the presence of multiple lightings, by exploring how color appearance in terms of hue, brightness and colorfulness changes in complex multiple lighting conditions, as opposed to single lighting conditions. Psychophysical experiment based on magnitude estimation technique was conducted to estimate color appearance and was composed of four sessions according to 1) illuminance of lighting either high or low, and 2) observer's adaptation to the lighting conditions for either single lighting or multiple lightings. Seven observers who were skillfully trained for color appearance estimation participated in the experiment and evaluated the color appearance of 50 color patches in terms of hue, colorfulness and brightness throughout the four sessions. As for the analyses of the results, human color perception data regarding hue, brightness and colorfulness of all observers were averaged and compared across sessions, based on the illuminance of lighting and the observer???s adaptation to the lighting conditions. Also, the color appearance model CIECAM02 performance was evaluated in terms of hue, brightness and colorfulness by comparing model prediction data with color perception data. As a result, through the color appearance study under two lightings with different illuminance levels, it turned out that hue appearance was not affected by the illuminance level of lighting and the observer???s adaptation to the lighting conditions. Perceived brightness and colorfulness were increased under higher illuminance level, but not affected by the observer's adaptation to the lighting conditions, explaining that observers locally adapted to the lighting where the color was directly shown. It was also found that the CIECAM02 H adeptly predicted hue appearance regardless of the illuminance level of lighting and the observer's adaptation to the lighting conditions. However, the CIECAM02 Q and the CIECAM02 M were overestimated under high illuminance lighting. The modification of the luminance-level adaptation factor, FL, by lowering the value from 1.20 to 0.67, helped the model not to overestimate Q and M. These results are based on color appearance perception when there are only two lightings having 19 times the illuminance difference. Consequently, it cannot be firmly concluded that these phenomena are common color appearance in multiple lighting environments. Therefore, it is necessary to conduct additional color appearance estimation research in multiple lighting conditions having more diverse illuminance level differences or having different configurations of lightings.clos

Selektive Farbkanalstimulation des menschlichen visuellen Systems

The objective of this thesis was to examine the interaction between the model-based excitation of the human color perception system and technical aspects of the necessary stimulators. Therefore, a novel integrative model for selective color channel stimulation was designed. Many methodological approaches were developed in this thesis, such as the classification and evaluation of color models, the inclusion of technical stimulator characteristics, the identification of an optimal stimulator, the development of compensation mechanisms for insufficient stimulators, the adaptation-based verification of the cone isolation technique, and the implementation of fundus-controlled selective color channel stimulation.Using an integrative model, three different studies were conducted. Two studies considered the validation of the new model. The third study focused on a newly established scientific field. First, the visual evoked potentials after selective S- and LM-cone stimulation were analyzed for validation. Significant differences between the konio- and parvocellula systems were identified as evidence of successful cone isolation. Based on these findings, a novel technique for simulating color vision deficiency in healthy volunteers was engineered using a bleaching approach. The absence of typical response signals after bleaching and the observed regeneration dynamic of the cones confirmed the integrative model. The major drawbacks of common electrophysiological examination methods are the inability to reveal the exact stimulus position on the retina and the absence of information about the individual fundus morphologies related to the area. By combining optical imaging, functional color channel stimulation, and electrophysiological investigation, a fundus-controlled stimulation technique was created. The novel technique has been published under the name “optoelectrophysiology” and overcomes the existing drawbacks. Within this thesis and in particular by conducting these studies, a new perspective on the unavoidable interactions between stimulators and color perception models has been demonstrated. This perspective can be subsumed into the integrative model developed. By using the model in future more efficient stimulations, more accurate fitting paradigms and novel multimodal stimulators will be possible.Die vorliegende Arbeit thematisiert die kombinierte Betrachtung der modellbasierten Reizung der farbprozessierenden Kanäle unter Einbezug wesentlicher technischer Aspekte des Stimulators. Hieraus wurde ein integratives Modell zur selektiven Farbkanalstimulation des menschlichen visuellen Systems erarbeitet und zur Anwendung gebracht. Im Rahmen der Arbeit konnte eine Vielzahl methodischer Herangehensweisen entwickelt werden. Diese bezogen sich u.a. auf die Klassifizierung und Bewertung von Farbmodellen, die Beachtung technischer Gesichtspunkte bzgl. einer Stimuluswirkung, die Frage nach dem optimalen Stimulator, die Möglichkeiten von Kompensationen unzureichender Stimulatoreigenschaften, auf Alternativen zur Nachweiserbringung der Wirkung von Farbmodellen sowie auf die Umsetzung einer funduskontrollierten selektiven Zapfenstimulation.Die Anwendungen beinhalteten drei abgeschlossene Studien. Die ersten beiden adressierten die Fragestellung der Validierung des neuentwickelten integrativen Modells. Durch die dritte Studie konnte die erstmalige Bearbeitung eines neu gegründeten Wissenschaftsfeldes realisiert werden. Zur Validierung wurden zunächst visuell evozierte Potenziale nach selektiver S- und LM-cone-Stimulation ausgewertet. Die gefundenen signifikanten Unterschiede zwischen koniozellulärem und parvozellulärem Kanal stellten Evidenzen einer erfolgreichen Modellanwendung dar. Darauf aufbauend erfolgte die Entwicklung einer adaptationsbasierten Methodik zur zeitlich terminierten Simulation von Farbfehlsichtigkeiten bei gesunden Probanden. Durch das Ausbleiben bestimmter Antwortsignale und die gefundene Regenerationsdynamik der Zapfen konnte der Nachweis einer selektiven Modellwirkung erbracht werden. Das bislang nicht besetzte Wissenschaftsgebiet der Optoelektrophysiologie resultierte aus der zu konstatierenden Unkenntnis über die individuelle Fundusmorphologie und eventuell vorhandene pathologische Auffälligkeiten, die zu Beginn einer jeden Untersuchung existiert. Dies lässt sich unter der Begrifflichkeit der fehlenden Funduskontrolle zusammenfassen. Durch die Kombination optischer Bildgebung, funktioneller Farbkanalstimulation und elektrophysiologischer Datengewinnung gelang es, eine funduskontrollierte Stimulationsmethodik zu schaffen.Innerhalb der Arbeit und insbesondere durch die Anwendungsfälle wurde eine neue Sichtweise auf unvermeidbare Wechselwirkungen zwischen Stimulator und eingesetztem Farbverarbeitungsmodell dargelegt. Diese lässt sich unter der Begrifflichkeit des integrativen Modells subsummieren und verfügt über das Potenzial, effizientere Stimulationen, passgenauere Paradigmen und neue multimodale Stimulatoren zu ermöglichen

Evaluation of changes in image appearance with changes in displayed image size

This research focused on the quantification of changes in image appearance when images are displayed at different image sizes on LCD devices. The final results provided in calibrated Just Noticeable Differences (JNDs) on relevant perceptual scales, allowing the prediction of sharpness and contrast appearance with changes in the displayed image size. A series of psychophysical experiments were conducted to enable appearance predictions. Firstly, a rank order experiment was carried out to identify the image attributes that were most affected by changes in displayed image size. Two digital cameras, exhibiting very different reproduction qualities, were employed to capture the same scenes, for the investigation of the effect of the original image quality on image appearance changes. A wide range of scenes with different scene properties was used as a test-set for the investigation of image appearance changes with scene type. The outcomes indicated that sharpness and contrast were the most important attributes for the majority of scene types and original image qualities. Appearance matching experiments were further conducted to quantify changes in perceived sharpness and contrast with respect to changes in the displayed image size. For the creation of sharpness matching stimuli, a set of frequency domain filters were designed to provide equal intervals in image quality, by taking into account the system’s Spatial Frequency Response (SFR) and the observation distance. For the creation of contrast matching stimuli, a series of spatial domain S-shaped filters were designed to provide equal intervals in image contrast, by gamma adjustments. Five displayed image sizes were investigated. Observers were always asked to match the appearance of the smaller version of each stimulus to its larger reference. Lastly, rating experiments were conducted to validate the derived JNDs in perceptual quality for both sharpness and contrast stimuli. Data obtained by these experiments finally converted into JND scales for each individual image attribute. Linear functions were fitted to the final data, which allowed the prediction of image appearance of images viewed at larger sizes than these investigated in this research

Non-parametric Methods for Automatic Exposure Control, Radiometric Calibration and Dynamic Range Compression

Imaging systems are essential to a wide range of modern day applications. With the continuous advancement in imaging systems, there is an on-going need to adapt and improve the imaging pipeline running inside the imaging systems. In this thesis, methods are presented to improve the imaging pipeline of digital cameras. Here we present three methods to improve important phases of the imaging process, which are (i) ``Automatic exposure adjustment'' (ii) ``Radiometric calibration'' (iii) ''High dynamic range compression''. These contributions touch the initial, intermediate and final stages of imaging pipeline of digital cameras. For exposure control, we propose two methods. The first makes use of CCD-based equations to formulate the exposure control problem. To estimate the exposure time, an initial image was acquired for each wavelength channel to which contrast adjustment techniques were applied. This helps to recover a reference cumulative distribution function of image brightness at each channel. The second method proposed for automatic exposure control is an iterative method applicable for a broad range of imaging systems. It uses spectral sensitivity functions such as the photopic response functions for the generation of a spectral power image of the captured scene. A target image is then generated using the spectral power image by applying histogram equalization. The exposure time is hence calculated iteratively by minimizing the squared difference between target and the current spectral power image. Here we further analyze the method by performing its stability and controllability analysis using a state space representation used in control theory. The applicability of the proposed method for exposure time calculation was shown on real world scenes using cameras with varying architectures. Radiometric calibration is the estimate of the non-linear mapping of the input radiance map to the output brightness values. The radiometric mapping is represented by the camera response function with which the radiance map of the scene is estimated. Our radiometric calibration method employs an L1 cost function by taking advantage of Weisfeld optimization scheme. The proposed calibration works with multiple input images of the scene with varying exposure. It can also perform calibration using a single input with few constraints. The proposed method outperforms, quantitatively and qualitatively, various alternative methods found in the literature of radiometric calibration. Finally, to realistically represent the estimated radiance maps on low dynamic range display (LDR) devices, we propose a method for dynamic range compression. Radiance maps generally have higher dynamic range (HDR) as compared to the widely used display devices. Thus, for display purposes, dynamic range compression is required on HDR images. Our proposed method generates few LDR images from the HDR radiance map by clipping its values at different exposures. Using contrast information of each LDR image generated, the method uses an energy minimization approach to estimate the probability map of each LDR image. These probability maps are then used as label set to form final compressed dynamic range image for the display device. The results of our method were compared qualitatively and quantitatively with those produced by widely cited and professionally used methods