Lightness, Brightness, and Transparency in Optical See-Through Augmented Reality

Augmented reality (AR), as a key component of the future metaverse, has leaped from the research labs to the consumer and enterprise markets. AR optical see-through (OST) devices utilize transparent optical combiners to provide visibility of the real environment as well as superimpose virtual content on top of it. OST displays distinct from existing media because of their optical additivity, meaning the light reaching the eyes is composed of both virtual content and real background. The composition results in the intended virtual colors being distorted and perceived transparent. When the luminance of the virtual content decreases, the perceived lightness and brightness decrease, and the perceived transparency increases. Lightness, brightness, and transparency are modulated by one physical dimension (luminance), and all interact with the background and each other. In this research, we aim to identify and quantify the three perceptual dimensions, as well as build mathematical models to predict them. In the first part of the study, we focused on the perceived brightness and lightness with two experiments: a brightness partition scaling experiment to build brightness scales, and a diffuse white adjustment experiment to determine the absolute luminance level required for diffuse white appearances on 2D and 3D AR stimuli. The second part of the research targeted at the perceived transparency in the AR environment with three experiments. The transparency was modulated by the background Michelson contrast reduction in either average luminance or peak-to-peak luminance difference to investigate, and later illustrated, the fundamental mechanism evoking transparency perception. The first experiment measured the transparency detection thresholds and confirmed that contrast sensitivity functions with contrast adaptation could model the thresholds. Subsequently, the transparency perception was investigated through direct anchored scaling experiment by building perceived transparency scales from the virtual content contrast ratio to the background. A contrast-ratio-based model was proposed predicting the perceived transparency scales. Finally, the transparency equivalency experiment between the two types of contrast modulation confirmed the mechanism difference and validated the proposed model

High Dynamic Range (HDR) Display Perception

Displays have undergone a huge development in the last several decades. From cathode-ray tube (CRT), liquid crystal display (LCD), to organic light-emitting diode (OLED), even Q-OLED, the new configurations of the display bring more and more functions into industry and daily life. In the recent several years, high dynamic range (HDR) displays become popular. HDR displays usually refer to that the black level of the display is darker and the peak being brighter compared with the standard dynamic range (SDR) display. Traditionally, the peak luminance level can be used as the white in characterization and calibration. However, for HDR displays, the peak luminance is higher than the traditional diffuse white level. Exploration of the perceptual diffuse white in HDR image when presented in displays is proposed, which can be beneficial to the characterizing and the optimizing the usage of the HDR display. Moreover, in addition to the ``diffuse white , 3D color gamut volume can be calculated in some specific color appearance models. Calculation and modeling of the 3D color gamut volume can be very useful for display design and better characterizing display color reproduction capability. Furthermore, the perceptional color gamut volume can be measured through psychophysical experiments. Comparison between the perceptional color gamut volume and the theoretical 3D gamut volume calculations will reveal some insights for optimizing the usage of HDR displays. Another advantage of the HDR display is its darker black compared with the SDR display. Compared with the real black object, what level of black is `perfect\u27 enough in displays? Experiments were proposed and conducted to evaluate that if the HDR display is capable of showing ``perfect black for different types of background images/patterns. A glare-based model was proposed to predict the visual ``perfect black. Additionally, the dynamic range of human vision system is very large. However, the simultaneous dynamic range of human vision system is much smaller and is important for the fine tuning usage of HDR displays. The simultaneous dynamic range was measured directly for different stimulus sizes. Also, it was found that the simultaneous dynamic range was peak luminance level dependent. A mathematical model was proposed based on the experimental data to predict the simultaneous dynamic range. Also the spatial frequency effect of the target pattern on the simultaneous dynamic range was measured and modeled. The four different assessments about HDR displays perception would provide experimental data and models for a better understanding of HDR perception and tuning of the HDR display

Studies in ambient intelligent lighting

The revolution in lighting we are arguably experiencing is led by technical developments in the area of solid state lighting technology. The improved lifetime, efficiency and environmentally friendly raw materials make LEDs the main contender for the light source of the future. The core of the change is, however, not in the basic technology, but in the way users interact with it and the way the quality of the produced effect on the environment is judged. With the new found freedom the users can switch their focus from the confines of the technology to the expression of their needs, regardless of the details of the lighting system. Identifying the user needs, creating an effective language to communicate them to the system, and translating them to control signals that fulfill them, as well as defining the means to measure the quality of the produced result are the topic of study of a new multidisciplinary area of study, Ambient Intelligent Lighting. This thesis describes a series of studies in the field of Ambient Intelligent Lighting, divided in two parts. The first part of the thesis demonstrates how, by adopting a user centric design philosophy, the traditional control paradigms can be superseded by novel, so-called effect driven controls. Chapter 3 describes an algorithm that, using statistical methods and image processing, generates a set of colors based on a term or set of terms. The algorithm uses Internet image search engines (Google Images, Flickr) to acquire a set of images that represent a term and subsequently extracts representative colors from the set. Additionally, an estimate of the quality of the extracted set of colors is computed. Based on the algorithm, a system that automatically enriches music with lyrics based images and lighting was built and is described. Chapter 4 proposes a novel effect driven control algorithm, enabling users easy, natural and system agnostic means to create a spatial light distribution. By using an emerging technology, visible light communication, and an intuitive effect definition, a real time interactive light design system was developed. Usability studies on a virtual prototype of the system demonstrated the perceived ease of use and increased efficiency of an effect driven approach. In chapter 5, using stochastic models, natural temporal light transitions are modeled and reproduced. Based on an example video of a natural light effect, a Markov model of the transitions between colors of a single light source representing the effect is learned. The model is a compact, easy to reproduce, and as the user studies show, recognizable representation of the original light effect. The second part of the thesis studies the perceived quality of one of the unique capabilities of LEDs, chromatic temporal transitions. Using psychophysical methods, existing spatial models of human color vision were found to be unsuitable for predicting the visibility of temporal artifacts caused by the digital controls. The chapters in this part demonstrate new perceptual effects and make the first steps towards building a temporal model of human color vision. In chapter 6 the perception of smoothness of digital light transitions is studied. The studies presented demonstrate the dependence of the visibility of digital steps in a temporal transition on the frequency of change, chromaticity, intensity and direction of change of the transition. Furthermore, a clear link between the visibility of digital steps and flicker visibility is demonstrated. Finally, a new, exponential law for the dependence of the threshold speed of smooth transitions on the changing frequency is hypothesized and proven in subsequent experiments. Chapter 7 studies the discrimination and preference of different color transitions between two colors. Due to memory effects, the discrimination threshold for complete transitions was shown to be larger than the discrimination threshold for two single colors. Two linear transitions in different color spaces were shown to be significantly preferred over a set of other, curved, transitions. Chapter 8 studies chromatic and achromatic flicker visibility in the periphery. A complex change of both the absolute visibility thresholds for different frequencies, as well as the critical flicker frequency is observed. Finally, an increase in the absolute visibility thresholds caused by an addition of a mental task in central vision is demonstrated

High dynamic range display systems

High contrast ratio (CR) enables a display system to faithfully reproduce the real objects. However, achieving high contrast, especially high ambient contrast (ACR), is a challenging task. In this dissertation, two display systems with high CR are discussed: high ACR augmented reality (AR) display and high dynamic range (HDR) display. For an AR display, we improved its ACR by incorporating a tunable transmittance liquid crystal (LC) film. The film has high tunable transmittance range, fast response time, and is fail-safe. To reduce the weight and size of a display system, we proposed a functional reflective polarizer, which can also help people with color vision deficiency. As for the HDR display, we improved all three aspects of the hardware requirements: contrast ratio, color gamut and bit-depth. By stacking two liquid crystal display (LCD) panels together, we have achieved CR over one million to one, 14-bit depth with 5V operation voltage, and pixel-by-pixel local dimming. To widen color gamut, both photoluminescent and electroluminescent quantum dots (QDs) have been investigated. Our analysis shows that with QD approach, it is possible to achieve over 90% of the Rec. 2020 color gamut for a HDR display. Another goal of an HDR display is to achieve the 12-bit perceptual quantizer (PQ) curve covering from 0 to 10,000 nits. Our experimental results indicate that this is difficult with a single LCD panel because of the sluggish response time. To overcome this challenge, we proposed a method to drive the light emitting diode (LED) backlight and the LCD panel simultaneously. Besides relatively fast response time, this approach can also mitigate the imaging noise. Finally yet importantly, we improved the display pipeline by using a HDR gamut mapping approach to display HDR contents adaptively based on display specifications. A psychophysical experiment was conducted to determine the display requirements

Fluorescent particle tracers for surface hydrology

Surface water processes control downstream runoff phenomena, waste and pollutant diffusion, erosion mechanics, and sediment transport. However, current observational methodologies do not allow for the identification and kinematic characterization of the physical processes contributing to catchment dynamics. Traditional methodologies are not capable to cope with extreme in-situ conditions, including practical logistic challenges as well as inherent flow complexity. In addition, available observational techniques are non-exhaustive for describing multiscale hydrological processes. This research addresses the need for novel observations of the hydrological community by developing pioneer flow characterization approaches that rely on the mutual integration of traditional tracing techniques and state-of-the-art image-based sensing procedures. These novel methodologies enable the in-situ direct observation of surface water processes through remote and unsupervised procedures, thus paving the way to the development of distributed networks of sensing platforms for catchment-scale environmental sensing. More specifically, the proposed flow characterization methodology is a low-cost measurement system that can be applied to a variety of real-world settings spanning from few centimeters rills in natural catchments to riverine ecosystems. The technique is based on the use of in-house synthesized environmentally-friendly fluorescent particle tracers through digital cameras for direct flow measurement and travel time estimations. Automated image analysis-based procedures are developed for real-time flow characterization based on image manipulation, template-based correlation, particle image velocimetry, and dimensionality reduction methodologies. The feasibility of the approach is assessed through laboratory-designed experiments, where the accuracy of the methodology is investigated with respect to well-established flow visualization techniques. Further, the transition of the proposed flow characterization approach to natural settings is studied through paradigmatic observations of natural stream flows in small scale channel and riverine settings and overland flows in hillslope environments. The integration of the proposed flow sensing system in a stand-alone, remote, and mobile platform is explored through the design, development, and testing of a miniature aerial vehicle for environmental monitoring through video acquisition and processing