High-dynamic-range Foveated Near-eye Display System

Wearable near-eye display has found widespread applications in education, gaming, entertainment, engineering, military training, and healthcare, just to name a few. However, the visual experience provided by current near-eye displays still falls short to what we can perceive in the real world. Three major challenges remain to be overcome: 1) limited dynamic range in display brightness and contrast, 2) inadequate angular resolution, and 3) vergence-accommodation conflict (VAC) issue. This dissertation is devoted to addressing these three critical issues from both display panel development and optical system design viewpoints. A high-dynamic-range (HDR) display requires both high peak brightness and excellent dark state. In the second and third chapters, two mainstream display technologies, namely liquid crystal display (LCD) and organic light emitting diode (OLED), are investigated to extend their dynamic range. On one hand, LCD can easily boost its peak brightness to over 1000 nits, but it is challenging to lower the dark state to \u3c 0.01 nits. To achieve HDR, we propose to use a mini-LED local dimming backlight. Based on our simulations and subjective experiments, we establish practical guidelines to correlate the device contrast ratio, viewing distance, and required local dimming zone number. On the other hand, self-emissive OLED display exhibits a true dark state, but boosting its peak brightness would unavoidably cause compromised lifetime. We propose a systematic approach to enhance OLED\u27s optical efficiency while keeping indistinguishable angular color shift. These findings will shed new light to guide future HDR display designs. In Chapter four, in order to improve angular resolution, we demonstrate a multi-resolution foveated display system with two display panels and an optical combiner. The first display panel provides wide field of view for peripheral vision, while the second panel offers ultra-high resolution for the central fovea. By an optical minifying system, both 4x and 5x enhanced resolutions are demonstrated. In addition, a Pancharatnam-Berry phase deflector is applied to actively shift the high-resolution region, in order to enable eye-tracking function. The proposed design effectively reduces the pixelation and screen-door effect in near-eye displays. The VAC issue in stereoscopic displays is believed to be the main cause of visual discomfort and fatigue when wearing VR headsets. In Chapter five, we propose a novel polarization-multiplexing approach to achieve multiplane display. A polarization-sensitive Pancharatnam-Berry phase lens and a spatial polarization modulator are employed to simultaneously create two independent focal planes. This method enables generation of two image planes without the need of temporal multiplexing. Therefore, it can effectively reduce the frame rate by one-half. In Chapter six, we briefly summarize our major accomplishments

Demonstrating a multi-primary high dynamic range display system for vision experiments.

We describe the design, construction, calibration, and characterization of a multi-primary high dynamic range (MPHDR) display system for use in vision research. The MPHDR display is the first system to our knowledge to allowfor spatially controllable, high dynamic range stimulus generation using multiple primaries.We demonstrate the high luminance, high dynamic range, and wide color gamut output of the MPHDR display. During characterization, the MPHDR display achieved a maximum luminance of 3200 cd=m2, a maximum contrast range of 3; 240; 000 V 1, and an expanded color gamut tailored to dedicated vision research tasks that spans beyond traditional sRGB displays. We discuss how the MPHDR display could be optimized for psychophysical experiments with photoreceptor isolating stimuli achieved through the method of silent substitution. We present an example case of a range of metameric pairs of melanopsin isolating stimuli across different luminance levels, from an available melanopsin contrast of117%at 75 cd=m2 to a melanopsin contrast of23%at 2000 cd=m2

Real time delivery of HDR video

High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the football when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, socalled trueHDR, it is crucial that all the dynamic range that was captured or generated is delivered to the display device and tone mapping does not occur anywhere in the pipeline prior to the display. This paper describes a system of encoding and delivering HDR video which enables us to send video with a greater dynamic range than existing solutions from an HDR source (such as an HDR-enabled camera) to any display, including mobile devices, where it can be manipulated in real-time

Reproducing reality with a high-dynamic-range multi-focal stereo display

With well-established methods for producing photo-realistic results, the next big challenge of graphics and display technologies is to achieve perceptual realism --- producing imagery indistinguishable from real-world 3D scenes. To deliver all necessary visual cues for perceptual realism, we built a High-Dynamic-Range Multi-Focal Stereo Display that achieves high resolution, accurate color, a wide dynamic range, and most depth cues, including binocular presentation and a range of focal depth. The display and associated imaging system have been designed to capture and reproduce a small near-eye three-dimensional object and to allow for a direct comparison between virtual and real scenes. To assess our reproduction of realism and demonstrate the capability of the display and imaging system, we conducted an experiment in which the participants were asked to discriminate between a virtual object and its physical counterpart. Our results indicate that the participants can only detect the discrepancy with a probability of 0.44. With such a level of perceptual realism, our display apparatus can facilitate a range of visual experiments that require the highest fidelity of reproduction while allowing for the full control of the displayed stimuli.</jats:p

High Dynamic Range (HDR) Display Perception

Displays have undergone a huge development in the last several decades. From cathode-ray tube (CRT), liquid crystal display (LCD), to organic light-emitting diode (OLED), even Q-OLED, the new configurations of the display bring more and more functions into industry and daily life. In the recent several years, high dynamic range (HDR) displays become popular. HDR displays usually refer to that the black level of the display is darker and the peak being brighter compared with the standard dynamic range (SDR) display. Traditionally, the peak luminance level can be used as the white in characterization and calibration. However, for HDR displays, the peak luminance is higher than the traditional diffuse white level. Exploration of the perceptual diffuse white in HDR image when presented in displays is proposed, which can be beneficial to the characterizing and the optimizing the usage of the HDR display. Moreover, in addition to the ``diffuse white , 3D color gamut volume can be calculated in some specific color appearance models. Calculation and modeling of the 3D color gamut volume can be very useful for display design and better characterizing display color reproduction capability. Furthermore, the perceptional color gamut volume can be measured through psychophysical experiments. Comparison between the perceptional color gamut volume and the theoretical 3D gamut volume calculations will reveal some insights for optimizing the usage of HDR displays. Another advantage of the HDR display is its darker black compared with the SDR display. Compared with the real black object, what level of black is `perfect\u27 enough in displays? Experiments were proposed and conducted to evaluate that if the HDR display is capable of showing ``perfect black for different types of background images/patterns. A glare-based model was proposed to predict the visual ``perfect black. Additionally, the dynamic range of human vision system is very large. However, the simultaneous dynamic range of human vision system is much smaller and is important for the fine tuning usage of HDR displays. The simultaneous dynamic range was measured directly for different stimulus sizes. Also, it was found that the simultaneous dynamic range was peak luminance level dependent. A mathematical model was proposed based on the experimental data to predict the simultaneous dynamic range. Also the spatial frequency effect of the target pattern on the simultaneous dynamic range was measured and modeled. The four different assessments about HDR displays perception would provide experimental data and models for a better understanding of HDR perception and tuning of the HDR display

Live HDR video streaming on commodity hardware

High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the soccer ball when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, so-called true HDR, it is crucial that all the dynamic range that was captured is delivered to the display device and tone mapping is confined only to the display. Furthermore, to ensure widespread uptake of HDR imaging, it should be low cost and available on commodity hardware. This paper describes an end-to-end HDR pipeline for capturing, encoding and streaming high-definition HDR video in real-time using off-the-shelf components. All the lighting that is captured by HDR-enabled consumer cameras is delivered via the pipeline to any display, including HDR displays and even mobile devices with minimum latency. The system thus provides an integrated HDR video pipeline that includes everything from capture to post-production, archival and storage, compression, transmission, and display

A new technique to reproduced high-dynamic-range images for low-dynamic-range display

Tone mapping is a process for reproduction of High-Dynamic-Range images (HDR) for Low-Dynamic-Range (LDR) output devices. In this report, author presents a new local tone-mapping operator, derived from the Contrast Limited Adaptive histogram Equalization (CLAHE) technique for displaying high dynamic range image. The CLAHE is a method which was originally developed for medical imaging. This method has effectively expanded the full dynamic range of display and it is fully automatic. Due to different luminance intervals could result in overlapped reaction on the limited response in limited response range of visual system, scene region splitting and merging were used to segment the scaled luminance and perform the image segmentation to segment image into smaller part. After the region splitting and merging, there will be some noise or variation of intensity that may result in holes or over segmentation. As the result, the morphological operation, opening and closing were used to perform the mask to applied different clip limit of the CLAHE operation

High dynamic range display systems

High contrast ratio (CR) enables a display system to faithfully reproduce the real objects. However, achieving high contrast, especially high ambient contrast (ACR), is a challenging task. In this dissertation, two display systems with high CR are discussed: high ACR augmented reality (AR) display and high dynamic range (HDR) display. For an AR display, we improved its ACR by incorporating a tunable transmittance liquid crystal (LC) film. The film has high tunable transmittance range, fast response time, and is fail-safe. To reduce the weight and size of a display system, we proposed a functional reflective polarizer, which can also help people with color vision deficiency. As for the HDR display, we improved all three aspects of the hardware requirements: contrast ratio, color gamut and bit-depth. By stacking two liquid crystal display (LCD) panels together, we have achieved CR over one million to one, 14-bit depth with 5V operation voltage, and pixel-by-pixel local dimming. To widen color gamut, both photoluminescent and electroluminescent quantum dots (QDs) have been investigated. Our analysis shows that with QD approach, it is possible to achieve over 90% of the Rec. 2020 color gamut for a HDR display. Another goal of an HDR display is to achieve the 12-bit perceptual quantizer (PQ) curve covering from 0 to 10,000 nits. Our experimental results indicate that this is difficult with a single LCD panel because of the sluggish response time. To overcome this challenge, we proposed a method to drive the light emitting diode (LED) backlight and the LCD panel simultaneously. Besides relatively fast response time, this approach can also mitigate the imaging noise. Finally yet importantly, we improved the display pipeline by using a HDR gamut mapping approach to display HDR contents adaptively based on display specifications. A psychophysical experiment was conducted to determine the display requirements

High-fidelity colour reproduction for high-dynamic-range imaging

The aim of this thesis is to develop a colour reproduction system for high-dynamic-range (HDR) imaging. Classical colour reproduction systems fail to reproduce HDR images because current characterisation methods and colour appearance models fail to cover the dynamic range of luminance present in HDR images. HDR tone-mapping algorithms have been developed to reproduce HDR images on low-dynamic-range media such as LCD displays. However, most of these models have only considered luminance compression from a photographic point of view and have not explicitly taken into account colour appearance. Motivated by the idea to bridge the gap between crossmedia colour reproduction and HDR imaging, this thesis investigates the fundamentals and the infrastructure of cross-media colour reproduction. It restructures cross-media colour reproduction with respect to HDR imaging, and develops a novel cross-media colour reproduction system for HDR imaging. First, our HDR characterisation method enables us to measure HDR radiance values to a high accuracy that rivals spectroradiometers. Second, our colour appearance model enables us to predict human colour perception under high luminance levels. We first built a high-luminance display in order to establish a controllable high-luminance viewing environment. We conducted a psychophysical experiment on this display device to measure perceptual colour attributes. A novel numerical model for colour appearance was derived from our experimental data, which covers the full working range of the human visual system. Our appearance model predicts colour and luminance attributes under high luminance levels. In particular, our model predicts perceived lightness and colourfulness to a significantly higher accuracy than other appearance models. Finally, a complete colour reproduction pipeline is proposed using our novel HDR characterisation and colour appearance models. Results indicate that our reproduction system outperforms other reproduction methods with statistical significance. Our colour reproduction system provides high-fidelity colour reproduction for HDR imaging, and successfully bridges the gap between cross-media colour reproduction and HDR imaging

A local tone mapping operator for high dynamic range images

In this paper, we present a new tonemapping operator to display high dynamic range image onto conventional displayable devices and printers. In our work, a new tone map algorithm, derived from the Contrast Limited Adaptive histogram Equalization (CLAHE) technique is presented. Due to different luminance intervals could result in overlapped reaction on the limited response in limited response range of visual system, we use scenes region splitting and merging to segment the scaled luminance, L(x, y) and perform the CLAHE in each segment with different clip limit in order to extending our visual response range to cope with the full dynamic range of high contrast. Until now, there is no fix standard of objective evaluation available to measuring the quality of displayed High Dynamic Range (HDR) images because it is difficult to know how the light or dark the image should be displayed to faithful to the original HDR image. As the result, the main evaluation is based on human's subjective evaluation. In this paper, we consider this to evaluate the performances with different tone mapping method