Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

Reconfigurable hardware for color space conversion

Color space conversion (CSC) is an important application in image and video processing systems. CSC has been implemented in software and various kinds of hardware. Hardware implementations can achieve a higher performance compared to software-only solutions. Application specific integrated circuits (ASICs) are efficient and have good performance. However, they lack the programmability of devices such as field programmable gate arrays (FPGAs). This thesis studies the performance vs. flexibility tradeoffs in the migration of an existing CSC design from an ASIC to an FPGA. The existing ASIC is used within a commercial color-printing pipeline. Performance is critical in this application. However, the flexibility of FPGAs is desirable for faster time to market and also the ability to reuse one physical device across multiple functions. This thesis investigates whether the reprogrammability of FPGAs can be used to reallocate idle resources and studies the suitability of FPGAs for image processing applications. In the ASIC design, two major conversion units that are never used at the same time are identified. The FPGA-based implementation instantiates only one of these two units at a time, thus saving area. Reconfiguring the FPGA switches which of the two units is instantiated. The goal is to configure the device and process an entire page within one second. The FPGA implementation is approximately a factor of three slower than the ASIC design, but fast enough to process one page per second. In the current setup, the configuration time is very high. It exceeds the total time allotted for both configuration and processing. However, other methods of configuration seem promising to reduce the time. Evaluation of the performance of the implementation and the reconfiguration time is presented. Methods to improve the performance and reduce the time and area for reconfiguration are discussed

High-dynamic-range Foveated Near-eye Display System

Wearable near-eye display has found widespread applications in education, gaming, entertainment, engineering, military training, and healthcare, just to name a few. However, the visual experience provided by current near-eye displays still falls short to what we can perceive in the real world. Three major challenges remain to be overcome: 1) limited dynamic range in display brightness and contrast, 2) inadequate angular resolution, and 3) vergence-accommodation conflict (VAC) issue. This dissertation is devoted to addressing these three critical issues from both display panel development and optical system design viewpoints. A high-dynamic-range (HDR) display requires both high peak brightness and excellent dark state. In the second and third chapters, two mainstream display technologies, namely liquid crystal display (LCD) and organic light emitting diode (OLED), are investigated to extend their dynamic range. On one hand, LCD can easily boost its peak brightness to over 1000 nits, but it is challenging to lower the dark state to \u3c 0.01 nits. To achieve HDR, we propose to use a mini-LED local dimming backlight. Based on our simulations and subjective experiments, we establish practical guidelines to correlate the device contrast ratio, viewing distance, and required local dimming zone number. On the other hand, self-emissive OLED display exhibits a true dark state, but boosting its peak brightness would unavoidably cause compromised lifetime. We propose a systematic approach to enhance OLED\u27s optical efficiency while keeping indistinguishable angular color shift. These findings will shed new light to guide future HDR display designs. In Chapter four, in order to improve angular resolution, we demonstrate a multi-resolution foveated display system with two display panels and an optical combiner. The first display panel provides wide field of view for peripheral vision, while the second panel offers ultra-high resolution for the central fovea. By an optical minifying system, both 4x and 5x enhanced resolutions are demonstrated. In addition, a Pancharatnam-Berry phase deflector is applied to actively shift the high-resolution region, in order to enable eye-tracking function. The proposed design effectively reduces the pixelation and screen-door effect in near-eye displays. The VAC issue in stereoscopic displays is believed to be the main cause of visual discomfort and fatigue when wearing VR headsets. In Chapter five, we propose a novel polarization-multiplexing approach to achieve multiplane display. A polarization-sensitive Pancharatnam-Berry phase lens and a spatial polarization modulator are employed to simultaneously create two independent focal planes. This method enables generation of two image planes without the need of temporal multiplexing. Therefore, it can effectively reduce the frame rate by one-half. In Chapter six, we briefly summarize our major accomplishments

차세대 액정 디스플레이용 광시야각 및 칼라구현 기술에 관한 연구

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 이신두.In the past decades, liquid crystal display (LCDs) have been extensively studied for use in flat panel displays (FPDs) because of light weight panel, thin thickness and low power consumption. Due to their high contrast, uniform brightness, and rapid switching times compared to conventional cathode ray tubes, LCDs have achieved a significant position in the FPD industry. However, according to the development of other FPD technologies including organic light-emitting didoes, the enhancement of LCD performances such as wide-viewing angle and vivid coloration has been demanded. In this thesis, novel image generation technologies for high performance LCD are proposed. At first, for the reduction of the gamma distortion, wide-viewing technology was demonstrated by sophisticated multi-domain configuration in which liquid crystal (LC) molecules are aligned along different polar directions at middle gray scale. In order to realize the multi-domain LC cell, a new alignment layer which can control polar LC alignment properties including the pretilt angle and the anchoring energy was developed. Uncured hydrophobic oligomers in a cross-linked polymer mold were transferred onto a substrate to change the surface wettability of the substrate. The thermal-transfer printing, which is a contact-transfer printing method including an additional heat treatment process, accelerated the molecular diffusion of oligomers from the polymer mold onto the substrate, resulting in the increase of the amount of transferred oligomer. Consequently, the surface energy of the substrate was manipulated according to the heat treatment temperature during the thermal-transfer printing. Then, the LC alignment properties in both azimuthal and polar directions were measured through LC cell fabrication and electro-optical characterization. It was found that the polar pretilt angle and polar anchoring energy were changed in a wide-range, resulting from the adjustment of the substrate surface energy. Based on the results described above, the reduction of the gamma distortion through the multi-domain configuration in the vertical alignment (VA) mode was demonstrated. The oligomer layer was formed onto a conventional VA alignment layer to control the polar alignment properties. The ultra-violet ozone (UVO) treatment for the enhancement of the oligomer transfer onto hydrophobic substrates was added in the fabrication process. The surface energy and anchoring energy of the stacked alignment layers depending on the UVO treatment was examined. The anchoring disparity, which is the anchoring energy difference between sub-domains as a result of the oligomer layer patterning, induced the threshold voltage difference in the sub-domains and constructed the multi-domain at applied voltages. The experimental measurements for the assessment of the gamma distortion were performed in a conventional patterned VA mode cell and proposed multi-domain cell. Next, for the expansion of the color space, a new type of coloration technique based on the photo-luminescent emission of the quantum dots (QDs) in an organic polymer matrix was described. Recently, QD patterning technologies based on the polymer matrix have been extensively studied to accurately express the primary color in pixel units, but suffered from the non-uniform light emission owing to uneven distribution of the QDs in the polymer matrix. Reactive mesogen (RM), which is photo-curable, transparent, and soluble in organic solvents, was used for the fabrication of uniformly distributed QDs in the RM matrix. The emission characteristics of QD-RM composites depending on the concentration ratios of QD and RM were examined. In addition, color-separated patterns of QD-RM composites on a single substrate through conventional photo-lithography processes were demonstrated. Finally, a novel QD emissive LCD based on the color-separated pattern for high color purity was proposed. The proposed LCD consists of modulation part and emission part. LC cells acts as a role of electrically tunable lightwave retarder depending on the applied voltage. The transmitted light through the modulation part emits the photo-luminescent light from the color-separated QD pattern. The color purity and color gamut of the QD emissive LCD was measured in comparison with previous LCDs such as the color filter LCD and QD compensated LCD. In conclusion, the applicability of proposed image generation technologies to the FPD industry was explored. The new concepts and experimental results will lead to the development of the advanced LCD performance.Chapter 1 Introduction 1 1.1 Overview of Flat Panel Displays 1 1.2 Outline of Thesis 9 Chapter 2 Emerging Technologies of Liquid Crystal Display 13 2.1 Operating Principles of LCD 13 2.1.1 Properties of LCs 14 2.1.2 Main LCD modes 21 2.2 Wide-Viewing Angle Technologies 28 2.2.1 General description of viewing properties 28 2.2.2 Recent technologies for wide-viewing 35 2.3 Coloration Technologies 43 2.3.1 Coloration in flat panel displays 43 2.3.2 Recent technologies for wide color gamut 52 Chapter 3 Enhancement of Viewing Angle of LCD by Multi-Domain 65 3.1 Control of Molecular Alignment by Thermo-Transfer Printing 65 3.1.1 Introduction 66 3.1.2 Thermo-transfer printing for surface modification 67 3.1.3 Fabrication process 69 3.1.4 Experimental results and discussions 72 3.1.5 Summary 83 3.2 Wide-Viewing by Anchoring Disparity 85 3.2.1 Introduction 85 3.2.2 Anchoring disparity by thermo-transfer printing 86 3.2.3 Fabrication of 8-domains 95 3.2.4 Experimental results and discussions 98 3.2.5 Summary 100 Chapter 4 Novel Quantum Dot-Based LCD for High Color Purity 101 4.1 Color-Separated Pattern of QDs 101 4.1.1 Introduction 101 4.1.2 Fabrication of color-separated QD patterns 103 4.1.3 Experimental results and discussions 105 4.2 QD Emissive LCD 109 4.2.1 Device concept 109 4.2.2 Fabrication of QD-based LCD 110 4.2.3 Experimental result and discussions 111 4.2.4 Summary 118 Chapter 5 Concluding Remarks 119 Bibliography 123 Publication 135 국문 초록 141Docto

Expanding Dimensionality in Cinema Color: Impacting Observer Metamerism through Multiprimary Display

Television and cinema display are both trending towards greater ranges and saturation of reproduced colors made possible by near-monochromatic RGB illumination technologies. Through current broadcast and digital cinema standards work, system designs employing laser light sources, narrow-band LED, quantum dots and others are being actively endorsed in promotion of Wide Color Gamut (WCG). Despite artistic benefits brought to creative content producers, spectrally selective excitations of naturally different human color response functions exacerbate variability of observer experience. An exaggerated variation in color-sensing is explicitly counter to the exhaustive controls and calibrations employed in modern motion picture pipelines. Further, singular standard observer summaries of human color vision such as found in the CIE’s 1931 and 1964 color matching functions and used extensively in motion picture color management are deficient in recognizing expected human vision variability. Many researchers have confirmed the magnitude of observer metamerism in color matching in both uniform colors and imagery but few have shown explicit color management with an aim of minimized difference in observer perception variability. This research shows that not only can observer metamerism influences be quantitatively predicted and confirmed psychophysically but that intentionally engineered multiprimary displays employing more than three primaries can offer increased color gamut with drastically improved consistency of experience. To this end, a seven-channel prototype display has been constructed based on observer metamerism models and color difference indices derived from the latest color vision demographic research. This display has been further proven in forced-choice paired comparison tests to deliver superior color matching to reference stimuli versus both contemporary standard RGB cinema projection and recently ratified standard laser projection across a large population of color-normal observers

Understanding and modeling of aesthetic response to shape and color in car body design

This study explored the phenomenon that a consumer's preference on color of car body may vary depending on shape of the car body. First, the study attempted to establish a theoretical framework that can account for this phenomenon. This framework is based on the (modern-) Darwinism approach to the so-called evolutionary psychology and aesthetics. It assumes that human's aesthetic sense works like an agent that seeks for environmental patterns that potentially afford to benefit the underlying needs of the agent, and this seeking process is evolutionary fitting. Second, by adopting the framework, a pattern called “fundamental aesthetic dimensions” was developed for identifying and modeling consumer’s aesthetic response to car body shape and color. Next, this study developed an effective tool that is capable in capturing and accommodating consumer’s color preference on a given car body shape. This tool was implemented by incorporating classic color theories and advanced digital technologies; it was named “Color-Shape Synthesizer”. Finally, an experiment was conducted to verify some of the theoretical developments. This study concluded (1) the fundamental aesthetics dimensions can be used for describing aesthetics in terms of shape and color; (2) the Color-Shape Synthesizer tool can be well applied in practicing car body designs; and (3) mapping between semantic representations of aesthetic response to the fundamental aesthetics dimensions can likely be a multiple-network structure

Automatic assessment and enhancement of streaming video quality under bandwidth and dynamic range limitations

The explosion in the amount of video content being streamed over the internet in recent years has accelerated the demand for effective and efficient methods for assessing and improving the perceptual quality of images and videos while adhering to internet bandwidth and display dynamic range limitations. Objective models of perceptual quality have found extensive use in optimizing video compression and enhancement parameters to achieve desirable streaming fidelity. In this dissertation, we develop a variety of quality modeling and quality enhancement methods targeting the streaming of standard and high dynamic range (SDR/HDR) videos over the internet, subjected to compression and tone mapping. The Visual Multimethod Assessment Fusion (VMAF) algorithm has recently emerged as a state-of-the-art approach to video quality prediction, that now pervades the streaming and social media industry. However, since VMAF requires the evaluation of a heterogeneous set of quality models, it is computationally expensive. Given other advances in hardware-accelerated encoding, quality assessment is emerging as a significant bottleneck in video compression pipelines. Towards alleviating this burden, we first propose a novel Fusion of Unified Quality Evaluators (FUNQUE) framework, by enabling computation sharing and by using a transform that is sensitive to visual perception to boost accuracy. Further, we expand the FUNQUE framework to define a collection of improved low-complexity fused-feature models that advance the state-of-the-art of video quality performance with respect to both accuracy, by 4.2\% to 5.3\%, and computational efficiency, by factors of 3.8 to 11 times! High Dynamic Range (HDR) videos are able to represent wider ranges of contrasts and colors than Standard Dynamic Range (SDR) videos, giving more vivid experiences. Due to this, HDR videos are expected to grow into the dominant video modality of the future. However, HDR videos are incompatible with existing SDR displays, which form the majority of affordable consumer displays on the market. Because of this, HDR videos must be processed by tone-mapping them to reduced bit-depths to service a broad swath of SDR-limited video consumers. Here, we analyzed the impact of tone-mapping operators on the visual quality of streaming HDR videos by building the first large-scale subjectively annotated open-source database of compressed tone-mapped HDR videos, containing 15,000 tone-mapped sequences derived from 40 unique HDR source contents. The videos in the database were labeled with more than 750,000 subjective quality annotations, collected from more than 1,600 unique human observers. We envision that the new LIVE Tone-Mapped HDR (LIVE-TMHDR) database will enable significant progress on HDR video tone mapping and quality assessment in the future. To this end, we make the database freely available to the community at https://live.ece.utexas.edu/research/LIVE_TMHDR/index.html. Server-side tone-mapping involves automating decisions regarding the choices of tone-mapping operators (TMOs) and their parameters to yield high-fidelity outputs. Moreover, these choices must be balanced against the effects of lossy compression, which is ubiquitous in streaming scenarios. To automate this process, we developed a novel, efficient model of objective video quality named Cut-FUNQUE that is able to accurately predict the visual quality of tone-mapped and compressed HDR videos. By evaluating Cut-FUNQUE on the LIVE-TMHDR database, we show that it achieves state-of-the-art accuracy. Finally, the deep learning revolution has strongly impacted low-level image processing tasks such as style/domain transfer, enhancement/restoration, and visual quality assessments. Despite often being treated separately, the aforementioned tasks share a common theme of understanding, editing, or enhancing the appearance of input images without modifying the underlying content. We leverage this observation to develop a novel disentangled representation learning method that decomposes inputs into content and appearance features. The model is trained in a self-supervised manner and we use the learned features to develop a new quality prediction model named DisQUE. We demonstrate through extensive evaluations that DisQUE achieves state-of-the-art accuracy across quality prediction tasks and distortion types. Moreover, we demonstrate that the same features may also be used for image processing tasks such as HDR tone mapping, where the desired output characteristics may be tuned using example input-output pairs.Electrical and Computer Engineerin

High-fidelity imaging : the computational models of the human visual system in high dynamic range video compression, visible difference prediction and image processing

As new displays and cameras offer enhanced color capabilities, there is a need to extend the precision of digital content. High Dynamic Range (HDR) imaging encodes images and video with higher than normal bit-depth precision, enabling representation of the complete color gamut and the full visible range of luminance. This thesis addresses three problems of HDR imaging: the measurement of visible distortions in HDR images, lossy compression for HDR video, and artifact-free image processing. To measure distortions in HDR images, we develop a visual difference predictor for HDR images that is based on a computational model of the human visual system. To address the problem of HDR image encoding and compression, we derive a perceptually motivated color space for HDR pixels that can efficiently encode all perceivable colors and distinguishable shades of brightness. We use the derived color space to extend the MPEG-4 video compression standard for encoding HDR movie sequences. We also propose a backward-compatible HDR MPEG compression algorithm that encodes both a low-dynamic range and an HDR video sequence into a single MPEG stream. Finally, we propose a framework for image processing in the contrast domain. The framework transforms an image into multi-resolution physical contrast images (maps), which are then rescaled in just-noticeable-difference (JND) units. The application of the framework is demonstrated with a contrast-enhancing tone mapping and a color to gray conversion that preserves color saliency.Aktuelle Innovationen in der Farbverarbeitung bei Bildschirmen und Kameras erzwingen eine Präzisionserweiterung bei digitalen Medien. High Dynamic Range (HDR) kodieren Bilder und Video mit einer grösseren Bittiefe pro Pixel, und ermöglichen damit die Darstellung des kompletten Farbraums und aller sichtbaren Helligkeitswerte. Diese Arbeit konzentriert sich auf drei Probleme in der HDR-Verarbeitung: Messung von für den Menschen störenden Fehlern in HDR-Bildern, verlustbehaftete Kompression von HDR-Video, und visuell verlustfreie HDR-Bildverarbeitung. Die Messung von HDR-Bildfehlern geschieht mittels einer Vorhersage von sichtbaren Unterschieden zweier HDR-Bilder. Die Vorhersage basiert dabei auf einer Modellierung der menschlichen Sehens. Wir addressieren die Kompression und Kodierung von HDR-Bildern mit der Ableitung eines perzeptuellen Farbraums für HDR-Pixel, der alle wahrnehmbaren Farben und deren unterscheidbaren Helligkeitsnuancen effizient abbildet. Danach verwenden wir diesen Farbraum für die Erweiterung des MPEG-4 Videokompressionsstandards, welcher sich hinfort auch für die Kodierung von HDR-Videosequenzen eignet. Wir unterbreiten weiters eine rückwärts-kompatible MPEG-Kompression von HDR-Material, welche die übliche YUV-Bildsequenz zusammen mit dessen HDRVersion in einen gemeinsamen MPEG-Strom bettet. Abschliessend erklären wir unser Framework zur Bildverarbeitung in der Kontrastdomäne. Das Framework transformiert Bilder in mehrere physikalische Kontrastauflösungen, um sie danach in Einheiten von just-noticeable-difference (JND, noch erkennbarem Unterschied) zu reskalieren. Wir demonstrieren den Nutzen dieses Frameworks anhand von einem kontrastverstärkenden Tone Mapping-Verfahren und einer Graukonvertierung, die die urspr ünglichen Farbkontraste bestmöglich beibehält