Gamut extension algorithm development and evaluation for the mapping of standard image content to wide-gamut displays

Wide-gamut display technology has provided an excellent opportunity to produce visually pleasing images, more so than in the past. However, through several studies, including Laird and Heynderick, 2008, it was shown that linearly mapping the standard sRGB content to the gamut boundary of a given wide-gamut display may not result in optimal results. Therefore, several algorithms were developed and evaluated for observer preference, including both linear and sigmoidal expansion algorithms, in an effort to define a single, versatile gamut expansion algorithm (GEA) that can be applied to current display technology and produce the most preferable images for observers. The outcome provided preference results from two displays, both of which resulted in large scene dependencies. However, the sigmoidal GEAs (SGEA) were competitive with the linear GEAs (LGEA), and in many cases, resulted in more pleasing reproductions. The SGEAs provide an excellent baseline, in which, with minor improvements, could be key to producing more impressive images on a wide-gamut display

Printing Beyond Color: Spectral and Specular Reproduction

For accurate printing (reproduction), two important appearance attributes to consider are color and gloss. These attributes are related to two topics focused on in this dissertation: spectral reproduction and specular (gloss) printing. In the conventional printing workflow known as the metameric printing workflow, which we use mostly nowadays, high-quality prints -- in terms of colorimetric accuracy -- can be achieved only under a predefined illuminant (i.e. an illuminant that the printing pipeline is adjusted to; e.g. daylight). While this printing workflow is useful and sufficient for many everyday purposes, in some special cases, such as artwork (e.g. painting) reproduction, security printing, accurate industrial color communication and so on, in which accurate reproduction of an original image under a variety of illumination conditions (e.g. daylight, tungsten light, museum light, etc.) is required, metameric reproduction may produce satisfactory results only with luck. Therefore, in these cases, another printing workflow, known as spectral printing pipeline must be used, with the ideal aim of illuminant-invariant match between the original image and the reproduction. In this workflow, the reproduction of spectral raw data (i.e. reflectances in the visible wavelength range), rather than reproduction of colorimetric values (colors) alone (under a predefined illuminant) is taken into account. Due to the limitations of printing systems extant, the reproduction of all reflectances is not possible even with multi-channel (multi-colorant) printers. Therefore, practical strategies are required in order to map non-reproducible reflectances into reproducible spectra and to choose appropriate combinations of printer colorants for the reproduction of the mapped reflectances. For this purpose, an approach called Spatio-Spectral Gamut Mapping and Separation, SSGMS, was proposed, which results in almost artifact-free spectral reproduction under a set of various illuminants. The quality control stage is usually the last stage in any printing pipeline. Nowadays, the quality of the printout is usually controlled only in terms of colorimetric accuracy and common printing artifacts. However, some gloss-related artifacts, such as gloss-differential (inconsistent gloss appearance across an image, caused mostly by variations in deposited ink area coverage on different spots), are ignored, because no strategy to avoid them exists. In order to avoid such gloss-related artifacts and to control the glossiness of the printout locally, three printing strategies were proposed. In general, for perceptually accurate reproduction of color and gloss appearance attributes, understanding the relationship between measured values and perceived magnitudes of these attributes is essential. There has been much research into reproduction of colors within perceptually meaningful color spaces, but little research from the gloss perspective has been carried out. Most of these studies are based on simulated display-based images (mostly with neutral colors) and do not take real objects into account. In this dissertation, three psychophysical experiments were conducted in order to investigate the relationship between measured gloss values (objective quantities) and perceived gloss magnitudes (subjective quantities) using real colored samples printed by the aforementioned proposed printing strategies. These experiments revealed that the relationship mentioned can be explained by a Power function according to Stevens' Power Law, considering almost the entire gloss range. Another psychophysical experiment was also conducted in order to investigate the interrelation between perceived surface gloss and texture, using 2.5D samples printed in two different texture types and with various gloss levels and texture elevations. According to the results of this experiment, different macroscopic texture types and levels (in terms of texture elevation) were found to influence the perceived surface gloss level slightly. No noticeable influence of surface gloss on the perceived texture level was observed, indicating texture constancy regardless of the gloss level printed. The SSGMS approach proposed for the spectral reproduction, the three printing strategies presented for gloss printing, and the results of the psychophysical experiments conducted on gloss printing and appearance can be used to improve the overall print quality in terms of color and gloss reproduction

Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

A Paradigm for color gamut mapping of pictorial images

In this thesis, a paradigm was generated for color gamut mapping of pictorial images. This involved the development and testing of: 1.) a hue-corrected version of the CIELAB color space, 2.) an image-dependent sigmoidal-lightness-rescaling process, 3.) an image-gamut- based chromatic-compression process, and 4.) a gamut-expansion process. This gamut-mapping paradigm was tested against some gamut-mapping strategies published in the literature. Reproductions generated by gamut mapping in a hue-corrected CIELAB color space more accurately preserved the perceived hue of the original scenes compared to reproductions generated using the CIELAB color space. The results of three gamut-mapping experiments showed that the contrast-preserving nature of the sigmoidal-lightness-remapping strategy generated gamut-mapped reproductions that were better matches to the originals than reproductions generated using linear-lightness-compression functions. In addition, chromatic-scaling functions that compressed colors at a higher rate near the gamut surface and less near the achromatic axis produced better matches to the originals than algorithms that performed linear chroma compression throughout color space. A constrained gamut-expansion process, similar to the inverse of the best gamut-compression process found in this experiment, produced reproductions preferred over an expansion process utilizing unconstrained linear expansion

Intuitive and Accurate Material Appearance Design and Editing

Creating and editing high-quality materials for photorealistic rendering can be a difficult task due to the diversity and complexity of material appearance. Material design is the process by which artists specify the reflectance properties of a surface, such as its diffuse color and specular roughness. Even with the support of commercial software packages, material design can be a time-consuming trial-and-error task due to the counter-intuitive nature of the complex reflectance models. Moreover, many material design tasks require the physical realization of virtually designed materials as the final step, which makes the process even more challenging due to rendering artifacts and the limitations of fabrication. In this dissertation, we propose a series of studies and novel techniques to improve the intuitiveness and accuracy of material design and editing. Our goal is to understand how humans visually perceive materials, simplify user interaction in the design process and, and improve the accuracy of the physical fabrication of designs. Our first work focuses on understanding the perceptual dimensions for measured material data. We build a perceptual space based on a low-dimensional reflectance manifold that is computed from crowd-sourced data using a multi-dimensional scaling model. Our analysis shows the proposed perceptual space is consistent with the physical interpretation of the measured data. We also put forward a new material editing interface that takes advantage of the proposed perceptual space. We visualize each dimension of the manifold to help users understand how it changes the material appearance. Our second work investigates the relationship between translucency and glossiness in material perception. We conduct two human subject studies to test if subsurface scattering impacts gloss perception and examine how the shape of an object influences this perception. Based on our results, we discuss why it is necessary to include transparent and translucent media for future research in gloss perception and material design. Our third work addresses user interaction in the material design system. We present a novel Augmented Reality (AR) material design prototype, which allows users to visualize their designs against a real environment and lighting. We believe introducing AR technology can make the design process more intuitive and improve the authenticity of the results for both novice and experienced users. To test this assumption, we conduct a user study to compare our prototype with the traditional material design system with gray-scale background and synthetic lighting. The results demonstrate that with the help of AR techniques, users perform better in terms of objectively measured accuracy and time and they are subjectively more satisfied with their results. Finally, our last work turns to the challenge presented by the physical realization of designed materials. We propose a learning-based solution to map the virtually designed appearance to a meso-scale geometry that can be easily fabricated. Essentially, this is a fitting problem, but compared with previous solutions, our method can provide the fabrication recipe with higher reconstruction accuracy for a large fitting gamut. We demonstrate the efficacy of our solution by comparing the reconstructions with existing solutions and comparing fabrication results with the original design. We also provide an application of bi-scale material editing using the proposed method

A Black-Point Adaption model for color reproduction

Based on the current state of CIECAM97s, there is a missing adjustment associated with a black-point unlike a white-point. As an attempt to improve the performance of CIECAM97s for color reproduction, six algorithms focusing on black-point adaptation were generated based on previous work on white-point adaptation methods and gamut mapping methods. The six algorithms were used to reproduce four original images targeted to four simulated hard-copy viewing environments that were only differentiated by their black-point settings. Then, the six algorithms were tested in a psychophysical experiment with 32 observers. As a result, linear lightness rescaling under the luminances of white and black of a specific setting was demonstrated to be the best color reproduction method across different black-point settings. The adapted black-point was defined as having the lowest lightness value with its default chromatic appearance correlates predicted by the current state of CIECAM97s under the input viewing environment and was reproduced accordingly with the same appearance correlates

The effect of image size on the color appearance of image reproductions

Original and reproduced art are usually viewed under quite different viewing conditions. One of the interesting differences in viewing condition is size difference. The main focus of this research was investigation of the effect of image size on color perception of rendered images. This research had several goals. The first goal was to develop an experimental paradigm for measuring the effect of image size on color appearance. The second goal was to identify the most affected image attributes for changes of image size. The final goal was to design and evaluate algorithms to compensate for the change of visual angle (size). To achieve the first goal, an exploratory experiment was performed using a colorimetrically characterized digital projector and LCD. The projector and LCD were light emitting devices and in this sense were similar soft-copy media. The physical sizes of the reproduced images on the LCD and projector screen could be very different. Additionally, one could benefit from flexibility of soft-copy reproduction devices such as real-time image rendering, which is essential for adjustment experiments. The capability of the experimental paradigm in revealing the change of appearance for a change of visual angle (size) was demonstrated by conducting a paired-comparison experiment. Through contrast matching experiments, achromatic and chromatic contrast and mean luminance of an image were identified as the most affected attributes for changes of image size. Measurement of the extent and trend of changes for each attribute were measured using matching experiments. Proper algorithms to compensate for the image size effect were design and evaluated. The correction algorithms were tested versus traditional colorimetric image rendering using a paired-comparison technique. The paired-comparison results confirmed superiority of the algorithms over the traditional colorimetric image rendering for the size effect compensation

Using Decoupled Features for Photo-realistic Style Transfer

In this work we propose a photorealistic style transfer method for image and video that is based on vision science principles and on a recent mathematical formulation for the deterministic decoupling of sample statistics. The novel aspects of our approach include matching decoupled moments of higher order than in common style transfer approaches, and matching a descriptor of the power spectrum so as to characterize and transfer diffusion effects between source and target, which is something that has not been considered before in the literature. The results are of high visual quality, without spatio-temporal artifacts, and validation tests in the form of observer preference experiments show that our method compares very well with the state-of-the-art. The computational complexity of the algorithm is low, and we propose a numerical implementation that is amenable for real-time video application. Finally, another contribution of our work is to point out that current deep learning approaches for photorealistic style transfer don't really achieve photorealistic quality outside of limited examples, because the results too often show unacceptable visual artifacts

Crowd-sourced data and its applications for new algorithms in photographic imaging

This thesis comprises two main themes. The first of these is concerned primarily with the validity and utility of data acquired from web-based psychophysical experiments. In recent years web-based experiments, and the crowd-sourced data they can deliver, have been rising in popularity among the research community for several key reasons – primarily ease of administration and easy access to a large population of diverse participants. However, the level of control with which traditional experiments are performed, and the severe lack of control we have over web-based alternatives may lead us to believe that these benefits come at the cost of reliable data. Indeed, the results reported early in this thesis support this assumption. However, we proceed to show that it is entirely possible to crowd-source data that is comparable with lab-based results. The second theme of the thesis explores the possibilities presented by the use of crowd-sourced data, taking a popular colour naming experiment as an example. After using the crowd-sourced data to construct a model for computational colour naming, we consider the value of colour names as image descriptors, with particular relevance to illuminant estimation and object indexing. We discover that colour names represent a particularly useful quantisation of colour space, allowing us to construct compact image descriptors for object indexing. We show that these descriptors are somewhat tolerant to errors in illuminant estimation and that their perceptual relevance offers even further utility. We go on to develop a novel algorithm which delivers perceptually-relevant, illumination-invariant image descriptors based on colour names