Digital Color Imaging

This paper surveys current technology and research in the area of digital color imaging. In order to establish the background and lay down terminology, fundamental concepts of color perception and measurement are first presented us-ing vector-space notation and terminology. Present-day color recording and reproduction systems are reviewed along with the common mathematical models used for representing these devices. Algorithms for processing color images for display and communication are surveyed, and a forecast of research trends is attempted. An extensive bibliography is provided

Camera based Display Image Quality Assessment

This thesis presents the outcomes of research carried out by the PhD candidate Ping Zhao during 2012 to 2015 in Gjøvik University College. The underlying research was a part of the HyPerCept project, in the program of Strategic Projects for University Colleges, which was funded by The Research Council of Norway. The research was engaged under the supervision of Professor Jon Yngve Hardeberg and co-supervision of Associate Professor Marius Pedersen, from The Norwegian Colour and Visual Computing Laboratory, in the Faculty of Computer Science and Media Technology of Gjøvik University College; as well as the co-supervision of Associate Professor Jean-Baptiste Thomas, from The Laboratoire Electronique, Informatique et Image, in the Faculty of Computer Science of Universit´e de Bourgogne. The main goal of this research was to develop a fast and an inexpensive camera based display image quality assessment framework. Due to the limited time frame, we decided to focus only on projection displays with static images displayed on them. However, the proposed methods were not limited to projection displays, and they were expected to work with other types of displays, such as desktop monitors, laptop screens, smart phone screens, etc., with limited modifications. The primary contributions from this research can be summarized as follows: 1. We proposed a camera based display image quality assessment framework, which was originally designed for projection displays but it can be used for other types of displays with limited modifications. 2. We proposed a method to calibrate the camera in order to eliminate unwanted vignetting artifact, which is mainly introduced by the camera lens. 3. We proposed a method to optimize the camera’s exposure with respect to the measured luminance of incident light, so that after the calibration all camera sensors share a common linear response region. 4. We proposed a marker-less and view-independent method to register one captured image with its original at a sub-pixel level, so that we can incorporate existing full reference image quality metrics without modifying them. 5. We identified spatial uniformity, contrast and sharpness as the most important image quality attributes for projection displays, and we used the proposed framework to evaluate the prediction performance of the state-of-the-art image quality metrics regarding these attributes. The proposed image quality assessment framework is the core contribution of this research. Comparing to conventional image quality assessment approaches, which were largely based on the measurements of colorimeter or spectroradiometer, using camera as the acquisition device has the advantages of quickly recording all displayed pixels in one shot, relatively inexpensive to purchase the instrument. Therefore, the consumption of time and resources for image quality assessment can be largely reduced. We proposed a method to calibrate the camera in order to eliminate unwanted vignetting artifact primarily introduced by the camera lens. We used a hazy sky as a closely uniform light source, and the vignetting mask was generated with respect to the median sensor responses over i only a few rotated shots of the same spot on the sky. We also proposed a method to quickly determine whether all camera sensors were sharing a common linear response region. In order to incorporate existing full reference image quality metrics without modifying them, an accurate registration of pairs of pixels between one captured image and its original is required. We proposed a marker-less and view-independent image registration method to solve this problem. The experimental results proved that the proposed method worked well in the viewing conditions with a low ambient light. We further identified spatial uniformity, contrast and sharpness as the most important image quality attributes for projection displays. Subsequently, we used the developed framework to objectively evaluate the prediction performance of the state-of-art image quality metrics regarding these attributes in a robust manner. In this process, the metrics were benchmarked with respect to the correlations between the prediction results and the perceptual ratings collected from subjective experiments. The analysis of the experimental results indicated that our proposed methods were effective and efficient. Subjective experiment is an essential component for image quality assessment; however it can be time and resource consuming, especially in the cases that additional image distortion levels are required to extend the existing subjective experimental results. For this reason, we investigated the possibility of extending subjective experiments with baseline adjustment method, and we found that the method could work well if appropriate strategies were applied. The underlying strategies referred to the best distortion levels to be included in the baseline, as well as the number of them

Evaluation of the color image and video processing chain and visual quality management for consumer systems

With the advent of novel digital display technologies, color processing is increasingly becoming a key aspect in consumer video applications. Today’s state-of-the-art displays require sophisticated color and image reproduction techniques in order to achieve larger screen size, higher luminance and higher resolution than ever before. However, from color science perspective, there are clearly opportunities for improvement in the color reproduction capabilities of various emerging and conventional display technologies. This research seeks to identify potential areas for improvement in color processing in a video processing chain. As part of this research, various processes involved in a typical video processing chain in consumer video applications were reviewed. Several published color and contrast enhancement algorithms were evaluated, and a novel algorithm was developed to enhance color and contrast in images and videos in an effective and coordinated manner. Further, a psychophysical technique was developed and implemented for performing visual evaluation of color image and consumer video quality. Based on the performance analysis and visual experiments involving various algorithms, guidelines were proposed for the development of an effective color and contrast enhancement method for images and video applications. It is hoped that the knowledge gained from this research will help build a better understanding of color processing and color quality management methods in consumer video

Colorimetric tolerances of various digital image displays

Visual experiments on four displays (two LCD, one CRT and hardcopy) were conducted to determine colorimetric tolerances of images systematically altered via three different transfer curves. The curves used were: Sigmoidal compression in L*, linear reduction in C*, and additive rotations in hab. More than 30 observers judged the detectability of these alterations on three pictorial images for each display. Standard probit analysis was then used to determine the detection thresholds for the alterations. It was found that the detection thresholds on LCD\u27s were similar or lower than for the CRT\u27s in this type of experiment. Summarizing pixel-by-pixel image differences using the 90th percentile color difference in E*ab was shown to be more consistent than similar measures in E94 and a prototype E2000. It was also shown that using the 90th percentile difference was more consistent than the average pixel wise difference. Furthermore, SCIELAB pre-filtering was shown to have little to no effect on the results of this experiment since only global color-changes were applied and no spatial alterations were used

Numerical techniques for Fresnel diffraction in computational holography

Optical holography can produce very realistic virtual images due to its capability to properly convey the depth cues that we use to interpret three-dimensional objects. Computational holography is the use of digital representations plus computational methods to carry out the holographic operations of construction and reconstruction. The large computational requirements of holographic simulations prohibit present-day existence of real-time holographic displays comparable in size to traditional two-dimensional displays. Fourier-based approaches to calculate the Fresnel diffraction of light provide one of the most efficient algorithms for holographic computations because this permits the use of the fast Fourier transform (FFT). The limitations on sampling imposed by Fourier-based algorithms have been overcome by the development, in this research, of a fast shifted Fresnel transform. This fast shifted Fresnel transform was used to develop a tiling approach to hologram construction and reconstruction, which computes the Fresnel propagation of light between parallel planes having different resolutions. A new method for hologram construction is presented, named partitioned hologram computation, which applies the concepts of the shifted Fresnel transform and tiling

Evaluation and improvement of the workflow of digital imaging of fine art reproduction in museums

Fine arts refer to a broad spectrum of art formats, ie~painting, calligraphy, photography, architecture, and so forth. Fine art reproductions are to create surrogates of the original artwork that are able to faithfully deliver the aesthetics and feelings of the original. Traditionally, reproductions of fine art are made in the form of catalogs, postcards or books by museums, libraries, archives, and so on (hereafter called museums for simplicity). With the widespread adoption of digital archiving in museums, more and more artwork is reproduced to be viewed on a display. For example, artwork collections are made available through museum websites and Google Art Project for art lovers to view on their own displays. In the thesis, we study the fine art reproduction of paintings in the form of soft copy viewed on displays by answering four questions: (1) what is the impact of the viewing condition and original on image quality evaluation? (2) can image quality be improved by avoiding visual editing in current workflows of fine art reproduction? (3) can lightweight spectral imaging be used for fine art reproduction? and (4) what is the performance of spectral reproductions compared with reproductions by current workflows? We started with evaluating the perceived image quality of fine art reproduction created by representative museums in the United States under controlled and uncontrolled environments with and without the presence of the original artwork. The experimental results suggest that the image quality is highly correlated with the color accuracy of the reproduction only when the original is present and the reproduction is evaluated on a characterized display. We then examined the workflows to create these reproductions, and found that current workflows rely heavily on visual editing and retouching (global and local color adjustments on the digital reproduction) to improve the color accuracy of the reproduction. Visual editing and retouching can be both time-consuming and subjective in nature (depending on experts\u27 own experience and understanding of the artwork) lowering the efficiency of artwork digitization considerably. We therefore propose to improve the workflow of fine art reproduction by (1) automating the process of visual editing and retouching in current workflows based on RGB acquisition systems and by (2) recovering the spectral reflectance of the painting with off-the-shelf equipment under commonly available lighting conditions. Finally, we studied the perceived image quality of reproductions created by current three-channel (RGB) workflows with those by spectral imaging and those based on an exemplar-based method

Illuminant metamerism on artistic paintings = Metamerija uvjetovana promjenom osvjetljenja u slikarstvu

Metamerija uvjetovana promjenom osvjetljenja sve se više koristi u različitim područjima kao što su znanosti o boji, tisku, pakiranju, vizualnoj percepciji, slikarstvu, muzejima itd. Kada se promatraju umjetničke slike metamerija se odnosi na pojavu u kojoj se čini da se različiti pigmenti ili kombinacije pigmenata podudaraju pod istim izvorom svjetlosti ali pokazuju primjetnu razliku u boji kada se gledaju pod drugim izvorom svjetla. U kontekstu umjetničkih slika, pigmenti se sastoje od različitih kemijskih spojeva koji apsorbiraju i reflektiraju specifične valne duljine svjetlosti i mogu imati različita spektralna svojstva i interakcije s uvjetima osvjetljenja, utječući na vidni sustav i percepciju boje. Karakteriziranje spektralnih karakteristika različitih boja i otisaka putem kolorimetrijskih mjerenja igra ključnu ulogu u kvantificiranju i analizi metamerije uvjetovane promjenom osvjetljenja usporedbom razlika u boji između različitih izvora svjetlosti. Ovaj doktorski rad fokusirao se na fenomen metamerije uvjetovane promjenom osvjetljenja u kontekstu umjetničkih slika izloženih različitim izvorima osvjetljenja. Sveobuhvatni cilj bio je pronaći optimalan izvor svjetlosti koji ublažava perceptivne nepravilnosti i poboljšava prosječnu vizualnu percepciju. Središnja hipoteza ove teze je da prisutnost metamerije uvjetovane promjenom osvjetljenja na slikama ovisi o specifičnim tehnikama slikanja koje se koriste. Istraživanje metamerije uvjetovane promjenom osvjetljenja dalo je uvjerljive rezultate, nedvosmisleno utvrđujući da različite tehnike slikanja daju različit izgled pod različitim uvjetima osvjetljenja, potvrđujući ideju da metamerija uvjetovana promjenom osvjetljenja igra ključnu ulogu u doživljaju umjetničkih slika. Nadalje, važno otkriće leži u učinkovitosti standardizirane LED rasvjete u ublažavanju manifestacije metamerije uvjetovane promjenom osvjetljenja, posebno kada je karakterizirana dobro definiranom spektralnom distribucijom. Glavni cilj ovoga rada usmjeren je na optimizaciju uvjeta osvjetljenja kako bi se smanjile varijacije na slikama. Postalo je očito da se kvantifikacija razlike doživljaja boja temelji na temelju razlike u boji ∆E, što dovodi do upotrebe indeksa metamerije (MI) za definiranje perceptivnih nijansi metamerije uvjetovane promjenom osvjetljenja unutar domene promatranja umjetničkih djela rađenih različitim tehnikama. U okviru ove disertacije provedeno je istraživanje kako bi se pronašle mogućnosti smanjenja metamerije uvjetovane promjenom osvjetljenja u različitim slikarskim tehnikama, standardizirala i smanjila metamerija uvjetovana promjenom osvjetljenja u percepciji slika. Korištenjem visokokvalitetnog procesa ispisa koji može točno reproducirati boje, primijenjenaje kalibracija pisača kako bi se osigurala vjernost i dosljednost boja. Ispis probnih uzoraka rađen je na mat papiru koji pomoći smanjuje refleksiji i pruža bolju percepciju sa smanjenim udjelom sjajnosti materijala. Ovo istraživanje definira usporedbu slika sa sličnim motivom i uzorkom višebojne reprodukcije, pružajući uvid u to kako ljudi percipiraju boju i doživljavaju veličinu metamerije uvjetovane promjenom osvjetljenja pod standardiziranim uvjetima različitih izvora svjetlosti: LED svjetlo 3000K, 4000K, 6500K, Wolfram svjetlo i halogeno svjetlo, odnosno pod njihovom spektralnom distribucijom snage (SPD). Zadatak ove disertacije bio je pronaći znanstvenu povezanost između metamerije uvjetovane promjenom osvjetljenja na slikama rađenim različitim slikarskim tehnikama, kao što su tempera, ulje, akril, pastel i akvarel. Proučavanje metamerije uvjetovane promjenom osvjetljenja na umjetničkim slikama provedeno je objektivnim i subjektivnim istraživanjem, pri čemu standardizirani promatrači procjenjuju podudarnosti ili razlike boja pod različitim izvorima svjetlosti. Koristeći takve modele, može se predvidjeti kako će slika izgledati pod različitim uvjetima osvjetljenja i bolje razumjeti čimbenike koji doprinose smanjenju metamerije uvjetovane promjenom osvjetljenja. Umjetničke slike, koje su primjeri sinteze različitih kemijskih spojeva, karakteriziraju pigmenti koji selektivno apsorbiraju i reflektiraju određene valne duljine svjetlosti. Odabir suptraktivnih boja s izvorne slike, predisponiranih metameriji uvjetovanoj promjenom osvjetljenja, stvara različite spektralne vrijednosti koji su u interakciji s različitim osvjetljenjima. Time se izrazito utječe na vizualni sustav, na razumijevanje doživljaja boja i manifestaciju metamerije uvjetovane promjenom osvjetljenja. Jedinstvene karakteristike pigmenta, povezane s različitim tehnikama slikanja, daju male razlike u refleksiji svjetlosti u usporedbi s ljudskim vizualnim sustavom boja. U biti, zagonetka metamerije uvjetovane promjenom osvjetljenja nalazi se unutar domene percepcije boja, zamršenog aspekta koji uvijek može stovoriti problem u doživljaju boja kod promatrača. Iako je procjena metamerije uvjetovane promjenom osvjetljenja unutar jednotonskih boja relativno jednostavna, procjena višetonskih slika pod različitim svjetlosnim uvjetima zahtjeva uspostavljanje objektivnog mjerenja koja može kvantificirati razlike u bojama.Ovo je istraživanje potkrijepilo je izvedivost mjerenja metamerije uvjetovane promjenom osvjetljenja kod višebojnih slika analizom polja balansa sive boje, precizno reproduciranog putem empirijski testiranih uzoraka. Razlike koje se mogu uočiti unutar polja sivog balansa podložne su kvantificiranju, čime se sažima bit metamerije uvjetovane promjenom osvjetljenja putem indeksa metamerije. Kako su se u radu istraživale različite slikarske tehnike u kombinaciji s raznim izvorima osvjetljenja, u radu je ustanovljeno da ljudi različite tehnike drugačije percipiraju kod različitihizvora osvjetljenja. Tako je ustanovljeno da se kod slika koje su rađene temperom metamerija uvjetovana promjenom osvjetljenja najmanji kada se koriste izvori osvjeteljanje s temperaturom boje u korelaciji (CCT) u rasponu od 3000K do 4000K. Kod ulja na platnu, najmanja metamerija uvjetovana promjenom osvjetljenja dogodio se kod temperature boje u korelaciji između 2700K i 4000K. Međutim ono što je ustanovljeno je također da kod ulja na platnu ljudi vide boje različitije, odnosno veća je dispergiranost promatrača i to prvenstveno u dijelu doživljaja razlike u svjetlini. Kod slika rađenih tehnikom akrila pokazalo se se da je metamerija uvjetovana promjenom osvjetljenja najmanji kada se koriste hladnije boje i to prvenstveno kod halogenog svjetla koje ima takvu spektralnu refleksiju. Kod ove tehnike je također ustanovljeno da ljudi radi navedene temperature boje u korelaciji bolje percipiraju boje koje su toplije, odnosno kod hladnijih boja došlo je do smanjene metamerije upravo radi interakcije s temperaturom boje svjetlosti. Kod akvarela je ustanovljeno da da se boje relativno jednako percipiraju u rasponu temperature boje u korelaciji od 2700K - 4000K, međutim, primjećeno je da se plave nijanse na takvim slikama manje percipiraju u navedenoj temperaturi boje u korelaciji, pa se ustanovilo da je za navedenu tehniku bolje osvjeteljenje hladnijeg izvora osvjetljenja s temperaturom boje u korelaciji od 6500 K. U kontekstu pastelnih umjetničkih djela, iako je najbolji raspon za najmanju metameriju uvjetovaua promjenom osvjetljenja, ustanovljeno je da se kod žutih tonaliteta ipak ustanovila manja metamerija uvjetovana promjenom osvjeteljnja, s tim da treba uzeti u obzir i različite izvore svjetlosti koji pokazuju drugačiji indeks metamerije u navedenom rasponu, što je detaljno objašnjeno u zaključcima disertacije

Perceived image contrast and observer preference

A large-scale investigation into the perception of contrast in color images was performed. Experiments were performed to determine the influence of image lightness, chroma, and sharpness transforms on perceived image contrast and observer preference. The influence of lightness, chroma, and sharpness manipulations was investigated separately by independent soft-copy, paired-comparison tests of contrast perception and image preference. The perception of contrast between images of different transforms and different subject matter was also investigated as was the perception of image contrast relative to the most preferred image manipulation. In all, five experiments of contrast perception and four experiments of image preference were performed by at least thirtytwo observers each. Results of the lightness, chroma, and sharpness-contrast experiments indicate perceived image contrast is a function of multiple image characteristics as opposed to simply being a function of the dynamic range of image intensity. In the Lightness-Contrast Experiments, images of identical white and black points were scaled to have significant differences in contrast based on their manipulations from the original image. In the Chroma-Contrast Experiments, images of identical lightness channels were scaled to have significant differences in perceived contrast due to relative chroma amount. In the Sharpness-Contrast Experiments, images of identical white and black points were scaled to have significantly different levels of perceived contrast due to sharpness. In the Scale- Linking Experiment, it was found that images of the above manipulations could be scaled similarly for perceived contrast. All scales of perceived contrast and image preference were found to be image independent among pictorial images. Empirical modeling of perceived contrast indicates differences in perceived contrast can be quantified as a function of image colorimetry, independent of original scene information

Computer Generation of Integral Images using Interpolative Shading Techniques

Research to produce artificial 3D images that duplicates the human stereovision has been ongoing for hundreds of years. What has taken millions of years to evolve in humans is proving elusive even for present day technological advancements. The difficulties are compounded when real-time generation is contemplated. The problem is one of depth. When perceiving the world around us it has been shown that the sense of depth is the result of many different factors. These can be described as monocular and binocular. Monocular depth cues include overlapping or occlusion, shading and shadows, texture etc. Another monocular cue is accommodation (and binocular to some extent) where the focal length of the crystalline lens is adjusted to view an image. The important binocular cues are convergence and parallax. Convergence allows the observer to judge distance by the difference in angle between the viewing axes of left and right eyes when both are focussing on a point. Parallax relates to the fact that each eye sees a slightly shifted view of the image. If a system can be produced that requires the observer to use all of these cues, as when viewing the real world, then the transition to and from viewing a 3D display will be seamless. However, for many 3D imaging techniques, which current work is primarily directed towards, this is not the case and raises a serious issue of viewer comfort. Researchers worldwide, in university and industry, are pursuing their approaches in the development of 3D systems, and physiological disturbances that can cause nausea in some observers will not be acceptable. The ideal 3D system would require, as minimum, accurate depth reproduction, multiviewer capability, and all-round seamless viewing. The necessity not to wear stereoscopic or polarising glasses would be ideal and lack of viewer fatigue essential. Finally, for whatever the use of the system, be it CAD, medical, scientific visualisation, remote inspection etc on the one hand, or consumer markets such as 3D video games and 3DTV on the other, the system has to be relatively inexpensive. Integral photography is a ‘real camera’ system that attempts to comply with this ideal; it was invented in 1908 but due to technological reasons was not capable of being a useful autostereoscopic system. However, more recently, along with advances in technology, it is becoming a more attractive proposition for those interested in developing a suitable system for 3DTV. The fast computer generation of integral images is the subject of this thesis; the adjective ‘fast’ being used to distinguish it from the much slower technique of ray tracing integral images. These two techniques are the standard in monoscopic computer graphics whereby ray tracing generates photo-realistic images and the fast forward geometric approach that uses interpolative shading techniques is the method used for real-time generation. Before this present work began it was not known if it was possible to create volumetric integral images using a similar fast approach as that employed by standard computer graphics, but it soon became apparent that it would be successful and hence a valuable contribution in this area. Presented herein is a full description of the development of two derived methods for producing rendered integral image animations using interpolative shading. The main body of the work is the development of code to put these methods into practice along with many observations and discoveries that the author came across during this task.The Defence and Research Agency (DERA), a contract (LAIRD) under the European Link/EPSRC photonics initiative, and DTI/EPSRC sponsorship within the PROMETHEUS project

Printing Beyond Color: Spectral and Specular Reproduction

For accurate printing (reproduction), two important appearance attributes to consider are color and gloss. These attributes are related to two topics focused on in this dissertation: spectral reproduction and specular (gloss) printing. In the conventional printing workflow known as the metameric printing workflow, which we use mostly nowadays, high-quality prints -- in terms of colorimetric accuracy -- can be achieved only under a predefined illuminant (i.e. an illuminant that the printing pipeline is adjusted to; e.g. daylight). While this printing workflow is useful and sufficient for many everyday purposes, in some special cases, such as artwork (e.g. painting) reproduction, security printing, accurate industrial color communication and so on, in which accurate reproduction of an original image under a variety of illumination conditions (e.g. daylight, tungsten light, museum light, etc.) is required, metameric reproduction may produce satisfactory results only with luck. Therefore, in these cases, another printing workflow, known as spectral printing pipeline must be used, with the ideal aim of illuminant-invariant match between the original image and the reproduction. In this workflow, the reproduction of spectral raw data (i.e. reflectances in the visible wavelength range), rather than reproduction of colorimetric values (colors) alone (under a predefined illuminant) is taken into account. Due to the limitations of printing systems extant, the reproduction of all reflectances is not possible even with multi-channel (multi-colorant) printers. Therefore, practical strategies are required in order to map non-reproducible reflectances into reproducible spectra and to choose appropriate combinations of printer colorants for the reproduction of the mapped reflectances. For this purpose, an approach called Spatio-Spectral Gamut Mapping and Separation, SSGMS, was proposed, which results in almost artifact-free spectral reproduction under a set of various illuminants. The quality control stage is usually the last stage in any printing pipeline. Nowadays, the quality of the printout is usually controlled only in terms of colorimetric accuracy and common printing artifacts. However, some gloss-related artifacts, such as gloss-differential (inconsistent gloss appearance across an image, caused mostly by variations in deposited ink area coverage on different spots), are ignored, because no strategy to avoid them exists. In order to avoid such gloss-related artifacts and to control the glossiness of the printout locally, three printing strategies were proposed. In general, for perceptually accurate reproduction of color and gloss appearance attributes, understanding the relationship between measured values and perceived magnitudes of these attributes is essential. There has been much research into reproduction of colors within perceptually meaningful color spaces, but little research from the gloss perspective has been carried out. Most of these studies are based on simulated display-based images (mostly with neutral colors) and do not take real objects into account. In this dissertation, three psychophysical experiments were conducted in order to investigate the relationship between measured gloss values (objective quantities) and perceived gloss magnitudes (subjective quantities) using real colored samples printed by the aforementioned proposed printing strategies. These experiments revealed that the relationship mentioned can be explained by a Power function according to Stevens' Power Law, considering almost the entire gloss range. Another psychophysical experiment was also conducted in order to investigate the interrelation between perceived surface gloss and texture, using 2.5D samples printed in two different texture types and with various gloss levels and texture elevations. According to the results of this experiment, different macroscopic texture types and levels (in terms of texture elevation) were found to influence the perceived surface gloss level slightly. No noticeable influence of surface gloss on the perceived texture level was observed, indicating texture constancy regardless of the gloss level printed. The SSGMS approach proposed for the spectral reproduction, the three printing strategies presented for gloss printing, and the results of the psychophysical experiments conducted on gloss printing and appearance can be used to improve the overall print quality in terms of color and gloss reproduction