Spectral imaging of human portraits and image quality

This dissertation addresses the problem of capturing spectral images for human portraits and evaluating image quality of spectral images. A new spectral imaging approach is proposed in this dissertation for spectral images of human portraits. Thorough statistical analysis is performed for spectral reflectances from various races and different face parts. A spectral imaging system has been designed and calibrated for human portraits. The calibrated imaging system has the ability to represent not only the facial skin but also the spectra of lips, eyes and hair from various races as well. The generated spectral images can be applied to color-imaging system design and analysis. To evaluate the image quality of spectral imaging systems, a visual psychophysical image quality experiment has been performed in this dissertation. The spectral images were simulated based on real spectral imaging system. Meaningful image quality results have been obtained for spectral images generated from different spectral imaging systems. To bridge the gap between the physical measures and subjective visual perceptions of image quality, four image distortion factors were defined. Image quality metrics were obtained and evaluated based statistical analysis and multiple analysis. The image quality metrics have high correlation with subjective assessment for image quality. The image quality contribution of the distortion factors were evaluated. As an extension of the work other researchers in MCSL have initiated, this dissertation research will, working with other researchers in MCSL, put effort to build a publicly accessible database of spectral images, Lippmann2000

Scene-Dependency of Spatial Image Quality Metrics

This thesis is concerned with the measurement of spatial imaging performance and the modelling of spatial image quality in digital capturing systems. Spatial imaging performance and image quality relate to the objective and subjective reproduction of luminance contrast signals by the system, respectively; they are critical to overall perceived image quality. The Modulation Transfer Function (MTF) and Noise Power Spectrum (NPS) describe the signal (contrast) transfer and noise characteristics of a system, respectively, with respect to spatial frequency. They are both, strictly speaking, only applicable to linear systems since they are founded upon linear system theory. Many contemporary capture systems use adaptive image signal processing, such as denoising and sharpening, to optimise output image quality. These non-linear processes change their behaviour according to characteristics of the input signal (i.e. the scene being captured). This behaviour renders system performance “scene-dependent” and difficult to measure accurately. The MTF and NPS are traditionally measured from test charts containing suitable predefined signals (e.g. edges, sinusoidal exposures, noise or uniform luminance patches). These signals trigger adaptive processes at uncharacteristic levels since they are unrepresentative of natural scene content. Thus, for systems using adaptive processes, the resultant MTFs and NPSs are not representative of performance “in the field” (i.e. capturing real scenes). Spatial image quality metrics for capturing systems aim to predict the relationship between MTF and NPS measurements and subjective ratings of image quality. They cascade both measures with contrast sensitivity functions that describe human visual sensitivity with respect to spatial frequency. The most recent metrics designed for adaptive systems use MTFs measured using the dead leaves test chart that is more representative of natural scene content than the abovementioned test charts. This marks a step toward modelling image quality with respect to real scene signals. This thesis presents novel scene-and-process-dependent MTFs (SPD-MTF) and NPSs (SPDNPS). They are measured from imaged pictorial scene (or dead leaves target) signals to account for system scene-dependency. Further, a number of spatial image quality metrics are revised to account for capture system and visual scene-dependency. Their MTF and NPS parameters were substituted for SPD-MTFs and SPD-NPSs. Likewise, their standard visual functions were substituted for contextual detection (cCSF) or discrimination (cVPF) functions. In addition, two novel spatial image quality metrics are presented (the log Noise Equivalent Quanta (NEQ) and Visual log NEQ) that implement SPD-MTFs and SPD-NPSs. The metrics, SPD-MTFs and SPD-NPSs were validated by analysing measurements from simulated image capture pipelines that applied either linear or adaptive image signal processing. The SPD-NPS measures displayed little evidence of measurement error, and the metrics performed most accurately when they used SPD-NPSs measured from images of scenes. The benefit of deriving SPD-MTFs from images of scenes was traded-off, however, against measurement bias. Most metrics performed most accurately with SPD-MTFs derived from dead leaves signals. Implementing the cCSF or cVPF did not increase metric accuracy. The log NEQ and Visual log NEQ metrics proposed in this thesis were highly competitive, outperforming metrics of the same genre. They were also more consistent than the IEEE P1858 Camera Phone Image Quality (CPIQ) metric when their input parameters were modified. The advantages and limitations of all performance measures and metrics were discussed, as well as their practical implementation and relevant applications

Flat panel display signal processing

Televisions (TVs) have shown considerable technological progress since their introduction almost a century ago. Starting out as small, dim and monochrome screens in wooden cabinets, TVs have evolved to large, bright and colorful displays in plastic boxes. It took until the turn of the century, however, for the TV to become like a ‘picture on the wall’. This happened when the bulky Cathode Ray Tube (CRT) was replaced with thin and light-weight Flat Panel Displays (FPDs), such as Liquid Crystal Displays (LCDs) or Plasma Display Panels (PDPs). However, the TV system and transmission formats are still strongly coupled to the CRT technology, whereas FPDs use very different principles to convert the electronic video signal to visible images. These differences result in image artifacts that the CRT never had, but at the same time provide opportunities to improve FPD image quality beyond that of the CRT. This thesis presents an analysis of the properties of flat panel displays, their relation to image quality, and video signal processing algorithms to improve the quality of the displayed images. To analyze different types of displays, the display signal chain is described using basic principles common to all displays. The main function of a display is to create visible images (light) from an electronic signal (video), requiring display chain functions like opto-electronic effect, spatial and temporal addressing and reconstruction, and color synthesis. The properties of these functions are used to describe CRT, LCDs, and PDPs, showing that these displays perform the same functions, using different implementations. These differences have a number of consequences, that are further investigated in this thesis. Spatial and temporal aspects, corresponding to ‘static’ and ‘dynamic’ resolution respectively, are covered in detail. Moreover, video signal processing is an essential part of the display signal chain for FPDs, because the display format will in general no longer match the source format. In this thesis, it is investigated how specific FPD properties, especially related to spatial and temporal addressing and reconstruction, affect the video signal processing chain. A model of the display signal chain is presented, and applied to analyze FPD spatial properties in relation to static resolution. In particular, the effect of the color subpixels, that enable color image reproduction in FPDs, is analyzed. The perceived display resolution is strongly influenced by the color subpixel arrangement. When taken into account in the signal chain, this improves the perceived resolution on FPDs, which clearly outperform CRTs in this respect. The cause and effect of this improvement, also for alternative subpixel arrangements, is studied using the display signal model. However, the resolution increase cannot be achieved without video processing. This processing is efficiently combined with image scaling, which is always required in the FPD display signal chain, resulting in an algorithm called ‘subpixel image scaling’. A comparison of the effects of subpixel scaling on several subpixel arrangements shows that the largest increase in perceived resolution is found for two-dimensional subpixel arrangements. FPDs outperform CRTs with respect to static resolution, but not with respect to ‘dynamic resolution’, i.e. the perceived resolution of moving images. Life-like reproduction of moving images is an important requirement for a TV display, but the temporal properties of FPDs cause artifacts in moving images (‘motion artifacts’), that are not found in CRTs. A model of the temporal aspects of the display signal chain is used to analyze dynamic resolution and motion artifacts on several display types, in particular LCD and PDP. Furthermore, video signal processing algorithms are developed that can reduce motion artifacts and increase the dynamic resolution. The occurrence of motion artifacts is explained by the fact that the human visual system tracks moving objects. This converts temporal effects on the display into perceived spatial effects, that can appear in very different ways. The analysis shows how addressing mismatches in the chain cause motion-dependent misalignment of image data, e.g. resulting in the ‘dynamic false contour’ artifact in PDPs. Also, non-ideal temporal reconstruction results in ‘motion blur’, i.e. a loss of sharpness of moving images, which is typical for LCDs. The relation between motion blur, dynamic resolution, and temporal properties of LCDs is analyzed using the display signal model in the temporal (frequency) domain. The concepts of temporal aperture, motion aperture and temporal display bandwidth are introduced, which enable characterization of motion blur in a simple and direct way. This is applied to compare several motion blur reduction methods, based on modified display design and driving. This thesis further describes the development of several video processing algorithms that can reduce motion artifacts. It is shown that the motion of objects in the image plays an essential role in these algorithms, i.e. they require motion estimation and compensation techniques. In LCDs, video processing for motion artifact reduction involves a compensation for the temporal reconstruction characteristics of the display, leading to the ‘motion compensated inverse filtering’ algorithm. The display chain model is used to analyze this algorithm, and several methods to increase its performance are presented. In PDPs, motion artifact reduction can be achieved with ‘motion compensated subfield generation’, for which an advanced algorithm is presented

Digital Image Processing

This book presents several recent advances that are related or fall under the umbrella of 'digital image processing', with the purpose of providing an insight into the possibilities offered by digital image processing algorithms in various fields. The presented mathematical algorithms are accompanied by graphical representations and illustrative examples for an enhanced readability. The chapters are written in a manner that allows even a reader with basic experience and knowledge in the digital image processing field to properly understand the presented algorithms. Concurrently, the structure of the information in this book is such that fellow scientists will be able to use it to push the development of the presented subjects even further

The effect of scene content on image quality

Device-dependent metrics attempt to predict image quality from an ‘average signal’, usually embodied on test targets. Consequently, the metrics perform well on individual ‘average looking’ scenes and test targets, but provide lower correlation with subjective assessments when working with a variety of scenes with different than ‘average signal’ characteristics. This study considers the issues of scene dependency on image quality. This study aims to quantify the change in quality with scene contents, to research the problem of scene dependency in relation to devicedependent image quality metrics and to provide a solution to it. A novel subjective scaling method was developed in order to derive individual attribute scales, using the results from the overall image quality assessments. This was an analytical top-down approach, which does not require separate scaling of individual attributes and does not assume that the attribute is not independent from other attributes. From the measurements, interval scales were created and the effective scene dependency factor was calculated, for each attribute. Two device-dependent image quality metrics, the Effective Pictorial Information Capacity (EPIC) and the Perceived Information Capacity (PIC), were used to predict subjective image quality for a test set that varied in sharpness and noisiness. These metrics were found to be reliable predictors of image quality. However, they were not equally successful in predicting quality for different images with varying scene content. Objective scene classification was thus considered and employed in order to deal with the problem of scene dependency in device-dependent metrics. It used objective scene descriptors, which correlated with subjective criteria on scene susceptibility. This process resulted in the development of a fully automatic classification of scenes into ‘standard’ and ‘non-standard’ groups, and the result allows the calculation of calibrated metric values for each group. The classification and metric calibration performance was quite encouraging, not only because it improved mean image quality predictions from all scenes, but also because it catered for nonstandard scenes, which originally produced low correlations. The findings indicate that the proposed automatic scene classification method has great potential for tackling the problem of scene dependency, when modelling device-dependent image quality. In addition, possible further studies of objective scene classification are discussed

New acquisition techniques for real objects and light sources in computer graphics

Accurate representations of objects and light sources in a scene model are a crucial prerequisite for realistic image synthesis using computer graphics techniques. This thesis presents techniques for the effcient acquisition of real world objects and real world light sources, as well as an assessment of the quality of the acquired models. Making use of color management techniques, we setup an appearance reproduction pipeline that ensures best-possible reproduction of local light reflection with the available input and output devices. We introduce a hierarchical model for the subsurface light transport in translucent objects, derive an acquisition methodology, and acquire models of several translucent objects that can be rendered interactively. Since geometry models of real world objects are often acquired using 3D range scanners, we also present a method based on the concept of modulation transfer functions to evaluate their accuracy. In order to illuminate a scene with realistic light sources, we propose a method to acquire a model of the near-field emission pattern of a light source with optical prefiltering. We apply this method to several light sources with different emission characteristics and demonstrate the integration of the acquired models into both, global illumination as well as hardware-accelerated rendering systems.Exakte Repräsentationen der Objekte und Lichtquellen in einem Modell einer Szene sind eine unerlässliche Voraussetzung für die realistische Bilderzeugung mit Techniken der Computergraphik. Diese Dissertation beschäftigt sich mit der effizienten Digitalisierung von realen Objekten und realen Lichtquellen. Dabei werden sowohl neue Digitalisierungstechniken als auch Methoden zur Bestimmung der Qualität der erzeugten Modelle vorgestellt. Wir schlagen eine Verarbeitungskette zur Digitalisierung und Wiedergabe der Farbe und Spekularität von Objekten vor, die durch Ausnutzung von Farbmanagementtechniken eine bestmögliche Wiedergabe des Objekts unter Verwendung der gegebenen Ein- und Ausgabegeräte ermöglicht. Wir führen weiterhin ein hierarchisches Modell für den Lichttransport im Inneren von Objekten aus durchscheinenden Materialien sowie eine zugehörige Akquisitionsmethode ein und digitalisieren mehrere reale Objekte. Die dabei erzeugten Modelle können in Echtzeit angezeigt werden. Die Geometrie realer Objekte spielt eine entscheidende Rolle in vielen Anwendungen und wird oftmals unter Verwendung von 3D Scannern digitalisiert. Wir entwickeln daher eine Methode zur Bestimmung der Genauigkeit eines 3D Scanners, die auf dem Konzept der Modulationstransferfunktion basiert. Um eine Szene mit realen Lichtquellen beleuchten zu können, schlagen wir ferner eine Methode zur Erfassung der Nahfeldabstrahlung eine Lichtquelle vor, bei der vor der Digitalisierung ein optischer Filterungsschritt durchgeführt wird. Wir wenden diese Methode zur Digitalisierung mehrerer Lichtquellen mit unterschiedlichen Abstrahlcharakteristika an und zeigen auf, wie die dabei erzeugten Modelle in globalen Beleuchtungsberechnungen sowie bei der Bildsynthese mittels moderner Graphikkarten verwendet werden können

Opto-mechanical eye models, a review on human vision applications and perspectives for use in industry

The purpose of this review is to aggregate technical information on existent optomechanical eye models (OME) described in the literature, for image quality assessment in different applications. Several physical eye models have been reviewed from peer-reviewed papers and patent applications. A typical eye model includes an artificial cornea, an intraocular lens or other lens to simulate the crystalline lens, an aperture as the pupil, and a posterior retinal surface, which may be connected to a light sensor. The interior of the eye model may be filled with a fluid to better emulate physiological conditions. The main focus of this review is the materials and physical characteristics used and the dimensional aspects of the main components including lenses, apertures, chambers, imaging sensors and filling medium. Various devices are described with their applications and technical details, which are systematically tabulated highlighting their main characteristics and applications. The models presented are detailed and discussed individually, and the features of different models are compared when applicable, highlighting strengths and limitations. In the end there is a brief discussion about the potential use of artificial eye models for industrial applications.This work is supported by European Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and Internationalization Programme (COMPETE 2020) [Project nº 39479; Funding Reference: POCI-01-0247-FEDER-39479]

High-dynamic-range Foveated Near-eye Display System

Wearable near-eye display has found widespread applications in education, gaming, entertainment, engineering, military training, and healthcare, just to name a few. However, the visual experience provided by current near-eye displays still falls short to what we can perceive in the real world. Three major challenges remain to be overcome: 1) limited dynamic range in display brightness and contrast, 2) inadequate angular resolution, and 3) vergence-accommodation conflict (VAC) issue. This dissertation is devoted to addressing these three critical issues from both display panel development and optical system design viewpoints. A high-dynamic-range (HDR) display requires both high peak brightness and excellent dark state. In the second and third chapters, two mainstream display technologies, namely liquid crystal display (LCD) and organic light emitting diode (OLED), are investigated to extend their dynamic range. On one hand, LCD can easily boost its peak brightness to over 1000 nits, but it is challenging to lower the dark state to \u3c 0.01 nits. To achieve HDR, we propose to use a mini-LED local dimming backlight. Based on our simulations and subjective experiments, we establish practical guidelines to correlate the device contrast ratio, viewing distance, and required local dimming zone number. On the other hand, self-emissive OLED display exhibits a true dark state, but boosting its peak brightness would unavoidably cause compromised lifetime. We propose a systematic approach to enhance OLED\u27s optical efficiency while keeping indistinguishable angular color shift. These findings will shed new light to guide future HDR display designs. In Chapter four, in order to improve angular resolution, we demonstrate a multi-resolution foveated display system with two display panels and an optical combiner. The first display panel provides wide field of view for peripheral vision, while the second panel offers ultra-high resolution for the central fovea. By an optical minifying system, both 4x and 5x enhanced resolutions are demonstrated. In addition, a Pancharatnam-Berry phase deflector is applied to actively shift the high-resolution region, in order to enable eye-tracking function. The proposed design effectively reduces the pixelation and screen-door effect in near-eye displays. The VAC issue in stereoscopic displays is believed to be the main cause of visual discomfort and fatigue when wearing VR headsets. In Chapter five, we propose a novel polarization-multiplexing approach to achieve multiplane display. A polarization-sensitive Pancharatnam-Berry phase lens and a spatial polarization modulator are employed to simultaneously create two independent focal planes. This method enables generation of two image planes without the need of temporal multiplexing. Therefore, it can effectively reduce the frame rate by one-half. In Chapter six, we briefly summarize our major accomplishments