Efficient and effective objective image quality assessment metrics

Acquisition, transmission, and storage of images and videos have been largely increased in recent years. At the same time, there has been an increasing demand for high quality images and videos to provide satisfactory quality-of-experience for viewers. In this respect, high dynamic range (HDR) imaging with higher than 8-bit depth has been an interesting approach in order to capture more realistic images and videos. Objective image and video quality assessment plays a significant role in monitoring and enhancing the image and video quality in several applications such as image acquisition, image compression, multimedia streaming, image restoration, image enhancement and displaying. The main contributions of this work are to propose efficient features and similarity maps that can be used to design perceptually consistent image quality assessment tools. In this thesis, perceptually consistent full-reference image quality assessment (FR-IQA) metrics are proposed to assess the quality of natural, synthetic, photo-retouched and tone-mapped images. In addition, efficient no-reference image quality metrics are proposed to assess JPEG compressed and contrast distorted images. Finally, we propose a perceptually consistent color to gray conversion method, perform a subjective rating and evaluate existing color to gray assessment metrics. Existing FR-IQA metrics may have the following limitations. First, their performance is not consistent for different distortions and datasets. Second, better performing metrics usually have high complexity. We propose in this thesis an efficient and reliable full-reference image quality evaluator based on new gradient and color similarities. We derive a general deviation pooling formulation and use it to compute a final quality score from the similarity maps. Extensive experimental results verify high accuracy and consistent performance of the proposed metric on natural, synthetic and photo retouched datasets as well as its low complexity. In order to visualize HDR images on standard low dynamic range (LDR) displays, tone-mapping operators are used in order to convert HDR into LDR. Given different depth bits of HDR and LDR, traditional FR-IQA metrics are not able to assess the quality of tone-mapped images. The existing full-reference metric for tone-mapped images called TMQI converts both HDR and LDR to an intermediate color space and measure their similarity in the spatial domain. We propose in this thesis a feature similarity full-reference metric in which local phase of HDR is compared with the local phase of LDR. Phase is an important information of images and previous studies have shown that human visual system responds strongly to points in an image where the phase information is ordered. Experimental results on two available datasets show the very promising performance of the proposed metric. No-reference image quality assessment (NR-IQA) metrics are of high interest because in the most present and emerging practical real-world applications, the reference signals are not available. In this thesis, we propose two perceptually consistent distortion-specific NR-IQA metrics for JPEG compressed and contrast distorted images. Based on edge statistics of JPEG compressed images, an efficient NR-IQA metric for blockiness artifact is proposed which is robust to block size and misalignment. Then, we consider the quality assessment of contrast distorted images which is a common distortion. Higher orders of Minkowski distance and power transformation are used to train a low complexity model that is able to assess contrast distortion with high accuracy. For the first time, the proposed model is used to classify the type of contrast distortions which is very useful additional information for image contrast enhancement. Unlike its traditional use in the assessment of distortions, objective IQA can be used in other applications. Examples are the quality assessment of image fusion, color to gray image conversion, inpainting, background subtraction, etc. In the last part of this thesis, a real-time and perceptually consistent color to gray image conversion methodology is proposed. The proposed correlation-based method and state-of-the-art methods are compared by subjective and objective evaluation. Then, a conclusion is made on the choice of the objective quality assessment metric for the color to gray image conversion. The conducted subjective ratings can be used in the development process of quality assessment metrics for the color to gray image conversion and to test their performance

Evaluation of color representation for texture analysis

Since more than 50 years texture in image material is a topic of research. Hereby, color was ignored mostly. This study compares 70 different configurations for texture analysis, using four features. For the configurations we used: (i) a gray value texture descriptor: the co-occurrence matrix and a color texture descriptor: the color correlogram, (ii) six color spaces, and (iii) several quantization schemes. A three classifier combination was used to classify the output of the configurations on the VisTex texture database. The results indicate that the use of a coarse HSV color space quantization can substantially improve texture recognition compared to various other gray and color quantization schemes

Spectrally Based Material Color Equivalency: Modeling and Manipulation

A spectrally based normalization methodology (Wpt normalization) for linearly transforming cone excitations or sensor values (sensor excitations) to a representation that preserves the perceptive concepts of lightness, chroma and hue is proposed resulting in a color space with the axes labeled W , p, t. Wpt (pronounced “Waypoint ) has been demonstrated to be an effective material color equivalency space that provides the basis for defining Material Adjustment Transforms that predict the changes in sensor excitations of material spectral reflectance colors due to variations in observer or illuminant. This is contrasted with Chromatic Adaptation Transforms that predict color appearance as defined by corresponding color experiments. Material color equivalency as provided by Wpt and Wpt normalization forms the underlying foundation of this doctoral research. A perceptually uniform material color equivalency space (“Waypoint Lab or WLab) was developed that represents a non-linear transformation of Wpt coordinates, and Euclidean WLab distances were found to not be statistically different from ∆E⋆94 and ∆E00 color differences. Sets of Wpt coordinates for variations in reflectance, illumination, or observers were used to form the basis of defining Wpt shift manifolds. WLab distances of corresponding points within or between these manifolds were utilized to define metrics for color inconstancy, metamerism, observer rendering, illuminant rendering, and differences in observing conditions. Spectral estimation and manipulation strategies are presented that preserve various aspects of “Wpt shift potential as represented by changes in Wpt shift manifolds. Two methods were explored for estimating Wpt normalization matrices based upon direct utilization of sensor excitations, and the use of a Wpt based Material Adjustment Transform to convert Cone Fundamentals to ”XYZ-like Color Matching Functions was investigated and contrasted with other methods such as direct regression and prediction of a common color matching primaries. Finally, linear relationships between Wpt and spectral reflectances were utilized to develop approaches for spectral estimation and spectral manipulation within a general spectral reflectance manipulation framework – thus providing the ability to define and achieve “spectrally preferred color rendering objectives. The presented methods of spectral estimation, spectral manipulation, and material adjustment where utilized to: define spectral reflectances for Munsell colors that minimize Wpt shift potential; manipulate spectral reflectances of actual printed characterization data sets to achieve colorimetry of reference printing conditions; and lastly to demonstrate the spectral estimation and manipulation of spectral reflectances using images and spectrally based profiles within an iccMAX color management workflow

Two Decades of Colorization and Decolorization for Images and Videos

Colorization is a computer-aided process, which aims to give color to a gray image or video. It can be used to enhance black-and-white images, including black-and-white photos, old-fashioned films, and scientific imaging results. On the contrary, decolorization is to convert a color image or video into a grayscale one. A grayscale image or video refers to an image or video with only brightness information without color information. It is the basis of some downstream image processing applications such as pattern recognition, image segmentation, and image enhancement. Different from image decolorization, video decolorization should not only consider the image contrast preservation in each video frame, but also respect the temporal and spatial consistency between video frames. Researchers were devoted to develop decolorization methods by balancing spatial-temporal consistency and algorithm efficiency. With the prevalance of the digital cameras and mobile phones, image and video colorization and decolorization have been paid more and more attention by researchers. This paper gives an overview of the progress of image and video colorization and decolorization methods in the last two decades.Comment: 12 pages, 19 figure

Gamut extension algorithm development and evaluation for the mapping of standard image content to wide-gamut displays

Wide-gamut display technology has provided an excellent opportunity to produce visually pleasing images, more so than in the past. However, through several studies, including Laird and Heynderick, 2008, it was shown that linearly mapping the standard sRGB content to the gamut boundary of a given wide-gamut display may not result in optimal results. Therefore, several algorithms were developed and evaluated for observer preference, including both linear and sigmoidal expansion algorithms, in an effort to define a single, versatile gamut expansion algorithm (GEA) that can be applied to current display technology and produce the most preferable images for observers. The outcome provided preference results from two displays, both of which resulted in large scene dependencies. However, the sigmoidal GEAs (SGEA) were competitive with the linear GEAs (LGEA), and in many cases, resulted in more pleasing reproductions. The SGEAs provide an excellent baseline, in which, with minor improvements, could be key to producing more impressive images on a wide-gamut display

Evaluation of the color image and video processing chain and visual quality management for consumer systems

With the advent of novel digital display technologies, color processing is increasingly becoming a key aspect in consumer video applications. Today’s state-of-the-art displays require sophisticated color and image reproduction techniques in order to achieve larger screen size, higher luminance and higher resolution than ever before. However, from color science perspective, there are clearly opportunities for improvement in the color reproduction capabilities of various emerging and conventional display technologies. This research seeks to identify potential areas for improvement in color processing in a video processing chain. As part of this research, various processes involved in a typical video processing chain in consumer video applications were reviewed. Several published color and contrast enhancement algorithms were evaluated, and a novel algorithm was developed to enhance color and contrast in images and videos in an effective and coordinated manner. Further, a psychophysical technique was developed and implemented for performing visual evaluation of color image and consumer video quality. Based on the performance analysis and visual experiments involving various algorithms, guidelines were proposed for the development of an effective color and contrast enhancement method for images and video applications. It is hoped that the knowledge gained from this research will help build a better understanding of color processing and color quality management methods in consumer video