Degraded Reference Image Quality Assessment

Images/videos are playing a more and more important role in the 21st century. The perceived quality of visual content often degrades during the process of acquisition, storage, transmission, display and rendering. Since subjective evaluation of such a large amount of visual content is impossible, the development of objective evaluation methods becomes highly desirable. Traditionally, there are three well established Image Quality Assessment (IQA) paradigms. They are Full Reference (FR) IQA which needs full access to the pristine quality reference, Reduced Reference (RR) IQA which requires partial information from the pristine reference and, No Reference (NR) IQA which does not require any reference information. While the strict requirement prohibits FR IQA from wide usage in many applications, RR and NR IQA methods cannot produce comparable performance. In the thesis, we aim to address this problem by exploring the Degraded Reference (DR) paradigm which makes no requirement on pristine reference but on reference of degraded quality, and at the same time, outperforms RR/NR methods. We address this problem in three steps. Firstly, we develop an FR model built upon a Deep Neural Network (DNN) that can handle multiply distorted images. The model structure of this FR model is then utilized to design DNN-based DR IQA models. We further improve the DR DNN model by adjusting the network structure. Finally, we use a two-step framework, which utilizes an NR model and an FR model as base modules followed by a regressor to create a single DR prediction for a given image. We test our models on subject-related datasets in IQA field. The testing results show that our FR model has state-of-the-art performance when handling multiply distorted images, and meanwhile produces great performance when handling singly distorted images. Our DR model developed using the two-step framework gives better performance than RR/NR models when the reference is not pristine

Image Quality Assessment: Addressing the Data Shortage and Multi-Stage Distortion Challenges

Visual content constitutes the vast majority of the ever increasing global Internet traffic, thus highlighting the central role that it plays in our daily lives. The perceived quality of such content can be degraded due to a number of distortions that it may undergo during the processes of acquisition, storage, transmission under bandwidth constraints, and display. Since the subjective evaluation of such large volumes of visual content is impossible, the development of perceptually well-aligned and practically applicable objective image quality assessment (IQA) methods has taken on crucial importance to ensure the delivery of an adequate quality of experience to the end user. Substantial strides have been made in the last two decades in designing perceptual quality methods and three major paradigms are now well-established in IQA research, these being Full-Reference (FR), Reduced-Reference (RR), and No-Reference (NR), which require complete, partial, and no access to the pristine reference content, respectively. Notwithstanding the progress made so far, significant challenges are restricting the development of practically applicable IQA methods. In this dissertation we aim to address two major challenges: 1) The data shortage challenge, and 2) The multi-stage distortion challenge. NR or blind IQA (BIQA) methods usually rely on machine learning methods, such as deep neural networks (DNNs), to learn a quality model by training on subject-rated IQA databases. Due to constraints of subjective-testing, such annotated datasets are quite small-scale, containing at best a few thousands of images. This is in sharp contrast to the area of visual recognition where tens of millions of annotated images are available. Such a data challenge has become a major hurdle on the breakthrough of DNN-based IQA approaches. We address the data challenge by developing the largest IQA dataset, called the Waterloo Exploration-II database, which consists of 3,570 pristine and around 3.45 million distorted images which are generated by using content adaptive distortion parameters and consist of both singly and multiply distorted content. As a prerequisite requirement of developing an alternative annotation mechanism, we conduct the largest performance evaluation survey in the IQA area to-date to ascertain the top performing FR and fused FR methods. Based on the findings of this survey, we develop a technique called Synthetic Quality Benchmark (SQB), to automatically assign highly perceptual quality labels to large-scale IQA datasets. We train a DNN-based BIQA model, called EONSS, on the SQB-annotated Waterloo Exploration-II database. Extensive tests on a large collection of completely independent and subject-rated IQA datasets show that EONSS outperforms the very state-of-the-art in BIQA, both in terms of perceptual quality prediction performance and computation time, thereby demonstrating the efficacy of our approach to address the data challenge. In practical media distribution systems, visual content undergoes a number of degradations as it is transmitted along the delivery chain, making it multiply distorted. Yet, research in IQA has mainly focused on the simplistic case of singly distorted content. In many practical systems, apart from the final multiply distorted content, access to earlier degraded versions of such content is available. However, the three major IQA paradigms (FR, RR, and, NR) are unable to take advantage of this additional information. To address this challenge, we make one of the first attempts to study the behavior of multiple simultaneous distortion combinations in a two-stage distortion pipeline. Next, we introduce a new major IQA paradigm, called degraded reference (DR) IQA, to evaluate the quality of multiply distorted images by also taking into consideration their respective degraded references. We construct two datasets for the purpose of DR IQA model development, and call them DR IQA database V1 and V2. These datasets are designed on the pattern of the Waterloo Exploration-II database and have 32,912 SQB-annotated distorted images, composed of both singly distorted degraded references and multiply distorted content. We develop distortion behavior based and SVR-based DR IQA models. Extensive testing on an independent set of IQA datasets, including three subject-rated datasets, demonstrates that by utilizing the additional information available in the form of degraded references, the DR IQA models perform significantly better than their BIQA counterparts, thereby establishing DR IQA as a new paradigm in IQA

Terahertz Security Image Quality Assessment by No-reference Model Observers

To provide the possibility of developing objective image quality assessment (IQA) algorithms for THz security images, we constructed the THz security image database (THSID) including a total of 181 THz security images with the resolution of 127*380. The main distortion types in THz security images were first analyzed for the design of subjective evaluation criteria to acquire the mean opinion scores. Subsequently, the existing no-reference IQA algorithms, which were 5 opinion-aware approaches viz., NFERM, GMLF, DIIVINE, BRISQUE and BLIINDS2, and 8 opinion-unaware approaches viz., QAC, SISBLIM, NIQE, FISBLIM, CPBD, S3 and Fish_bb, were executed for the evaluation of the THz security image quality. The statistical results demonstrated the superiority of Fish_bb over the other testing IQA approaches for assessing the THz image quality with PLCC (SROCC) values of 0.8925 (-0.8706), and with RMSE value of 0.3993. The linear regression analysis and Bland-Altman plot further verified that the Fish__bb could substitute for the subjective IQA. Nonetheless, for the classification of THz security images, we tended to use S3 as a criterion for ranking THz security image grades because of the relatively low false positive rate in classifying bad THz image quality into acceptable category (24.69%). Interestingly, due to the specific property of THz image, the average pixel intensity gave the best performance than the above complicated IQA algorithms, with the PLCC, SROCC and RMSE of 0.9001, -0.8800 and 0.3857, respectively. This study will help the users such as researchers or security staffs to obtain the THz security images of good quality. Currently, our research group is attempting to make this research more comprehensive.Comment: 13 pages, 8 figures, 4 table

BIQ2021: A Large-Scale Blind Image Quality Assessment Database

The assessment of the perceptual quality of digital images is becoming increasingly important as a result of the widespread use of digital multimedia devices. Smartphones and high-speed internet are just two examples of technologies that have multiplied the amount of multimedia content available. Thus, obtaining a representative dataset, which is required for objective quality assessment training, is a significant challenge. The Blind Image Quality Assessment Database, BIQ2021, is presented in this article. By selecting images with naturally occurring distortions and reliable labeling, the dataset addresses the challenge of obtaining representative images for no-reference image quality assessment. The dataset consists of three sets of images: those taken without the intention of using them for image quality assessment, those taken with intentionally introduced natural distortions, and those taken from an open-source image-sharing platform. It is attempted to maintain a diverse collection of images from various devices, containing a variety of different types of objects and varying degrees of foreground and background information. To obtain reliable scores, these images are subjectively scored in a laboratory environment using a single stimulus method. The database contains information about subjective scoring, human subject statistics, and the standard deviation of each image. The dataset's Mean Opinion Scores (MOS) make it useful for assessing visual quality. Additionally, the proposed database is used to evaluate existing blind image quality assessment approaches, and the scores are analyzed using Pearson and Spearman's correlation coefficients. The image database and MOS are freely available for use and benchmarking