Deep Quality: A Deep No-reference Quality Assessment System

Image quality assessment (IQA) continues to garner great interestin the research community, particularly given the tremendousrise in consumer video capture and streaming. Despite significantresearch effort in IQA in the past few decades, the area of noreferenceimage quality assessment remains a great challenge andis largely unsolved. In this paper, we propose a novel no-referenceimage quality assessment system called Deep Quality, which leveragesthe power of deep learning to model the complex relationshipbetween visual content and the perceived quality. Deep Qualityconsists of a novel multi-scale deep convolutional neural network,trained to learn to assess image quality based on training samplesconsisting of different distortions and degradations such as blur,Gaussian noise, and compression artifacts. Preliminary results usingthe CSIQ benchmark image quality dataset showed that DeepQuality was able to achieve strong quality prediction performance(89% patch-level and 98% image-level prediction accuracy), beingable to achieve similar performance as full-reference IQA methods

Deep Quality: A Deep No-reference Quality Assessment System

Image quality assessment (IQA) continues to garner great interestin the research community, particularly given the tremendousrise in consumer video capture and streaming. Despite significantresearch effort in IQA in the past few decades, the area of noreferenceimage quality assessment remains a great challenge andis largely unsolved. In this paper, we propose a novel no-referenceimage quality assessment system called Deep Quality, which leveragesthe power of deep learning to model the complex relationshipbetween visual content and the perceived quality. Deep Qualityconsists of a novel multi-scale deep convolutional neural network,trained to learn to assess image quality based on training samplesconsisting of different distortions and degradations such as blur,Gaussian noise, and compression artifacts. Preliminary results usingthe CSIQ benchmark image quality dataset showed that DeepQuality was able to achieve strong quality prediction performance(89% patch-level and 98% image-level prediction accuracy), beingable to achieve similar performance as full-reference IQA methods

An {\alpha}-Matte Boundary Defocus Model Based Cascaded Network for Multi-focus Image Fusion

Capturing an all-in-focus image with a single camera is difficult since the depth of field of the camera is usually limited. An alternative method to obtain the all-in-focus image is to fuse several images focusing at different depths. However, existing multi-focus image fusion methods cannot obtain clear results for areas near the focused/defocused boundary (FDB). In this paper, a novel {\alpha}-matte boundary defocus model is proposed to generate realistic training data with the defocus spread effect precisely modeled, especially for areas near the FDB. Based on this {\alpha}-matte defocus model and the generated data, a cascaded boundary aware convolutional network termed MMF-Net is proposed and trained, aiming to achieve clearer fusion results around the FDB. More specifically, the MMF-Net consists of two cascaded sub-nets for initial fusion and boundary fusion, respectively; these two sub-nets are designed to first obtain a guidance map of FDB and then refine the fusion near the FDB. Experiments demonstrate that with the help of the new {\alpha}-matte boundary defocus model, the proposed MMF-Net outperforms the state-of-the-art methods both qualitatively and quantitatively.Comment: 10 pages, 8 figures, journal Unfortunately, I cannot spell one of the authors' name coorectl

Image Quality Assessment: Addressing the Data Shortage and Multi-Stage Distortion Challenges

Visual content constitutes the vast majority of the ever increasing global Internet traffic, thus highlighting the central role that it plays in our daily lives. The perceived quality of such content can be degraded due to a number of distortions that it may undergo during the processes of acquisition, storage, transmission under bandwidth constraints, and display. Since the subjective evaluation of such large volumes of visual content is impossible, the development of perceptually well-aligned and practically applicable objective image quality assessment (IQA) methods has taken on crucial importance to ensure the delivery of an adequate quality of experience to the end user. Substantial strides have been made in the last two decades in designing perceptual quality methods and three major paradigms are now well-established in IQA research, these being Full-Reference (FR), Reduced-Reference (RR), and No-Reference (NR), which require complete, partial, and no access to the pristine reference content, respectively. Notwithstanding the progress made so far, significant challenges are restricting the development of practically applicable IQA methods. In this dissertation we aim to address two major challenges: 1) The data shortage challenge, and 2) The multi-stage distortion challenge. NR or blind IQA (BIQA) methods usually rely on machine learning methods, such as deep neural networks (DNNs), to learn a quality model by training on subject-rated IQA databases. Due to constraints of subjective-testing, such annotated datasets are quite small-scale, containing at best a few thousands of images. This is in sharp contrast to the area of visual recognition where tens of millions of annotated images are available. Such a data challenge has become a major hurdle on the breakthrough of DNN-based IQA approaches. We address the data challenge by developing the largest IQA dataset, called the Waterloo Exploration-II database, which consists of 3,570 pristine and around 3.45 million distorted images which are generated by using content adaptive distortion parameters and consist of both singly and multiply distorted content. As a prerequisite requirement of developing an alternative annotation mechanism, we conduct the largest performance evaluation survey in the IQA area to-date to ascertain the top performing FR and fused FR methods. Based on the findings of this survey, we develop a technique called Synthetic Quality Benchmark (SQB), to automatically assign highly perceptual quality labels to large-scale IQA datasets. We train a DNN-based BIQA model, called EONSS, on the SQB-annotated Waterloo Exploration-II database. Extensive tests on a large collection of completely independent and subject-rated IQA datasets show that EONSS outperforms the very state-of-the-art in BIQA, both in terms of perceptual quality prediction performance and computation time, thereby demonstrating the efficacy of our approach to address the data challenge. In practical media distribution systems, visual content undergoes a number of degradations as it is transmitted along the delivery chain, making it multiply distorted. Yet, research in IQA has mainly focused on the simplistic case of singly distorted content. In many practical systems, apart from the final multiply distorted content, access to earlier degraded versions of such content is available. However, the three major IQA paradigms (FR, RR, and, NR) are unable to take advantage of this additional information. To address this challenge, we make one of the first attempts to study the behavior of multiple simultaneous distortion combinations in a two-stage distortion pipeline. Next, we introduce a new major IQA paradigm, called degraded reference (DR) IQA, to evaluate the quality of multiply distorted images by also taking into consideration their respective degraded references. We construct two datasets for the purpose of DR IQA model development, and call them DR IQA database V1 and V2. These datasets are designed on the pattern of the Waterloo Exploration-II database and have 32,912 SQB-annotated distorted images, composed of both singly distorted degraded references and multiply distorted content. We develop distortion behavior based and SVR-based DR IQA models. Extensive testing on an independent set of IQA datasets, including three subject-rated datasets, demonstrates that by utilizing the additional information available in the form of degraded references, the DR IQA models perform significantly better than their BIQA counterparts, thereby establishing DR IQA as a new paradigm in IQA

IEEE Transactions On Image Processing : Vol. 24, No. 9 - 10, September - October 2015

1. Feature-Based Lucas-Kanade and Active Appearance Models 2. An Approach Toward Fast Gradient-Based Image Segmentation 3. Inverse Sparse Tracker With a Locally Weighted Distance Metric 4. Video Deraining and Desnowing Using Temporal Correlation and Low-Rank Matrix Completion 5. Enhancement of Textural Differences Based on Morphological Component Analysis 6. Neutral Face Classification Using Personalized Appearance Models for Fast and Robust Emotion Detection 7.Stacked Multilayer Self-Organizing Map for Background Modeling 8. SLED : Semantic Label Embedding Dictionary Representation for Multilabel Image Annotation 9. Objective Quality Assessment for Multiexposure Multifocus Image Fusion 10. A No-Reference Texture Regularity Metric Based on Visual Saliency Etc