Improve of contrast-distorted image quality assessment based on convolutional neural networks

Many image quality assessment algorithms (IQAs) have been developed during the past decade. However, most of them are designed for images distorted by compression, noise and blurring. There are very few IQAs designed specifically for Contrast Distorted Images (CDI), e.g. Reduced-reference Image Quality Metric for Contrast-changed images (RIQMC) and NR-IQA for Contrast-Distorted Images (NR-IQA-CDI). The existing NR-IQA-CDI relies on features designed by human or handcrafted features because considerable level of skill, domain expertise and efforts are required to design good handcrafted features. Recently, there is great advancement in machine learning with the introduction of deep learning through Convolutional Neural Networks (CNN) which enable machine to learn good features from raw image automatically without any human intervention. Therefore, it is tempting to explore the ways to transform the existing NR-IQA-CDI from using handcrafted features to machine-crafted features using deep learning, specifically Convolutional Neural Networks (CNN).The results show that NR-IQA-CDI based on non-pre-trained CNN (NR-IQA-CDI-NonPreCNN) significantly outperforms those which are based on handcrafted features. In addition to showing best performance, NR-IQA-CDI-NonPreCNN also enjoys the advantage of zero human intervention in designing feature, making it the most attractive solution for NR-IQA-CDI

TTL-IQA: transitive transfer learning based no-reference image quality assessment

Image quality assessment (IQA) based on deep learning faces the overfitting problem due to limited training samples available in existing IQA databases. Transfer learning is a plausible solution to the problem, in which the shared features derived from the large-scale Imagenet source domain could be transferred from the original recognition task to the intended IQA task. However, the Imagenet source domain and the IQA target domain as well as their corresponding tasks are not directly related. In this paper, we propose a new transitive transfer learning method for no-reference image quality assessment (TTL-IQA). First, the architecture of the multi-domain transitive transfer learning for IQA is developed to transfer the Imagenet source domain to the auxiliary domain, and then to the IQA target domain. Second, the auxiliary domain and the auxiliary task are constructed by a new generative adversarial network based on distortion translation (DT-GAN). Furthermore, a TTL network of the semantic features transfer (SFTnet) is proposed to optimize the shared features for the TTL-IQA. Experiments are conducted to evaluate the performance of the proposed method on various IQA databases, including the LIVE, TID2013, CSIQ, LIVE multiply distorted and LIVE challenge. The results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, our proposed method demonstrates a strong generalization ability

Multipurpose Image Quality Assessment for Both Human and Computer Vision Systems via Convolutional Neural Network

Computer vision algorithms have been widely used for many applications, including traffic monitoring, autonomous driving, robot path planning and navigation, object detection and medical image analysis, etc. Images and videos are typical input to computer vision algorithms and the performance of computer vision algorithms are highly correlated with the quality of input signal. The quality of videos and images are impacted by vision sensors; environmental conditions, such as lighting, rain, fog and wind. Therefore, it is a very active research issue to determine the failure mode of computer vision by automatically measuring the quality of images and videos. In the literature, many algorithms have been proposed to measure image and video qualities using reference images. However, measuring the quality of image and video without using a reference image, known as no-reference image quality assessment, is a very challenging problem. Most existing methods use a manual feature extraction and a classification technique to model image and video quality. Internal image statics are considered as feature vectors and classical machine learning techniques such as support vector machine and naive Bayes as the classifier. Using convolutional neural network (CNN) to learn the internal statistic of distorted images is a newly developed but efficient way to solve the problem. However, there are also new challenges in image quality assessment field. One of them is the wide spread of computer vision systems. Those systems, like human viewers, also demand a certain method to measure the quality of input images, but with their own standards. Inspired by the challenge, in this thesis, we propose to build an image quality assessment system based on convolutional neural network that can work for both human and computer vision system. In specific, we build 2 models: DAQ1 and DAQ2 with different design concept and evaluate their performance. Both models can work well with human visual system and outperform most former state-of-art Image Quality Assessment (IQA) methods. On computer vision system side, the models also show certain level of prediction power and reveal the potential of CNNs in facing this challenge. The performance in estimating image quality is first evaluated using 2 standard data-sets and against three state-of-the art image quality methods. Further, the performance in automatically detecting the failure mode computer vision algorithm is evaluated using Miovision's computer vision algorithm and datasets

A survey of DNN methods for blind image quality assessment

Blind image quality assessment (BIQA) methods aim to predict quality of images as perceived by humans without access to a reference image. Recently, deep learning methods have gained substantial attention in the research community and have proven useful for BIQA. Although previous study of deep neural networks (DNN) methods is presented, some novelty DNN methods, which are recently proposed, are not summarized for BIQA. In this paper, we provide a survey covering various DNN methods for BIQA. First, we systematically analyze the existing DNN-based quality assessment methods according to the role of DNN. Then, we compare the prediction performance of various DNN methods on the synthetic databases (LIVE, TID2013, CSIQ, LIVE multiply distorted) and authentic databases (LIVE challenge), providing important information that can help understand the underlying properties between different DNN methods for BIQA. Finally, we describe some emerging challenges in designing and training DNN-based BIQA, along with few directions that are worth further investigations in the future

Computational Methods for Matrix/Tensor Factorization and Deep Learning Image Denoising

Feature learning is a technique to automatically extract features from raw data. It is widely used in areas such as computer vision, image processing, data mining and natural language processing. In this thesis, we are interested in the computational aspects of feature learning. We focus on rank matrix and tensor factorization and deep neural network models for image denoising. With respect to matrix and tensor factorization, we first present a technique to speed up alternating least squares (ALS) and gradient descent (GD) − two commonly used strategies for tensor factorization. We introduce an efficient, scalable and distributed algorithm that addresses the data explosion problem. Instead of a computationally challenging sub-step of ALS and GD, we implement the algorithm on parallel machines by using only two sparse matrix-vector products. Not only is the algorithm scalable but it is also on average 4 to 10 times faster than competing algorithms on various data sets. Next, we discuss our results of non-negative matrix factorization for hyperspectral image data in the presence of noise. We introduce a spectral total variation regularization and derive four variants of the alternating direction method of multiplier algorithm. While all four methods belong to the same family of algorithms, some perform better than others. Thus, we compare the algorithms using stimulated Raman spectroscopic image will be demonstrated. For deep neural network models, we focus on its application to image denoising. We first demonstrate how an optimal procedure leveraging deep neural networks and convex optimization can combine a given set of denoisers to produce an overall better result. The proposed framework estimates the mean squared error (MSE) of individual denoised outputs using a deep neural network; optimally combines the denoised outputs via convex optimization; and recovers lost details of the combined images using another deep neural network. The framework consistently improves denoising performance for both deterministic denoisers and neural network denoisers. Next, we apply the deep neural network to solve the image reconstruction issues of the Quanta Image Sensor (QIS), which is a single-photon image sensor that oversamples the light field to generate binary measures

NEW LEARNING FRAMEWORKS FOR BLIND IMAGE QUALITY ASSESSMENT MODEL

The focus of this thesis is on image quality assessment, specifically for problems of assessing the quality of an image blindly or without reference information. There are significant efforts over the last decade in developing objective blind models that can assess image quality as perceived by humans. Various models have been introduced, achieving highly competitive performances and high in correlation with subjective perceptual measures. However, there are still limitations on these models before they can be viable replacements to traditional image metrics over a wide range of image processing applications. This thesis addresses several limitations. The thesis first proposes a new framework to learn a blind image quality model with minimal training requirements, operates locally and has ability to identify distortion in the assessed image. To increase the model’s performance, the thesis then modifies the framework by considering an aspect of human vision tendency, which is often ignored by previous models. Finally, the thesis presents another framework that enable a model to simultaneously learn quality prediction for images affected by different distortion types