Towards a Semantic Perceptual Image Metric

We present a full reference, perceptual image metric based on VGG-16, an artificial neural network trained on object classification. We fit the metric to a new database based on 140k unique images annotated with ground truth by human raters who received minimal instruction. The resulting metric shows competitive performance on TID 2013, a database widely used to assess image quality assessments methods. More interestingly, it shows strong responses to objects potentially carrying semantic relevance such as faces and text, which we demonstrate using a visualization technique and ablation experiments. In effect, the metric appears to model a higher influence of semantic context on judgments, which we observe particularly in untrained raters. As the vast majority of users of image processing systems are unfamiliar with Image Quality Assessment (IQA) tasks, these findings may have significant impact on real-world applications of perceptual metrics

Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment

We present a deep neural network-based approach to image quality assessment (IQA). The network is trained end-to-end and comprises ten convolutional layers and five pooling layers for feature extraction, and two fully connected layers for regression, which makes it significantly deeper than related IQA models. Unique features of the proposed architecture are that: 1) with slight adaptations it can be used in a no-reference (NR) as well as in a full-reference (FR) IQA setting and 2) it allows for joint learning of local quality and local weights, i.e., relative importance of local quality to the global quality estimate, in an unified framework. Our approach is purely data-driven and does not rely on hand-crafted features or other types of prior domain knowledge about the human visual system or image statistics. We evaluate the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the LIVE In the wild image quality challenge database and show superior performance to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation shows a high ability to generalize between different databases, indicating a high robustness of the learned features

Deep Quality: A Deep No-reference Quality Assessment System

Image quality assessment (IQA) continues to garner great interestin the research community, particularly given the tremendousrise in consumer video capture and streaming. Despite significantresearch effort in IQA in the past few decades, the area of noreferenceimage quality assessment remains a great challenge andis largely unsolved. In this paper, we propose a novel no-referenceimage quality assessment system called Deep Quality, which leveragesthe power of deep learning to model the complex relationshipbetween visual content and the perceived quality. Deep Qualityconsists of a novel multi-scale deep convolutional neural network,trained to learn to assess image quality based on training samplesconsisting of different distortions and degradations such as blur,Gaussian noise, and compression artifacts. Preliminary results usingthe CSIQ benchmark image quality dataset showed that DeepQuality was able to achieve strong quality prediction performance(89% patch-level and 98% image-level prediction accuracy), beingable to achieve similar performance as full-reference IQA methods

Deep Quality: A Deep No-reference Quality Assessment System

Image quality assessment (IQA) continues to garner great interestin the research community, particularly given the tremendousrise in consumer video capture and streaming. Despite significantresearch effort in IQA in the past few decades, the area of noreferenceimage quality assessment remains a great challenge andis largely unsolved. In this paper, we propose a novel no-referenceimage quality assessment system called Deep Quality, which leveragesthe power of deep learning to model the complex relationshipbetween visual content and the perceived quality. Deep Qualityconsists of a novel multi-scale deep convolutional neural network,trained to learn to assess image quality based on training samplesconsisting of different distortions and degradations such as blur,Gaussian noise, and compression artifacts. Preliminary results usingthe CSIQ benchmark image quality dataset showed that DeepQuality was able to achieve strong quality prediction performance(89% patch-level and 98% image-level prediction accuracy), beingable to achieve similar performance as full-reference IQA methods

Deep Learning frameworks for Image Quality Assessment

Technology is advancing by the arrival of deep learning and it finds huge application in image processing also. Deep learning itself sufficient to perform over all the statistical methods. As a research work, I implemented image quality assessment techniques using deep learning. Here I proposed two full reference image quality assessment algorithms and two no reference image quality algorithms. Among the two algorithms on each method, one is in a supervised manner and other is in an unsupervised manner. First proposed method is the full reference image quality assessment using autoencoder. Existing literature shows that statistical features of pristine images will get distorted in presence of distortion. It will be more advantageous if algorithm itself learns the distortion discriminating features. It will be more complex if the feature length is more. So autoencoder is trained using a large number of pristine images. An autoencoder will give the best lower dimensional representation of the input. It is showed that encoded distance features have good distortion discrimination properties. The proposed algorithm delivers competitive performance over standard databases. If we are giving both reference and distorted images to the model and the model learning itself and gives the scores will reduce the load of extracting features and doing post-processing. But model should be capable one for discriminating the features by itself. Second method which I proposed is a full reference and no reference image quality assessment using deep convolutional neural networks. A network is trained in a supervised manner with subjective scores as targets. The algorithm is performing e�ciently for the distortions that are learned while training the model. Last proposed method is a classiffication based no reference image quality assessment. Distortion level in an image may vary from one region to another region. We may not be able to view distortion in some part but it may be present in other parts. A classiffication model is able to tell whether a given input patch is of low quality or high quality. It is shown that aggregate of the patch quality scores is having a high correlation with the subjective scores