Deep Learning frameworks for Image Quality Assessment

Technology is advancing by the arrival of deep learning and it finds huge application in image processing also. Deep learning itself sufficient to perform over all the statistical methods. As a research work, I implemented image quality assessment techniques using deep learning. Here I proposed two full reference image quality assessment algorithms and two no reference image quality algorithms. Among the two algorithms on each method, one is in a supervised manner and other is in an unsupervised manner. First proposed method is the full reference image quality assessment using autoencoder. Existing literature shows that statistical features of pristine images will get distorted in presence of distortion. It will be more advantageous if algorithm itself learns the distortion discriminating features. It will be more complex if the feature length is more. So autoencoder is trained using a large number of pristine images. An autoencoder will give the best lower dimensional representation of the input. It is showed that encoded distance features have good distortion discrimination properties. The proposed algorithm delivers competitive performance over standard databases. If we are giving both reference and distorted images to the model and the model learning itself and gives the scores will reduce the load of extracting features and doing post-processing. But model should be capable one for discriminating the features by itself. Second method which I proposed is a full reference and no reference image quality assessment using deep convolutional neural networks. A network is trained in a supervised manner with subjective scores as targets. The algorithm is performing e�ciently for the distortions that are learned while training the model. Last proposed method is a classiffication based no reference image quality assessment. Distortion level in an image may vary from one region to another region. We may not be able to view distortion in some part but it may be present in other parts. A classiffication model is able to tell whether a given input patch is of low quality or high quality. It is shown that aggregate of the patch quality scores is having a high correlation with the subjective scores

Learned Perceptual Image Enhancement

Learning a typical image enhancement pipeline involves minimization of a loss function between enhanced and reference images. While L1 and L2 losses are perhaps the most widely used functions for this purpose, they do not necessarily lead to perceptually compelling results. In this paper, we show that adding a learned no-reference image quality metric to the loss can significantly improve enhancement operators. This metric is implemented using a CNN (convolutional neural network) trained on a large-scale dataset labelled with aesthetic preferences of human raters. This loss allows us to conveniently perform back-propagation in our learning framework to simultaneously optimize for similarity to a given ground truth reference and perceptual quality. This perceptual loss is only used to train parameters of image processing operators, and does not impose any extra complexity at inference time. Our experiments demonstrate that this loss can be effective for tuning a variety of operators such as local tone mapping and dehazing

Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank

For many applications the collection of labeled data is expensive laborious. Exploitation of unlabeled data during training is thus a long pursued objective of machine learning. Self-supervised learning addresses this by positing an auxiliary task (different, but related to the supervised task) for which data is abundantly available. In this paper, we show how ranking can be used as a proxy task for some regression problems. As another contribution, we propose an efficient backpropagation technique for Siamese networks which prevents the redundant computation introduced by the multi-branch network architecture. We apply our framework to two regression problems: Image Quality Assessment (IQA) and Crowd Counting. For both we show how to automatically generate ranked image sets from unlabeled data. Our results show that networks trained to regress to the ground truth targets for labeled data and to simultaneously learn to rank unlabeled data obtain significantly better, state-of-the-art results for both IQA and crowd counting. In addition, we show that measuring network uncertainty on the self-supervised proxy task is a good measure of informativeness of unlabeled data. This can be used to drive an algorithm for active learning and we show that this reduces labeling effort by up to 50%.Comment: Accepted at TPAMI. (Keywords: Learning from rankings, image quality assessment, crowd counting, active learning). arXiv admin note: text overlap with arXiv:1803.0309

CAS-CNN: A Deep Convolutional Neural Network for Image Compression Artifact Suppression

Lossy image compression algorithms are pervasively used to reduce the size of images transmitted over the web and recorded on data storage media. However, we pay for their high compression rate with visual artifacts degrading the user experience. Deep convolutional neural networks have become a widespread tool to address high-level computer vision tasks very successfully. Recently, they have found their way into the areas of low-level computer vision and image processing to solve regression problems mostly with relatively shallow networks. We present a novel 12-layer deep convolutional network for image compression artifact suppression with hierarchical skip connections and a multi-scale loss function. We achieve a boost of up to 1.79 dB in PSNR over ordinary JPEG and an improvement of up to 0.36 dB over the best previous ConvNet result. We show that a network trained for a specific quality factor (QF) is resilient to the QF used to compress the input image - a single network trained for QF 60 provides a PSNR gain of more than 1.5 dB over the wide QF range from 40 to 76.Comment: 8 page