Geometry-Aware Neighborhood Search for Learning Local Models for Image Reconstruction

Local learning of sparse image models has proven to be very effective to solve inverse problems in many computer vision applications. To learn such models, the data samples are often clustered using the K-means algorithm with the Euclidean distance as a dissimilarity metric. However, the Euclidean distance may not always be a good dissimilarity measure for comparing data samples lying on a manifold. In this paper, we propose two algorithms for determining a local subset of training samples from which a good local model can be computed for reconstructing a given input test sample, where we take into account the underlying geometry of the data. The first algorithm, called Adaptive Geometry-driven Nearest Neighbor search (AGNN), is an adaptive scheme which can be seen as an out-of-sample extension of the replicator graph clustering method for local model learning. The second method, called Geometry-driven Overlapping Clusters (GOC), is a less complex nonadaptive alternative for training subset selection. The proposed AGNN and GOC methods are evaluated in image super-resolution, deblurring and denoising applications and shown to outperform spectral clustering, soft clustering, and geodesic distance based subset selection in most settings.Comment: 15 pages, 10 figures and 5 table

Regional Differential Information Entropy for Super-Resolution Image Quality Assessment

PSNR and SSIM are the most widely used metrics in super-resolution problems, because they are easy to use and can evaluate the similarities between generated images and reference images. However, single image super-resolution is an ill-posed problem, there are multiple corresponding high-resolution images for the same low-resolution image. The similarities can't totally reflect the restoration effect. The perceptual quality of generated images is also important, but PSNR and SSIM do not reflect perceptual quality well. To solve the problem, we proposed a method called regional differential information entropy to measure both of the similarities and perceptual quality. To overcome the problem that traditional image information entropy can't reflect the structure information, we proposed to measure every region's information entropy with sliding window. Considering that the human visual system is more sensitive to the brightness difference at low brightness, we take $\gamma$ quantization rather than linear quantization. To accelerate the method, we reorganized the calculation procedure of information entropy with a neural network. Through experiments on our IQA dataset and PIPAL, this paper proves that RDIE can better quantify perceptual quality of images especially GAN-based images.Comment: 8 pages, 9 figures, 4 table

The Perception-Distortion Tradeoff

Image restoration algorithms are typically evaluated by some distortion measure (e.g. PSNR, SSIM, IFC, VIF) or by human opinion scores that quantify perceived perceptual quality. In this paper, we prove mathematically that distortion and perceptual quality are at odds with each other. Specifically, we study the optimal probability for correctly discriminating the outputs of an image restoration algorithm from real images. We show that as the mean distortion decreases, this probability must increase (indicating worse perceptual quality). As opposed to the common belief, this result holds true for any distortion measure, and is not only a problem of the PSNR or SSIM criteria. We also show that generative-adversarial-nets (GANs) provide a principled way to approach the perception-distortion bound. This constitutes theoretical support to their observed success in low-level vision tasks. Based on our analysis, we propose a new methodology for evaluating image restoration methods, and use it to perform an extensive comparison between recent super-resolution algorithms.Comment: CVPR 2018 (long oral presentation), see talk at: https://youtu.be/_aXbGqdEkjk?t=39m43

A Survey of Super-Resolution in Iris Biometrics With Evaluation of Dictionary-Learning

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksThe lack of resolution has a negative impact on the performance of image-based biometrics. While many generic super-resolution methods have been proposed to restore low-resolution images, they usually aim to enhance their visual appearance. However, an overall visual enhancement of biometric images does not necessarily correlate with a better recognition performance. Reconstruction approaches thus need to incorporate the specific information from the target biometric modality to effectively improve recognition performance. This paper presents a comprehensive survey of iris super-resolution approaches proposed in the literature. We have also adapted an eigen-patches’ reconstruction method based on the principal component analysis eigen-transformation of local image patches. The structure of the iris is exploited by building a patch-position-dependent dictionary. In addition, image patches are restored separately, having their own reconstruction weights. This allows the solution to be locally optimized, helping to preserve local information. To evaluate the algorithm, we degraded the high-resolution images from the CASIA Interval V3 database. Different restorations were considered, with 15 × 15 pixels being the smallest resolution evaluated. To the best of our knowledge, this is the smallest resolutions employed in the literature. The experimental framework is complemented with six publicly available iris comparators that were used to carry out biometric verification and identification experiments. The experimental results show that the proposed method significantly outperforms both the bilinear and bicubic interpolations at a very low resolution. The performance of a number of comparators attains an impressive equal error rate as low as 5% and a Top-1 accuracy of 77%–84% when considering the iris images of only 15 × 15 pixels. These results clearly demonstrate the benefit of using trained super-resolution techniques to improve the quality of iris images prior to matchingThis work was supported by the EU COST Action under Grant IC1106. The work of F. Alonso-Fernandez and J. Bigun was supported in part by the Swedish Research Council, in part by the Swedish Innovation Agency, and in part by the Swedish Knowledge Foundation through the CAISR/SIDUS-AIR projects. The work of J. Fierrez was supported by the Spanish MINECO/FEDER through the CogniMetrics Project under Grant TEC2015-70627-R. The authors acknowledge the Halmstad University Library for its support with the open access fee

Deep Learning for Robust Super-resolution

Super Resolution (SR) is a process in which a high-resolution counterpart of an image is reconstructed from its low-resolution sample. Generative Adversarial Networks (GAN), known for their ability of hyper-realistic image generation, demonstrate promising results in performing SR task. High-scale SR, where the super-resolved image is notably larger than low-resolution input, is a challenging but very beneficial task. By employing an SR model, the data can be compressed, more details can be extracted from cheap sensors and cameras, and the noise level will be reduced dramatically. As a result, the high-scale SR model can contribute significantly to face-related tasks, such as identification, face detection, and surveillance systems. Moreover, the resolution of medical scans will be notably increased. So more details can be detected and the early-stage diagnosis will be possible for many diseases such as cancer. Moreover, cheaper and more available scanning devices can be used for accurate abnormality detection. As a result, more lives can be saved because of the enhancement of the accuracy and the availability of scans. In this thesis, the first multi-scale gradient capsule GAN for SR is proposed. First, this model is trained on CelebA dataset for face SR. The performance of the proposed model is compared with state-of-the-art works and its supremacy in all similarity metrics is demonstrated. A new perceptual similarity index is introduced as well and the proposed architecture outperforms related works in this metric with a notable margin. A robustness test is conducted and the drop in similarity metrics is investigated. As a result, the proposed SR model is not only more accurate but also more robust than the state-of-the-art works. Since the proposed model is considered as a general SR system, it is also employed for prostate MRI SR. Prostate cancer is a very common disease among adult men. One in seven Canadian men is diagnosed with this cancer in their lifetime. SR can facilitate early diagnosis and potentially save many lives. The proposed model is trained on the Prostate-Diagnosis and PROSTATEx datasets. The proposed model outperformed SRGAN, the state-of-the-art prostate SR model. A new task-specific similarity assessment is introduced as well. A classifier is trained for severe cancer detection and the drop in the accuracy of this model when dealing with super-resolved images is used for evaluating the ability of medical detail reconstruction of the SR models. This proposed SR model is a step towards an efficient and accurate general SR platform