Example-based learning for single-image super-resolution and JPEG artifact removal

This paper proposes a framework for single-image super-resolution and JPEG artifact removal. The underlying idea is to learn a map from input low-quality images (suitably preprocessed low-resolution or JPEG encoded images) to target high-quality images based on example pairs of input and output images. To retain the complexity of the resulting learning problem at a moderate level, a patch-based approach is taken such that kernel ridge regression (KRR) scans the input image with a small window (patch) and produces a patchvalued output for each output pixel location. These constitute a set of candidate images each of which reflects different local information. An image output is then obtained as a convex combination of candidates for each pixel based on estimated confidences of candidates. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as it has been done in existing example-based super-resolution algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing super-resolution and JPEG artifact removal methods shows the effectiveness of the proposed method. Furthermore, the proposed method is generic in that it has the potential to be applied to many other image enhancement applications

Generative Prior for Unsupervised Image Restoration

The challenge of restoring real world low-quality images is due to a lack of appropriate training data and difficulty in determining how the image was degraded. Recently, generative models have demonstrated great potential for creating high- quality images by utilizing the rich and diverse information contained within the model’s trained weights and learned latent representations. One popular type of generative model is the generative adversarial network (GAN). Many new methods have been developed to harness the information found in GANs for image manipulation. Our proposed approach is to utilize generative models for both understanding the degradation of an image and restoring it. We propose using a combination of cycle consistency losses and self-attention to enhance face images by first learning the degradation and then using this information to train a style-based neural network. We also aim to use the latent representation to achieve a high level of magnification for face images (x64). By incorporating the weights of a pre-trained StyleGAN into a restoration network with a vision transformer layer, we hope to improve the current state-of-the-art in face image restoration. Finally, we present a projection-based image-denoising algorithm named Noise2Code in the latent space of the VQGAN model with a fixed-point regularization strategy. The fixed-point condition follows the observation that the pre-trained VQGAN affects the clean and noisy images in a drastically different way. Unlike previous projection-based image restoration in the latent space, both the denoising network and VQGAN model parameters are jointly trained, although the latter is not needed during the testing. We report experimental results to demonstrate that the proposed Noise2Code approach is conceptually simple, computationally efficient, and generalizable to real-world degradation scenarios

Image Restoration

This book represents a sample of recent contributions of researchers all around the world in the field of image restoration. The book consists of 15 chapters organized in three main sections (Theory, Applications, Interdisciplinarity). Topics cover some different aspects of the theory of image restoration, but this book is also an occasion to highlight some new topics of research related to the emergence of some original imaging devices. From this arise some real challenging problems related to image reconstruction/restoration that open the way to some new fundamental scientific questions closely related with the world we interact with

Poly-GAN: A Multi-Conditioned GAN for Multiple Tasks

We present Poly-GAN, a novel conditional GAN architecture that is motivated by different Image generation and manipulation applications like Fashion Synthesis, an application where garments are automatically placed on images of human models at an arbitrary pose, image inpainting, an application where we try to recover a damaged image using the edges or a rough sketch of the image. While different applications use different GAN setup for image generation, we propose only one architecture for multiple applications with little to no change in the pipeline. Poly-GAN allows conditioning on multiple inputs and is suitable for many different tasks. Our novel architecture enforces the conditions at all layers of the encoder and utilizes skip connections from the coarse layers of the encoder to the respective layers of the decoder. Coarse layers are easier to manipulate in shape change using condition, which results in higher level change in the result. Our system achieves state-of-the-art quantitative results on Fashion Synthesis based on the Structural Similarity Index metric and Inception Score metric using the DeepFashion dataset. For the image inpainting task we are achieving competitive results compared to current state of the art methods

Single frame super-resolution image system

The estimation of some unknown quantity information from known observable information can be viewed as a specific statistical process which needs an extra source of information prediction strategy. In this regard, image super-resolution is an important application In this thesis, we proposed a new image interpolation method based on Redundant Discrete Wavelet Transform (RDWT) and self-adaptive processes in which edge direction details are considered to solve single-frame image super-resolution task. Information about sharp variations, both in horizontal and vertical directions derived from wavelet transform sub-bands are considered, followed by detection and modification of the aliasing part in the preliminary output in order to increase the visual effect. By exploiting fundamental properties of images such as property of edge direction, different parts of the source image are considered separately in order to predict the vertical and horizontal details accurately, helping to consummate the whole framework in reconstructing the high-resolution image. Extensive tests of the proposed method show that both objective quality (PSNR) and subjective quality are obviously improved compared to several other state-of-the-art methods. And this work also leaved capacious space for further research, not only theoretical but also practical. Some of the related research applications based on this algorithm strategy are also briefly introduced

Development Of A High Performance Mosaicing And Super-Resolution Algorithm

In this dissertation, a high-performance mosaicing and super-resolution algorithm is described. The scale invariant feature transform (SIFT)-based mosaicing algorithm builds an initial mosaic which is iteratively updated by the robust super resolution algorithm to achieve the final high-resolution mosaic. Two different types of datasets are used for testing: high altitude balloon data and unmanned aerial vehicle data. To evaluate our algorithm, five performance metrics are employed: mean square error, peak signal to noise ratio, singular value decomposition, slope of reciprocal singular value curve, and cumulative probability of blur detection. Extensive testing shows that the proposed algorithm is effective in improving the captured aerial data and the performance metrics are accurate in quantifying the evaluation of the algorithm