186 research outputs found
Neural Degradation Representation Learning for All-In-One Image Restoration
Existing methods have demonstrated effective performance on a single
degradation type. In practical applications, however, the degradation is often
unknown, and the mismatch between the model and the degradation will result in
a severe performance drop. In this paper, we propose an all-in-one image
restoration network that tackles multiple degradations. Due to the
heterogeneous nature of different types of degradations, it is difficult to
process multiple degradations in a single network. To this end, we propose to
learn a neural degradation representation (NDR) that captures the underlying
characteristics of various degradations. The learned NDR decomposes different
types of degradations adaptively, similar to a neural dictionary that
represents basic degradation components. Subsequently, we develop a degradation
query module and a degradation injection module to effectively recognize and
utilize the specific degradation based on NDR, enabling the all-in-one
restoration ability for multiple degradations. Moreover, we propose a
bidirectional optimization strategy to effectively drive NDR to learn the
degradation representation by optimizing the degradation and restoration
processes alternately. Comprehensive experiments on representative types of
degradations (including noise, haze, rain, and downsampling) demonstrate the
effectiveness and generalization capability of our method
Learning Representations for Controllable Image Restoration
Deep Convolutional Neural Networks have sparked a renaissance in all the sub-fields of computer vision. Tremendous progress has been made in the area of image restoration. The research community has pushed the boundaries of image deblurring, super-resolution, and denoising. However, given a distorted image, most existing methods typically produce a single restored output. The tasks mentioned above are inherently ill-posed, leading to an infinite number of plausible solutions. This thesis focuses on designing image restoration techniques capable of producing multiple restored results and granting users more control over the restoration process. Towards this goal, we demonstrate how one could leverage the power of unsupervised representation learning.
Image restoration is vital when applied to distorted images of human faces due to their social significance. Generative Adversarial Networks enable an unprecedented level of generated facial details combined with smooth latent space. We leverage the power of GANs towards the goal of learning controllable neural face representations. We demonstrate how to learn an inverse mapping from image space to these latent representations, tuning these representations towards a specific task, and finally manipulating latent codes in these spaces. For example, we show how GANs and their inverse mappings enable the restoration and editing of faces in the context of extreme face super-resolution and the generation of novel view sharp videos from a single motion-blurred image of a face.
This thesis also addresses more general blind super-resolution, denoising, and scratch removal problems, where blur kernels and noise levels are unknown. We resort to contrastive representation learning and first learn the latent space of degradations. We demonstrate that the learned representation allows inference of ground-truth degradation parameters and can guide the restoration process. Moreover, it enables control over the amount of deblurring and denoising in the restoration via manipulation of latent degradation features
Deep Learning for Single Image Super-Resolution: A Brief Review
Single image super-resolution (SISR) is a notoriously challenging ill-posed
problem, which aims to obtain a high-resolution (HR) output from one of its
low-resolution (LR) versions. To solve the SISR problem, recently powerful deep
learning algorithms have been employed and achieved the state-of-the-art
performance. In this survey, we review representative deep learning-based SISR
methods, and group them into two categories according to their major
contributions to two essential aspects of SISR: the exploration of efficient
neural network architectures for SISR, and the development of effective
optimization objectives for deep SISR learning. For each category, a baseline
is firstly established and several critical limitations of the baseline are
summarized. Then representative works on overcoming these limitations are
presented based on their original contents as well as our critical
understandings and analyses, and relevant comparisons are conducted from a
variety of perspectives. Finally we conclude this review with some vital
current challenges and future trends in SISR leveraging deep learning
algorithms.Comment: Accepted by IEEE Transactions on Multimedia (TMM
DifFace: Blind Face Restoration with Diffused Error Contraction
While deep learning-based methods for blind face restoration have achieved
unprecedented success, they still suffer from two major limitations. First,
most of them deteriorate when facing complex degradations out of their training
data. Second, these methods require multiple constraints, e.g., fidelity,
perceptual, and adversarial losses, which require laborious hyper-parameter
tuning to stabilize and balance their influences. In this work, we propose a
novel method named DifFace that is capable of coping with unseen and complex
degradations more gracefully without complicated loss designs. The key of our
method is to establish a posterior distribution from the observed low-quality
(LQ) image to its high-quality (HQ) counterpart. In particular, we design a
transition distribution from the LQ image to the intermediate state of a
pre-trained diffusion model and then gradually transmit from this intermediate
state to the HQ target by recursively applying a pre-trained diffusion model.
The transition distribution only relies on a restoration backbone that is
trained with loss on some synthetic data, which favorably avoids the
cumbersome training process in existing methods. Moreover, the transition
distribution can contract the error of the restoration backbone and thus makes
our method more robust to unknown degradations. Comprehensive experiments show
that DifFace is superior to current state-of-the-art methods, especially in
cases with severe degradations. Our code and model are available at
https://github.com/zsyOAOA/DifFace.Comment: 21 page
Decomposition Ascribed Synergistic Learning for Unified Image Restoration
Learning to restore multiple image degradations within a single model is
quite beneficial for real-world applications. Nevertheless, existing works
typically concentrate on regarding each degradation independently, while their
relationship has been less exploited to ensure the synergistic learning. To
this end, we revisit the diverse degradations through the lens of singular
value decomposition, with the observation that the decomposed singular vectors
and singular values naturally undertake the different types of degradation
information, dividing various restoration tasks into two groups,\ie, singular
vector dominated and singular value dominated. The above analysis renders a
more unified perspective to ascribe the diverse degradations, compared to
previous task-level independent learning. The dedicated optimization of
degraded singular vectors and singular values inherently utilizes the potential
relationship among diverse restoration tasks, attributing to the Decomposition
Ascribed Synergistic Learning (DASL). Specifically, DASL comprises two
effective operators, namely, Singular VEctor Operator (SVEO) and Singular VAlue
Operator (SVAO), to favor the decomposed optimization, which can be lightly
integrated into existing convolutional image restoration backbone. Moreover,
the congruous decomposition loss has been devised for auxiliary. Extensive
experiments on blended five image restoration tasks demonstrate the
effectiveness of our method, including image deraining, image dehazing, image
denoising, image deblurring, and low-light image enhancement.Comment: 13 page
- …