3,049 research outputs found
FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
Face Super-Resolution (SR) is a domain-specific super-resolution problem. The
specific facial prior knowledge could be leveraged for better super-resolving
face images. We present a novel deep end-to-end trainable Face Super-Resolution
Network (FSRNet), which makes full use of the geometry prior, i.e., facial
landmark heatmaps and parsing maps, to super-resolve very low-resolution (LR)
face images without well-aligned requirement. Specifically, we first construct
a coarse SR network to recover a coarse high-resolution (HR) image. Then, the
coarse HR image is sent to two branches: a fine SR encoder and a prior
information estimation network, which extracts the image features, and
estimates landmark heatmaps/parsing maps respectively. Both image features and
prior information are sent to a fine SR decoder to recover the HR image. To
further generate realistic faces, we propose the Face Super-Resolution
Generative Adversarial Network (FSRGAN) to incorporate the adversarial loss
into FSRNet. Moreover, we introduce two related tasks, face alignment and
parsing, as the new evaluation metrics for face SR, which address the
inconsistency of classic metrics w.r.t. visual perception. Extensive benchmark
experiments show that FSRNet and FSRGAN significantly outperforms state of the
arts for very LR face SR, both quantitatively and qualitatively. Code will be
made available upon publication.Comment: Chen and Tai contributed equally to this pape
Generative Prior for Unsupervised Image Restoration
The challenge of restoring real world low-quality images is due to a lack of appropriate training data and difficulty in determining how the image was degraded. Recently, generative models have demonstrated great potential for creating high- quality images by utilizing the rich and diverse information contained within the model’s trained weights and learned latent representations. One popular type of generative model is the generative adversarial network (GAN). Many new methods have been developed to harness the information found in GANs for image manipulation. Our proposed approach is to utilize generative models for both understanding the degradation of an image and restoring it. We propose using a combination of cycle consistency losses and self-attention to enhance face images by first learning the degradation and then using this information to train a style-based neural network. We also aim to use the latent representation to achieve a high level of magnification for face images (x64). By incorporating the weights of a pre-trained StyleGAN into a restoration network with a vision transformer layer, we hope to improve the current state-of-the-art in face image restoration. Finally, we present a projection-based image-denoising algorithm named Noise2Code in the latent space of the VQGAN model with a fixed-point regularization strategy. The fixed-point condition follows the observation that the pre-trained VQGAN affects the clean and noisy images in a drastically different way. Unlike previous projection-based image restoration in the latent space, both the denoising network and VQGAN model parameters are jointly trained, although the latter is not needed during the testing. We report experimental results to demonstrate that the proposed Noise2Code approach is conceptually simple, computationally efficient, and generalizable to real-world degradation scenarios
Heavy Rain Face Image Restoration: Integrating Physical Degradation Model and Facial Component Guided Adversarial Learning
With the recent increase in intelligent CCTVs for visual surveillance, a new
image degradation that integrates resolution conversion and synthetic rain
models is required. For example, in heavy rain, face images captured by CCTV
from a distance have significant deterioration in both visibility and
resolution. Unlike traditional image degradation models (IDM), such as rain
removal and superresolution, this study addresses a new IDM referred to as a
scale-aware heavy rain model and proposes a method for restoring
high-resolution face images (HR-FIs) from low-resolution heavy rain face images
(LRHR-FI). To this end, a 2-stage network is presented. The first stage
generates low-resolution face images (LR-FIs), from which heavy rain has been
removed from the LRHR-FIs to improve visibility. To realize this, an
interpretable IDM-based network is constructed to predict physical parameters,
such as rain streaks, transmission maps, and atmospheric light. In addition,
the image reconstruction loss is evaluated to enhance the estimates of the
physical parameters. For the second stage, which aims to reconstruct the HR-FIs
from the LR-FIs outputted in the first stage, facial component guided
adversarial learning (FCGAL) is applied to boost facial structure expressions.
To focus on informative facial features and reinforce the authenticity of
facial components, such as the eyes and nose, a face-parsing-guided generator
and facial local discriminators are designed for FCGAL. The experimental
results verify that the proposed approach based on physical-based network
design and FCGAL can remove heavy rain and increase the resolution and
visibility simultaneously. Moreover, the proposed heavy-rain face image
restoration outperforms state-of-the-art models of heavy rain removal,
image-to-image translation, and superresolution
Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution
Real-world face super-resolution (SR) is a highly ill-posed image restoration
task. The fully-cycled Cycle-GAN architecture is widely employed to achieve
promising performance on face SR, but prone to produce artifacts upon
challenging cases in real-world scenarios, since joint participation in the
same degradation branch will impact final performance due to huge domain gap
between real-world and synthetic LR ones obtained by generators. To better
exploit the powerful generative capability of GAN for real-world face SR, in
this paper, we establish two independent degradation branches in the forward
and backward cycle-consistent reconstruction processes, respectively, while the
two processes share the same restoration branch. Our Semi-Cycled Generative
Adversarial Networks (SCGAN) is able to alleviate the adverse effects of the
domain gap between the real-world LR face images and the synthetic LR ones, and
to achieve accurate and robust face SR performance by the shared restoration
branch regularized by both the forward and backward cycle-consistent learning
processes. Experiments on two synthetic and two real-world datasets demonstrate
that, our SCGAN outperforms the state-of-the-art methods on recovering the face
structures/details and quantitative metrics for real-world face SR. The code
will be publicly released at https://github.com/HaoHou-98/SCGAN
- …