98 research outputs found
FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
Face Super-Resolution (SR) is a domain-specific super-resolution problem. The
specific facial prior knowledge could be leveraged for better super-resolving
face images. We present a novel deep end-to-end trainable Face Super-Resolution
Network (FSRNet), which makes full use of the geometry prior, i.e., facial
landmark heatmaps and parsing maps, to super-resolve very low-resolution (LR)
face images without well-aligned requirement. Specifically, we first construct
a coarse SR network to recover a coarse high-resolution (HR) image. Then, the
coarse HR image is sent to two branches: a fine SR encoder and a prior
information estimation network, which extracts the image features, and
estimates landmark heatmaps/parsing maps respectively. Both image features and
prior information are sent to a fine SR decoder to recover the HR image. To
further generate realistic faces, we propose the Face Super-Resolution
Generative Adversarial Network (FSRGAN) to incorporate the adversarial loss
into FSRNet. Moreover, we introduce two related tasks, face alignment and
parsing, as the new evaluation metrics for face SR, which address the
inconsistency of classic metrics w.r.t. visual perception. Extensive benchmark
experiments show that FSRNet and FSRGAN significantly outperforms state of the
arts for very LR face SR, both quantitatively and qualitatively. Code will be
made available upon publication.Comment: Chen and Tai contributed equally to this pape
Facial Attribute Capsules for Noise Face Super Resolution
Existing face super-resolution (SR) methods mainly assume the input image to
be noise-free. Their performance degrades drastically when applied to
real-world scenarios where the input image is always contaminated by noise. In
this paper, we propose a Facial Attribute Capsules Network (FACN) to deal with
the problem of high-scale super-resolution of noisy face image. Capsule is a
group of neurons whose activity vector models different properties of the same
entity. Inspired by the concept of capsule, we propose an integrated
representation model of facial information, which named Facial Attribute
Capsule (FAC). In the SR processing, we first generated a group of FACs from
the input LR face, and then reconstructed the HR face from this group of FACs.
Aiming to effectively improve the robustness of FAC to noise, we generate FAC
in semantic, probabilistic and facial attributes manners by means of integrated
learning strategy. Each FAC can be divided into two sub-capsules: Semantic
Capsule (SC) and Probabilistic Capsule (PC). Them describe an explicit facial
attribute in detail from two aspects of semantic representation and probability
distribution. The group of FACs model an image as a combination of facial
attribute information in the semantic space and probabilistic space by an
attribute-disentangling way. The diverse FACs could better combine the face
prior information to generate the face images with fine-grained semantic
attributes. Extensive benchmark experiments show that our method achieves
superior hallucination results and outperforms state-of-the-art for very low
resolution (LR) noise face image super resolution.Comment: To appear in AAAI 202
- …