3,723 research outputs found
Learn to synthesize and synthesize to learn
Attribute guided face image synthesis aims to manipulate attributes on a face
image. Most existing methods for image-to-image translation can either perform
a fixed translation between any two image domains using a single attribute or
require training data with the attributes of interest for each subject.
Therefore, these methods could only train one specific model for each pair of
image domains, which limits their ability in dealing with more than two
domains. Another disadvantage of these methods is that they often suffer from
the common problem of mode collapse that degrades the quality of the generated
images. To overcome these shortcomings, we propose attribute guided face image
generation method using a single model, which is capable to synthesize multiple
photo-realistic face images conditioned on the attributes of interest. In
addition, we adopt the proposed model to increase the realism of the simulated
face images while preserving the face characteristics. Compared to existing
models, synthetic face images generated by our method present a good
photorealistic quality on several face datasets. Finally, we demonstrate that
generated facial images can be used for synthetic data augmentation, and
improve the performance of the classifier used for facial expression
recognition.Comment: Accepted to Computer Vision and Image Understanding (CVIU
Geometry-Aware Face Completion and Editing
Face completion is a challenging generation task because it requires
generating visually pleasing new pixels that are semantically consistent with
the unmasked face region. This paper proposes a geometry-aware Face Completion
and Editing NETwork (FCENet) by systematically studying facial geometry from
the unmasked region. Firstly, a facial geometry estimator is learned to
estimate facial landmark heatmaps and parsing maps from the unmasked face
image. Then, an encoder-decoder structure generator serves to complete a face
image and disentangle its mask areas conditioned on both the masked face image
and the estimated facial geometry images. Besides, since low-rank property
exists in manually labeled masks, a low-rank regularization term is imposed on
the disentangled masks, enforcing our completion network to manage occlusion
area with various shape and size. Furthermore, our network can generate diverse
results from the same masked input by modifying estimated facial geometry,
which provides a flexible mean to edit the completed face appearance. Extensive
experimental results qualitatively and quantitatively demonstrate that our
network is able to generate visually pleasing face completion results and edit
face attributes as well
Prominent Attribute Modification using Attribute Dependent Generative Adversarial Network
Modifying the facial images with desired attributes is important, though
challenging tasks in computer vision, where it aims to modify single or
multiple attributes of the face image. Some of the existing methods are either
based on attribute independent approaches where the modification is done in the
latent representation or attribute dependent approaches. The attribute
independent methods are limited in performance as they require the desired
paired data for changing the desired attributes. Secondly, the attribute
independent constraint may result in the loss of information and, hence, fail
in generating the required attributes in the face image. In contrast, the
attribute dependent approaches are effective as these approaches are capable of
modifying the required features along with preserving the information in the
given image. However, attribute dependent approaches are sensitive and require
a careful model design in generating high-quality results. To address this
problem, we propose an attribute dependent face modification approach. The
proposed approach is based on two generators and two discriminators that
utilize the binary as well as the real representation of the attributes and, in
return, generate high-quality attribute modification results. Experiments on
the CelebA dataset show that our method effectively performs the multiple
attribute editing with preserving other facial details intactly
Using Photorealistic Face Synthesis and Domain Adaptation to Improve Facial Expression Analysis
Cross-domain synthesizing realistic faces to learn deep models has attracted
increasing attention for facial expression analysis as it helps to improve the
performance of expression recognition accuracy despite having small number of
real training images. However, learning from synthetic face images can be
problematic due to the distribution discrepancy between low-quality synthetic
images and real face images and may not achieve the desired performance when
the learned model applies to real world scenarios. To this end, we propose a
new attribute guided face image synthesis to perform a translation between
multiple image domains using a single model. In addition, we adopt the proposed
model to learn from synthetic faces by matching the feature distributions
between different domains while preserving each domain's characteristics. We
evaluate the effectiveness of the proposed approach on several face datasets on
generating realistic face images. We demonstrate that the expression
recognition performance can be enhanced by benefiting from our face synthesis
model. Moreover, we also conduct experiments on a near-infrared dataset
containing facial expression videos of drivers to assess the performance using
in-the-wild data for driver emotion recognition.Comment: 8 pages, 8 figures, 5 tables, accepted by FG 2019. arXiv admin note:
substantial text overlap with arXiv:1905.0028
Hierarchy Composition GAN for High-fidelity Image Synthesis
Despite the rapid progress of generative adversarial networks (GANs) in image
synthesis in recent years, the existing image synthesis approaches work in
either geometry domain or appearance domain alone which often introduces
various synthesis artifacts. This paper presents an innovative Hierarchical
Composition GAN (HIC-GAN) that incorporates image synthesis in geometry and
appearance domains into an end-to-end trainable network and achieves superior
synthesis realism in both domains simultaneously. We design an innovative
hierarchical composition mechanism that is capable of learning realistic
composition geometry and handling occlusions while multiple foreground objects
are involved in image composition. In addition, we introduce a novel attention
mask mechanism that guides to adapt the appearance of foreground objects which
also helps to provide better training reference for learning in geometry
domain. Extensive experiments on scene text image synthesis, portrait editing
and indoor rendering tasks show that the proposed HIC-GAN achieves superior
synthesis performance qualitatively and quantitatively.Comment: 11 pages, 8 figure
Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation
Generative Neural Radiance Fields (GNeRF) based 3D-aware GANs have
demonstrated remarkable capabilities in generating high-quality images while
maintaining strong 3D consistency. Notably, significant advancements have been
made in the domain of face generation. However, most existing models prioritize
view consistency over disentanglement, resulting in limited semantic/attribute
control during generation. To address this limitation, we propose a conditional
GNeRF model incorporating specific attribute labels as input to enhance the
controllability and disentanglement abilities of 3D-aware generative models.
Our approach builds upon a pre-trained 3D-aware face model, and we introduce a
Training as Init and Optimizing for Tuning (TRIOT) method to train a
conditional normalized flow module to enable the facial attribute editing, then
optimize the latent vector to improve attribute-editing precision further. Our
extensive experiments demonstrate that our model produces high-quality edits
with superior view consistency while preserving non-target regions. Code is
available at https://github.com/zhangqianhui/TT-GNeRF.Comment: 13 page
- …