12,036 research outputs found
Region Based Adversarial Synthesis of Facial Action Units
Facial expression synthesis or editing has recently received increasing
attention in the field of affective computing and facial expression modeling.
However, most existing facial expression synthesis works are limited in paired
training data, low resolution, identity information damaging, and so on. To
address those limitations, this paper introduces a novel Action Unit (AU) level
facial expression synthesis method called Local Attentive Conditional
Generative Adversarial Network (LAC-GAN) based on face action units
annotations. Given desired AU labels, LAC-GAN utilizes local AU regional rules
to control the status of each AU and attentive mechanism to combine several of
them into the whole photo-realistic facial expressions or arbitrary facial
expressions. In addition, unpaired training data is utilized in our proposed
method to train the manipulation module with the corresponding AU labels, which
learns a mapping between a facial expression manifold. Extensive qualitative
and quantitative evaluations are conducted on the commonly used BP4D dataset to
verify the effectiveness of our proposed AU synthesis method.Comment: Accepted by MMM202
MoFaNeRF: Morphable Facial Neural Radiance Field
We propose a parametric model that maps free-view images into a vector space
of coded facial shape, expression and appearance with a neural radiance field,
namely Morphable Facial NeRF. Specifically, MoFaNeRF takes the coded facial
shape, expression and appearance along with space coordinate and view direction
as input to an MLP, and outputs the radiance of the space point for
photo-realistic image synthesis. Compared with conventional 3D morphable models
(3DMM), MoFaNeRF shows superiority in directly synthesizing photo-realistic
facial details even for eyes, mouths, and beards. Also, continuous face
morphing can be easily achieved by interpolating the input shape, expression
and appearance codes. By introducing identity-specific modulation and texture
encoder, our model synthesizes accurate photometric details and shows strong
representation ability. Our model shows strong ability on multiple applications
including image-based fitting, random generation, face rigging, face editing,
and novel view synthesis. Experiments show that our method achieves higher
representation ability than previous parametric models, and achieves
competitive performance in several applications. To the best of our knowledge,
our work is the first facial parametric model built upon a neural radiance
field that can be used in fitting, generation and manipulation. The code and
data is available at https://github.com/zhuhao-nju/mofanerf.Comment: accepted to ECCV2022; code available at
http://github.com/zhuhao-nju/mofaner
Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks
Generating realistic 3D faces is of high importance for computer graphics and
computer vision applications. Generally, research on 3D face generation
revolves around linear statistical models of the facial surface. Nevertheless,
these models cannot represent faithfully either the facial texture or the
normals of the face, which are very crucial for photo-realistic face synthesis.
Recently, it was demonstrated that Generative Adversarial Networks (GANs) can
be used for generating high-quality textures of faces. Nevertheless, the
generation process either omits the geometry and normals, or independent
processes are used to produce 3D shape information. In this paper, we present
the first methodology that generates high-quality texture, shape, and normals
jointly, which can be used for photo-realistic synthesis. To do so, we propose
a novel GAN that can generate data from different modalities while exploiting
their correlations. Furthermore, we demonstrate how we can condition the
generation on the expression and create faces with various facial expressions.
The qualitative results shown in this paper are compressed due to size
limitations, full-resolution results and the accompanying video can be found in
the supplementary documents. The code and models are available at the project
page: https://github.com/barisgecer/TBGAN.Comment: Check project page: https://github.com/barisgecer/TBGAN for the full
resolution results and the accompanying vide
Using Photorealistic Face Synthesis and Domain Adaptation to Improve Facial Expression Analysis
Cross-domain synthesizing realistic faces to learn deep models has attracted
increasing attention for facial expression analysis as it helps to improve the
performance of expression recognition accuracy despite having small number of
real training images. However, learning from synthetic face images can be
problematic due to the distribution discrepancy between low-quality synthetic
images and real face images and may not achieve the desired performance when
the learned model applies to real world scenarios. To this end, we propose a
new attribute guided face image synthesis to perform a translation between
multiple image domains using a single model. In addition, we adopt the proposed
model to learn from synthetic faces by matching the feature distributions
between different domains while preserving each domain's characteristics. We
evaluate the effectiveness of the proposed approach on several face datasets on
generating realistic face images. We demonstrate that the expression
recognition performance can be enhanced by benefiting from our face synthesis
model. Moreover, we also conduct experiments on a near-infrared dataset
containing facial expression videos of drivers to assess the performance using
in-the-wild data for driver emotion recognition.Comment: 8 pages, 8 figures, 5 tables, accepted by FG 2019. arXiv admin note:
substantial text overlap with arXiv:1905.0028
HeadOn: Real-time Reenactment of Human Portrait Videos
We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.Comment: Video: https://www.youtube.com/watch?v=7Dg49wv2c_g Presented at
Siggraph'1
Learn to synthesize and synthesize to learn
Attribute guided face image synthesis aims to manipulate attributes on a face
image. Most existing methods for image-to-image translation can either perform
a fixed translation between any two image domains using a single attribute or
require training data with the attributes of interest for each subject.
Therefore, these methods could only train one specific model for each pair of
image domains, which limits their ability in dealing with more than two
domains. Another disadvantage of these methods is that they often suffer from
the common problem of mode collapse that degrades the quality of the generated
images. To overcome these shortcomings, we propose attribute guided face image
generation method using a single model, which is capable to synthesize multiple
photo-realistic face images conditioned on the attributes of interest. In
addition, we adopt the proposed model to increase the realism of the simulated
face images while preserving the face characteristics. Compared to existing
models, synthetic face images generated by our method present a good
photorealistic quality on several face datasets. Finally, we demonstrate that
generated facial images can be used for synthetic data augmentation, and
improve the performance of the classifier used for facial expression
recognition.Comment: Accepted to Computer Vision and Image Understanding (CVIU
- …