1,165 research outputs found
Conditional Adversarial Synthesis of 3D Facial Action Units
Employing deep learning-based approaches for fine-grained facial expression
analysis, such as those involving the estimation of Action Unit (AU)
intensities, is difficult due to the lack of a large-scale dataset of real
faces with sufficiently diverse AU labels for training. In this paper, we
consider how AU-level facial image synthesis can be used to substantially
augment such a dataset. We propose an AU synthesis framework that combines the
well-known 3D Morphable Model (3DMM), which intrinsically disentangles
expression parameters from other face attributes, with models that
adversarially generate 3DMM expression parameters conditioned on given target
AU labels, in contrast to the more conventional approach of generating facial
images directly. In this way, we are able to synthesize new combinations of
expression parameters and facial images from desired AU labels. Extensive
quantitative and qualitative results on the benchmark DISFA dataset demonstrate
the effectiveness of our method on 3DMM facial expression parameter synthesis
and data augmentation for deep learning-based AU intensity estimation
Dynamic Facial Expression Generation on Hilbert Hypersphere with Conditional Wasserstein Generative Adversarial Nets
In this work, we propose a novel approach for generating videos of the six
basic facial expressions given a neutral face image. We propose to exploit the
face geometry by modeling the facial landmarks motion as curves encoded as
points on a hypersphere. By proposing a conditional version of manifold-valued
Wasserstein generative adversarial network (GAN) for motion generation on the
hypersphere, we learn the distribution of facial expression dynamics of
different classes, from which we synthesize new facial expression motions. The
resulting motions can be transformed to sequences of landmarks and then to
images sequences by editing the texture information using another conditional
Generative Adversarial Network. To the best of our knowledge, this is the first
work that explores manifold-valued representations with GAN to address the
problem of dynamic facial expression generation. We evaluate our proposed
approach both quantitatively and qualitatively on two public datasets;
Oulu-CASIA and MUG Facial Expression. Our experimental results demonstrate the
effectiveness of our approach in generating realistic videos with continuous
motion, realistic appearance and identity preservation. We also show the
efficiency of our framework for dynamic facial expressions generation, dynamic
facial expression transfer and data augmentation for training improved emotion
recognition models
Region Based Adversarial Synthesis of Facial Action Units
Facial expression synthesis or editing has recently received increasing
attention in the field of affective computing and facial expression modeling.
However, most existing facial expression synthesis works are limited in paired
training data, low resolution, identity information damaging, and so on. To
address those limitations, this paper introduces a novel Action Unit (AU) level
facial expression synthesis method called Local Attentive Conditional
Generative Adversarial Network (LAC-GAN) based on face action units
annotations. Given desired AU labels, LAC-GAN utilizes local AU regional rules
to control the status of each AU and attentive mechanism to combine several of
them into the whole photo-realistic facial expressions or arbitrary facial
expressions. In addition, unpaired training data is utilized in our proposed
method to train the manipulation module with the corresponding AU labels, which
learns a mapping between a facial expression manifold. Extensive qualitative
and quantitative evaluations are conducted on the commonly used BP4D dataset to
verify the effectiveness of our proposed AU synthesis method.Comment: Accepted by MMM202
- …