35 research outputs found
A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization
We propose NeuFace, a 3D face mesh pseudo annotation method on videos via
neural re-parameterized optimization. Despite the huge progress in 3D face
reconstruction methods, generating reliable 3D face labels for in-the-wild
dynamic videos remains challenging. Using NeuFace optimization, we annotate the
per-view/-frame accurate and consistent face meshes on large-scale face videos,
called the NeuFace-dataset. We investigate how neural re-parameterization helps
to reconstruct image-aligned facial details on 3D meshes via gradient analysis.
By exploiting the naturalness and diversity of 3D faces in our dataset, we
demonstrate the usefulness of our dataset for 3D face-related tasks: improving
the reconstruction accuracy of an existing 3D face reconstruction model and
learning 3D facial motion prior. Code and datasets will be available at
https://neuface-dataset.github.io.Comment: 9 pages, 7 figures, and 3 tables for the main paper. 8 pages, 6
figures and 3 tables for the appendi
Scene-Adaptive Video Frame Interpolation via Meta-Learning
Video frame interpolation is a challenging problem because there are
different scenarios for each video depending on the variety of foreground and
background motion, frame rate, and occlusion. It is therefore difficult for a
single network with fixed parameters to generalize across different videos.
Ideally, one could have a different network for each scenario, but this is
computationally infeasible for practical applications. In this work, we propose
to adapt the model to each video by making use of additional information that
is readily available at test time and yet has not been exploited in previous
works. We first show the benefits of `test-time adaptation' through simple
fine-tuning of a network, then we greatly improve its efficiency by
incorporating meta-learning. We obtain significant performance gains with only
a single gradient update without any additional parameters. Finally, we show
that our meta-learning framework can be easily employed to any video frame
interpolation network and can consistently improve its performance on multiple
benchmark datasets.Comment: CVPR 202
LaughTalk: Expressive 3D Talking Head Generation with Laughter
Laughter is a unique expression, essential to affirmative social interactions
of humans. Although current 3D talking head generation methods produce
convincing verbal articulations, they often fail to capture the vitality and
subtleties of laughter and smiles despite their importance in social context.
In this paper, we introduce a novel task to generate 3D talking heads capable
of both articulate speech and authentic laughter. Our newly curated dataset
comprises 2D laughing videos paired with pseudo-annotated and human-validated
3D FLAME parameters and vertices. Given our proposed dataset, we present a
strong baseline with a two-stage training scheme: the model first learns to
talk and then acquires the ability to express laughter. Extensive experiments
demonstrate that our method performs favorably compared to existing approaches
in both talking head generation and expressing laughter signals. We further
explore potential applications on top of our proposed method for rigging
realistic avatars.Comment: Accepted to WACV202
Recommended from our members
Propagation of Load Shed in Cascading Line Outages Simulated by OPA
Current Limiter Product Informatio
Recommended from our members
Testing branching process estimators of cascading failure with data from a simulation of transmission line outages
Report prepared for the California Energy Commission