A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization

We propose NeuFace, a 3D face mesh pseudo annotation method on videos via neural re-parameterized optimization. Despite the huge progress in 3D face reconstruction methods, generating reliable 3D face labels for in-the-wild dynamic videos remains challenging. Using NeuFace optimization, we annotate the per-view/-frame accurate and consistent face meshes on large-scale face videos, called the NeuFace-dataset. We investigate how neural re-parameterization helps to reconstruct image-aligned facial details on 3D meshes via gradient analysis. By exploiting the naturalness and diversity of 3D faces in our dataset, we demonstrate the usefulness of our dataset for 3D face-related tasks: improving the reconstruction accuracy of an existing 3D face reconstruction model and learning 3D facial motion prior. Code and datasets will be available at https://neuface-dataset.github.io.Comment: 9 pages, 7 figures, and 3 tables for the main paper. 8 pages, 6 figures and 3 tables for the appendi

CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes

1

Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers

1

Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise

1

Multi-stage Adaptive Rank Statistic Pruning for Lightweight Human 3D Mesh Recovery Model

We present a rank statistic adaptive multi-stage pruning method to find lightweight neural networks for 3D human mesh recovery while minimizing accuracy drop. We observe that some feature maps often have prominent low-rank patterns regardless of input human images. Furthermore, even after pruning, feature channels that should have been pruned according to pruning criteria frequently re-appear in test time. From these observations, we design rank statistic adaptive multi-stage pruning; thereby, we can prune more filters with recovering mesh reconstruction accuracy. We demonstrate that, for DenseNet-121, 60.0% of parameters and 67.9% of FLOPs are saved while maintaining comparable accuracy to that of the original full model. This is a notable improvement compared to the competing method based on the L1 filter pruning, where the error is increased by 17.55% at the same pruning rate.11Nsciescopu

Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise

We propose an end-to-end unified 3D mesh recovery of humans and quadruped animals trained in a weakly-supervised way. Unlike recent work focusing on a single target class only, we aim to recover 3D mesh of broader classes with a single multi-task model. However, there exists no dataset that can directly enable multi-task learning due to the absence of both human and animal annotations for a single object, e.g., a human image does not have animal pose annotations; thus, we have to devise a new way to exploit heterogeneous datasets. To make the unstable disjoint multi-task learning jointly trainable, we propose to exploit the morphological similarity between humans and animals, motivated by animal exercise where humans imitate animal poses. We realize the morphological similarity by semantic correspondences, called sub-keypoint, which enables joint training of human and animal mesh regression branches. Besides, we propose class-sensitive regularization methods to avoid a mean-shape bias and to improve the distinctiveness across multi-classes. Our method performs favorably against recent uni-modal models on various human and animal datasets while being far more compact