2,584 research outputs found
Recommended from our members
The role of HG in the analysis of temporal iteration and interaural correlation
LiveCap: Real-time Human Performance Capture from Monocular Video
We present the first real-time human performance capture approach that
reconstructs dense, space-time coherent deforming geometry of entire humans in
general everyday clothing from just a single RGB video. We propose a novel
two-stage analysis-by-synthesis optimization whose formulation and
implementation are designed for high performance. In the first stage, a skinned
template model is jointly fitted to background subtracted input video, 2D and
3D skeleton joint positions found using a deep neural network, and a set of
sparse facial landmark detections. In the second stage, dense non-rigid 3D
deformations of skin and even loose apparel are captured based on a novel
real-time capable algorithm for non-rigid tracking using dense photometric and
silhouette constraints. Our novel energy formulation leverages automatically
identified material regions on the template to model the differing non-rigid
deformation behavior of skin and apparel. The two resulting non-linear
optimization problems per-frame are solved with specially-tailored
data-parallel Gauss-Newton solvers. In order to achieve real-time performance
of over 25Hz, we design a pipelined parallel architecture using the CPU and two
commodity GPUs. Our method is the first real-time monocular approach for
full-body performance capture. Our method yields comparable accuracy with
off-line performance capture techniques, while being orders of magnitude
faster
Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation
Morphable models are essential for the statistical modeling of 3D faces.
Previous works on morphable models mostly focus on large-scale facial geometry
but ignore facial details. This paper augments morphable models in representing
facial details by learning a Structure-aware Editable Morphable Model (SEMM).
SEMM introduces a detail structure representation based on the distance field
of wrinkle lines, jointly modeled with detail displacements to establish better
correspondences and enable intuitive manipulation of wrinkle structure.
Besides, SEMM introduces two transformation modules to translate expression
blendshape weights and age values into changes in latent space, allowing
effective semantic detail editing while maintaining identity. Extensive
experiments demonstrate that the proposed model compactly represents facial
details, outperforms previous methods in expression animation qualitatively and
quantitatively, and achieves effective age editing and wrinkle line editing of
facial details. Code and model are available at
https://github.com/gerwang/facial-detail-manipulation.Comment: ECCV 202
- …