293,677 research outputs found
ASM: Adaptive Skinning Model for High-Quality 3D Face Modeling
The research fields of parametric face models and 3D face reconstruction have
been extensively studied. However, a critical question remains unanswered: how
to tailor the face model for specific reconstruction settings. We argue that
reconstruction with multi-view uncalibrated images demands a new model with
stronger capacity. Our study shifts attention from data-dependent 3D Morphable
Models (3DMM) to an understudied human-designed skinning model. We propose
Adaptive Skinning Model (ASM), which redefines the skinning model with more
compact and fully tunable parameters. With extensive experiments, we
demonstrate that ASM achieves significantly improved capacity than 3DMM, with
the additional advantage of model size and easy implementation for new
topology. We achieve state-of-the-art performance with ASM for multi-view
reconstruction on the Florence MICC Coop benchmark. Our quantitative analysis
demonstrates the importance of a high-capacity model for fully exploiting
abundant information from multi-view input in reconstruction. Furthermore, our
model with physical-semantic parameters can be directly utilized for real-world
applications, such as in-game avatar creation. As a result, our work opens up
new research directions for the parametric face models and facilitates future
research on multi-view reconstruction
Volumetric performance capture from minimal camera viewpoints
We present a convolutional autoencoder that enables high fidelity volumetric
reconstructions of human performance to be captured from multi-view video
comprising only a small set of camera views. Our method yields similar
end-to-end reconstruction error to that of a probabilistic visual hull computed
using significantly more (double or more) viewpoints. We use a deep prior
implicitly learned by the autoencoder trained over a dataset of view-ablated
multi-view video footage of a wide range of subjects and actions. This opens up
the possibility of high-end volumetric performance capture in on-set and
prosumer scenarios where time or cost prohibit a high witness camera count
From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion
We consider the problem of Multi-view 3D Face Reconstruction (MVR) with
weakly supervised learning that leverages a limited number of 2D face images
(e.g. 3) to generate a high-quality 3D face model with very light annotation.
Despite their encouraging performance, present MVR methods simply concatenate
multi-view image features and pay less attention to critical areas (e.g. eye,
brow, nose and mouth). To this end, we propose a novel model called Deep Fusion
MVR (DF-MVR) and design a multi-view encoding to a single decoding framework
with skip connections, able to extract, integrate, and compensate deep features
with attention from multi-view images. In addition, we develop a multi-view
face parse network to learn, identify, and emphasize the critical common face
area. Finally, though our model is trained with a few 2D images, it can
reconstruct an accurate 3D model even if one single 2D image is input. We
conduct extensive experiments to evaluate various multi-view 3D face
reconstruction methods. Our proposed model attains superior performance,
leading to 11.4% RMSE improvement over the existing best weakly supervised
MVRs. Source codes are available in the supplementary materials
Geometric Structure Extraction and Reconstruction
Geometric structure extraction and reconstruction is a long-standing problem in research communities including computer graphics, computer vision, and machine learning. Within different communities, it can be interpreted as different subproblems such as skeleton extraction from the point cloud, surface reconstruction from multi-view images, or manifold learning from high dimensional data. All these subproblems are building blocks of many modern applications, such as scene reconstruction for AR/VR, object recognition for robotic vision and structural analysis for big data. Despite its importance, the extraction and reconstruction of a geometric structure from real-world data are ill-posed, where the main challenges lie in the incompleteness, noise, and inconsistency of the raw input data. To address these challenges, three studies are conducted in this thesis: i) a new point set representation for shape completion, ii) a structure-aware data consolidation method, and iii) a data-driven deep learning technique for multi-view consistency. In addition to theoretical contributions, the algorithms we proposed significantly improve the performance of several state-of-the-art geometric structure extraction and reconstruction approaches, validated by extensive experimental results
- …