3,303 research outputs found
Nonrigid reconstruction of 3D breast surfaces with a low-cost RGBD camera for surgical planning and aesthetic evaluation
Accounting for 26% of all new cancer cases worldwide, breast cancer remains
the most common form of cancer in women. Although early breast cancer has a
favourable long-term prognosis, roughly a third of patients suffer from a
suboptimal aesthetic outcome despite breast conserving cancer treatment.
Clinical-quality 3D modelling of the breast surface therefore assumes an
increasingly important role in advancing treatment planning, prediction and
evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive
and either infrastructure-heavy or subject to motion artefacts. In this paper
we employ a single consumer-grade RGBD camera with an ICP-based registration
approach to jointly align all points from a sequence of depth images
non-rigidly. Subtle body deformation due to postural sway and respiration is
successfully mitigated leading to a higher geometric accuracy through
regularised locally affine transformations. We present results from 6 clinical
cases where our method compares well with the gold standard and outperforms a
previous approach. We show that our method produces better reconstructions
qualitatively by visual assessment and quantitatively by consistently obtaining
lower landmark error scores and yielding more accurate breast volume estimates
Human Performance Modeling and Rendering via Neural Animated Mesh
We have recently seen tremendous progress in the neural advances for
photo-real human modeling and rendering. However, it's still challenging to
integrate them into an existing mesh-based pipeline for downstream
applications. In this paper, we present a comprehensive neural approach for
high-quality reconstruction, compression, and rendering of human performances
from dense multi-view videos. Our core intuition is to bridge the traditional
animated mesh workflow with a new class of highly efficient neural techniques.
We first introduce a neural surface reconstructor for high-quality surface
generation in minutes. It marries the implicit volumetric rendering of the
truncated signed distance field (TSDF) with multi-resolution hash encoding. We
further propose a hybrid neural tracker to generate animated meshes, which
combines explicit non-rigid tracking with implicit dynamic deformation in a
self-supervised framework. The former provides the coarse warping back into the
canonical space, while the latter implicit one further predicts the
displacements using the 4D hash encoding as in our reconstructor. Then, we
discuss the rendering schemes using the obtained animated meshes, ranging from
dynamic texturing to lumigraph rendering under various bandwidth settings. To
strike an intricate balance between quality and bandwidth, we propose a
hierarchical solution by first rendering 6 virtual views covering the performer
and then conducting occlusion-aware neural texture blending. We demonstrate the
efficacy of our approach in a variety of mesh-based applications and
photo-realistic free-view experiences on various platforms, i.e., inserting
virtual human performances into real environments through mobile AR or
immersively watching talent shows with VR headsets.Comment: 18 pages, 17 figure
HDHumans: A Hybrid Approach for High-fidelity Digital Humans
Photo-real digital human avatars are of enormous importance in graphics, asthey enable immersive communication over the globe, improve gaming andentertainment experiences, and can be particularly beneficial for AR and VRsettings. However, current avatar generation approaches either fall short inhigh-fidelity novel view synthesis, generalization to novel motions,reproduction of loose clothing, or they cannot render characters at the highresolution offered by modern displays. To this end, we propose HDHumans, whichis the first method for HD human character synthesis that jointly produces anaccurate and temporally coherent 3D deforming surface and highlyphoto-realistic images of arbitrary novel views and of motions not seen attraining time. At the technical core, our method tightly integrates a classicaldeforming character template with neural radiance fields (NeRF). Our method iscarefully designed to achieve a synergy between classical surface deformationand NeRF. First, the template guides the NeRF, which allows synthesizing novelviews of a highly dynamic and articulated character and even enables thesynthesis of novel motions. Second, we also leverage the dense pointcloudsresulting from NeRF to further improve the deforming surface via 3D-to-3Dsupervision. We outperform the state of the art quantitatively andqualitatively in terms of synthesis quality and resolution, as well as thequality of 3D surface reconstruction.<br
- …