182,381 research outputs found
UV-Based 3D Hand-Object Reconstruction with Grasp Optimization
We propose a novel framework for 3D hand shape reconstruction and hand-object
grasp optimization from a single RGB image. The representation of hand-object
contact regions is critical for accurate reconstructions. Instead of
approximating the contact regions with sparse points, as in previous works, we
propose a dense representation in the form of a UV coordinate map. Furthermore,
we introduce inference-time optimization to fine-tune the grasp and improve
interactions between the hand and the object. Our pipeline increases hand shape
reconstruction accuracy and produces a vibrant hand texture. Experiments on
datasets such as Ho3D, FreiHAND, and DexYCB reveal that our proposed method
outperforms the state-of-the-art.Comment: BMVC 2022 Spotligh
Single-Sweep Methods for Free Energy Calculations
A simple, efficient, and accurate method is proposed to map multi-dimensional
free energy landscapes. The method combines the temperature-accelerated
molecular dynamics (TAMD) proposed in [Maragliano & Vanden-Eijnden, Chem. Phys.
Lett. 426, 168 (2006)] with a variational reconstruction method using
radial-basis functions for the representation of the free energy. TAMD is used
to rapidly sweep through the important regions of the free energy landscape and
compute the gradient of the free energy locally at points in these regions. The
variational method is then used to reconstruct the free energy globally from
the mean force at these points. The algorithmic aspects of the single-sweep
method are explained in detail, and the method is tested on simple examples,
compared to metadynamics, and finally used to compute the free energy of the
solvated alanine dipeptide in two and four dihedral angles
SAIR: Learning Semantic-aware Implicit Representation
Implicit representation of an image can map arbitrary coordinates in the
continuous domain to their corresponding color values, presenting a powerful
capability for image reconstruction. Nevertheless, existing implicit
representation approaches only focus on building continuous appearance mapping,
ignoring the continuities of the semantic information across pixels. As a
result, they can hardly achieve desired reconstruction results when the
semantic information within input images is corrupted, for example, a large
region misses. To address the issue, we propose to learn semantic-aware
implicit representation (SAIR), that is, we make the implicit representation of
each pixel rely on both its appearance and semantic information (\eg, which
object does the pixel belong to). To this end, we propose a framework with two
modules: (1) building a semantic implicit representation (SIR) for a corrupted
image whose large regions miss. Given an arbitrary coordinate in the continuous
domain, we can obtain its respective text-aligned embedding indicating the
object the pixel belongs. (2) building an appearance implicit representation
(AIR) based on the SIR. Given an arbitrary coordinate in the continuous domain,
we can reconstruct its color whether or not the pixel is missed in the input.
We validate the novel semantic-aware implicit representation method on the
image inpainting task, and the extensive experiments demonstrate that our
method surpasses state-of-the-art approaches by a significant margin
Extreme 3D Face Reconstruction: Seeing Through Occlusions
Existing single view, 3D face reconstruction methods can produce beautifully
detailed 3D results, but typically only for near frontal, unobstructed
viewpoints. We describe a system designed to provide detailed 3D
reconstructions of faces viewed under extreme conditions, out of plane
rotations, and occlusions. Motivated by the concept of bump mapping, we propose
a layered approach which decouples estimation of a global shape from its
mid-level details (e.g., wrinkles). We estimate a coarse 3D face shape which
acts as a foundation and then separately layer this foundation with details
represented by a bump map. We show how a deep convolutional encoder-decoder can
be used to estimate such bump maps. We further show how this approach naturally
extends to generate plausible details for occluded facial regions. We test our
approach and its components extensively, quantitatively demonstrating the
invariance of our estimated facial details. We further provide numerous
qualitative examples showing that our method produces detailed 3D face shapes
in viewing conditions where existing state of the art often break down.Comment: Accepted to CVPR'18. Previously titled: "Extreme 3D Face
Reconstruction: Looking Past Occlusions
High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference
We propose a data-driven method for recovering miss-ing parts of 3D shapes.
Our method is based on a new deep learning architecture consisting of two
sub-networks: a global structure inference network and a local geometry
refinement network. The global structure inference network incorporates a long
short-term memorized context fusion module (LSTM-CF) that infers the global
structure of the shape based on multi-view depth information provided as part
of the input. It also includes a 3D fully convolutional (3DFCN) module that
further enriches the global structure representation according to volumetric
information in the input. Under the guidance of the global structure network,
the local geometry refinement network takes as input lo-cal 3D patches around
missing regions, and progressively produces a high-resolution, complete surface
through a volumetric encoder-decoder architecture. Our method jointly trains
the global structure inference and local geometry refinement networks in an
end-to-end manner. We perform qualitative and quantitative evaluations on six
object categories, demonstrating that our method outperforms existing
state-of-the-art work on shape completion.Comment: 8 pages paper, 11 pages supplementary material, ICCV spotlight pape
Meshed Up: Learnt Error Correction in 3D Reconstructions
Dense reconstructions often contain errors that prior work has so far
minimised using high quality sensors and regularising the output. Nevertheless,
errors still persist. This paper proposes a machine learning technique to
identify errors in three dimensional (3D) meshes. Beyond simply identifying
errors, our method quantifies both the magnitude and the direction of depth
estimate errors when viewing the scene. This enables us to improve the
reconstruction accuracy.
We train a suitably deep network architecture with two 3D meshes: a
high-quality laser reconstruction, and a lower quality stereo image
reconstruction. The network predicts the amount of error in the lower quality
reconstruction with respect to the high-quality one, having only view the
former through its input. We evaluate our approach by correcting
two-dimensional (2D) inverse-depth images extracted from the 3D model, and show
that our method improves the quality of these depth reconstructions by up to a
relative 10% RMSE.Comment: Accepted for the International Conference on Robotics and Automation
(ICRA) 201
- …