57 research outputs found
Shape Completion with Points in the Shadow
Single-view point cloud completion aims to recover the full geometry of an
object based on only limited observation, which is extremely hard due to the
data sparsity and occlusion. The core challenge is to generate plausible
geometries to fill the unobserved part of the object based on a partial scan,
which is under-constrained and suffers from a huge solution space. Inspired by
the classic shadow volume technique in computer graphics, we propose a new
method to reduce the solution space effectively. Our method considers the
camera a light source that casts rays toward the object. Such light rays build
a reasonably constrained but sufficiently expressive basis for completion. The
completion process is then formulated as a point displacement optimization
problem. Points are initialized at the partial scan and then moved to their
goal locations with two types of movements for each point: directional
movements along the light rays and constrained local movement for shape
refinement. We design neural networks to predict the ideal point movements to
get the completion results. We demonstrate that our method is accurate, robust,
and generalizable through exhaustive evaluation and comparison. Moreover, it
outperforms state-of-the-art methods qualitatively and quantitatively on MVP
datasets.Comment: SIGGRAPH Aisa 2022 Conference Pape
Asynchronous Collaborative Autoscanning with Mode Switching for Multi-Robot Scene Reconstruction
When conducting autonomous scanning for the online reconstruction of unknown
indoor environments, robots have to be competent at exploring scene structure
and reconstructing objects with high quality. Our key observation is that
different tasks demand specialized scanning properties of robots: rapid moving
speed and far vision for global exploration and slow moving speed and narrow
vision for local object reconstruction, which are referred as two different
scanning modes: explorer and reconstructor, respectively. When requiring
multiple robots to collaborate for efficient exploration and fine-grained
reconstruction, the questions on when to generate and how to assign those tasks
should be carefully answered. Therefore, we propose a novel asynchronous
collaborative autoscanning method with mode switching, which generates two
kinds of scanning tasks with associated scanning modes, i.e., exploration task
with explorer mode and reconstruction task with reconstructor mode, and assign
them to the robots to execute in an asynchronous collaborative manner to highly
boost the scanning efficiency and reconstruction quality. The task assignment
is optimized by solving a modified Multi-Depot Multiple Traveling Salesman
Problem (MDMTSP). Moreover, to further enhance the collaboration and increase
the efficiency, we propose a task-flow model that actives the task generation
and assignment process immediately when any of the robots finish all its tasks
with no need to wait for all other robots to complete the tasks assigned in the
previous iteration. Extensive experiments have been conducted to show the
importance of each key component of our method and the superiority over
previous methods in scanning efficiency and reconstruction quality.Comment: 13pages, 12 figures, Conference: SIGGRAPH Asia 202
AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose
How human interact with objects depends on the functional roles of the target
objects, which introduces the problem of affordance-aware hand-object
interaction. It requires a large number of human demonstrations for the
learning and understanding of plausible and appropriate hand-object
interactions. In this work, we present AffordPose, a large-scale dataset of
hand-object interactions with affordance-driven hand pose. We first annotate
the specific part-level affordance labels for each object, e.g. twist, pull,
handle-grasp, etc, instead of the general intents such as use or handover, to
indicate the purpose and guide the localization of the hand-object
interactions. The fine-grained hand-object interactions reveal the influence of
hand-centered affordances on the detailed arrangement of the hand poses, yet
also exhibit a certain degree of diversity. We collect a total of 26.7K
hand-object interactions, each including the 3D object shape, the part-level
affordance label, and the manually adjusted hand poses. The comprehensive data
analysis shows the common characteristics and diversity of hand-object
interactions per affordance via the parameter statistics and contacting
computation. We also conduct experiments on the tasks of hand-object affordance
understanding and affordance-oriented hand-object interaction generation, to
validate the effectiveness of our dataset in learning the fine-grained
hand-object interactions. Project page:
https://github.com/GentlesJan/AffordPose.Comment: Accepted by ICCV 202
Interaction-Driven Active 3D Reconstruction with Object Interiors
We introduce an active 3D reconstruction method which integrates visual
perception, robot-object interaction, and 3D scanning to recover both the
exterior and interior, i.e., unexposed, geometries of a target 3D object.
Unlike other works in active vision which focus on optimizing camera viewpoints
to better investigate the environment, the primary feature of our
reconstruction is an analysis of the interactability of various parts of the
target object and the ensuing part manipulation by a robot to enable scanning
of occluded regions. As a result, an understanding of part articulations of the
target object is obtained on top of complete geometry acquisition. Our method
operates fully automatically by a Fetch robot with built-in RGBD sensors. It
iterates between interaction analysis and interaction-driven reconstruction,
scanning and reconstructing detected moveable parts one at a time, where both
the articulated part detection and mesh reconstruction are carried out by
neural networks. In the final step, all the remaining, non-articulated parts,
including all the interior structures that had been exposed by prior part
manipulations and subsequently scanned, are reconstructed to complete the
acquisition. We demonstrate the performance of our method via qualitative and
quantitative evaluation, ablation studies, comparisons to alternatives, as well
as experiments in a real environment.Comment: Accepted to SIGGRAPH Asia 2023, project page at
https://vcc.tech/research/2023/InterReco
ARO-Net: Learning Implicit Fields from Anchored Radial Observations
We introduce anchored radial observations (ARO), a novel shape encoding for
learning implicit field representation of 3D shapes that is category-agnostic
and generalizable amid significant shape variations. The main idea behind our
work is to reason about shapes through partial observations from a set of
viewpoints, called anchors. We develop a general and unified shape
representation by employing a fixed set of anchors, via Fibonacci sampling, and
designing a coordinate-based deep neural network to predict the occupancy value
of a query point in space. Differently from prior neural implicit models that
use global shape feature, our shape encoder operates on contextual,
query-specific features. To predict point occupancy, locally observed shape
information from the perspective of the anchors surrounding the input query
point are encoded and aggregated through an attention module, before implicit
decoding is performed. We demonstrate the quality and generality of our
network, coined ARO-Net, on surface reconstruction from sparse point clouds,
with tests on novel and unseen object categories, "one-shape" training, and
comparisons to state-of-the-art neural and classical methods for reconstruction
and tessellation.Comment: Accepted by CVPR 2023. Code: https://github.com/yizhiwang96/ARO-Ne
- …