6,367 research outputs found
CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigm
Point cloud completion aims to predict complete shape from its partial
observation. Current approaches mainly consist of generation and refinement
stages in a coarse-to-fine style. However, the generation stage often lacks
robustness to tackle different incomplete variations, while the refinement
stage blindly recovers point clouds without the semantic awareness. To tackle
these challenges, we unify point cloud Completion by a generic
Pretrain-Prompt-Predict paradigm, namely CP3. Inspired by prompting approaches
from NLP, we creatively reinterpret point cloud generation and refinement as
the prompting and predicting stages, respectively. Then, we introduce a concise
self-supervised pretraining stage before prompting. It can effectively increase
robustness of point cloud generation, by an Incompletion-Of-Incompletion (IOI)
pretext task. Moreover, we develop a novel Semantic Conditional Refinement
(SCR) network at the predicting stage. It can discriminatively modulate
multi-scale refinement with the guidance of semantics. Finally, extensive
experiments demonstrate that our CP3 outperforms the state-of-the-art methods
with a large margin
Point-PC: Point Cloud Completion Guided by Prior Knowledge via Causal Inference
Point cloud completion aims to recover raw point clouds captured by scanners
from partial observations caused by occlusion and limited view angles. Many
approaches utilize a partial-complete paradigm in which missing parts are
directly predicted by a global feature learned from partial inputs. This makes
it hard to recover details because the global feature is unlikely to capture
the full details of all missing parts. In this paper, we propose a novel
approach to point cloud completion called Point-PC, which uses a memory network
to retrieve shape priors and designs an effective causal inference model to
choose missing shape information as additional geometric information to aid
point cloud completion. Specifically, we propose a memory operating mechanism
where the complete shape features and the corresponding shapes are stored in
the form of ``key-value'' pairs. To retrieve similar shapes from the partial
input, we also apply a contrastive learning-based pre-training scheme to
transfer features of incomplete shapes into the domain of complete shape
features. Moreover, we use backdoor adjustment to get rid of the confounder,
which is a part of the shape prior that has the same semantic structure as the
partial input. Experimental results on the ShapeNet-55, PCN, and KITTI datasets
demonstrate that Point-PC performs favorably against the state-of-the-art
methods
Variational Relational Point Completion Network for Robust 3D Classification
Real-scanned point clouds are often incomplete due to viewpoint, occlusion,
and noise, which hampers 3D geometric modeling and perception. Existing point
cloud completion methods tend to generate global shape skeletons and hence lack
fine local details. Furthermore, they mostly learn a deterministic
partial-to-complete mapping, but overlook structural relations in man-made
objects. To tackle these challenges, this paper proposes a variational
framework, Variational Relational point Completion Network (VRCNet) with two
appealing properties: 1) Probabilistic Modeling. In particular, we propose a
dual-path architecture to enable principled probabilistic modeling across
partial and complete clouds. One path consumes complete point clouds for
reconstruction by learning a point VAE. The other path generates complete
shapes for partial point clouds, whose embedded distribution is guided by
distribution obtained from the reconstruction path during training. 2)
Relational Enhancement. Specifically, we carefully design point self-attention
kernel and point selective kernel module to exploit relational point features,
which refines local shape details conditioned on the coarse completion. In
addition, we contribute multi-view partial point cloud datasets (MVP and MVP-40
dataset) containing over 200,000 high-quality scans, which render partial 3D
shapes from 26 uniformly distributed camera poses for each 3D CAD model.
Extensive experiments demonstrate that VRCNet outperforms state-of-the-art
methods on all standard point cloud completion benchmarks. Notably, VRCNet
shows great generalizability and robustness on real-world point cloud scans.
Moreover, we can achieve robust 3D classification for partial point clouds with
the help of VRCNet, which can highly increase classification accuracy.Comment: 12 pages, 10 figures, accepted by PAMI. project webpage:
https://mvp-dataset.github.io/. arXiv admin note: substantial text overlap
with arXiv:2104.1015
Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformer
Most existing point cloud completion methods suffer from the discrete nature
of point clouds and the unstructured prediction of points in local regions,
which makes it difficult to reveal fine local geometric details. To resolve
this issue, we propose SnowflakeNet with snowflake point deconvolution (SPD) to
generate complete point clouds. SPD models the generation of point clouds as
the snowflake-like growth of points, where child points are generated
progressively by splitting their parent points after each SPD. Our insight into
the detailed geometry is to introduce a skip-transformer in the SPD to learn
the point splitting patterns that can best fit the local regions. The
skip-transformer leverages attention mechanism to summarize the splitting
patterns used in the previous SPD layer to produce the splitting in the current
layer. The locally compact and structured point clouds generated by SPD
precisely reveal the structural characteristics of the 3D shape in local
patches, which enables us to predict highly detailed geometries. Moreover,
since SPD is a general operation that is not limited to completion, we explore
its applications in other generative tasks, including point cloud
auto-encoding, generation, single image reconstruction, and upsampling. Our
experimental results outperform state-of-the-art methods under widely used
benchmarks.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence
(TPAMI), 2022. This work is a journal extension of our ICCV 2021 paper
arXiv:2108.04444 . The first two authors contributed equall
- …