1,449 research outputs found
Multi-view PointNet for 3D Scene Understanding
Fusion of 2D images and 3D point clouds is important because information from
dense images can enhance sparse point clouds. However, fusion is challenging
because 2D and 3D data live in different spaces. In this work, we propose
MVPNet (Multi-View PointNet), where we aggregate 2D multi-view image features
into 3D point clouds, and then use a point based network to fuse the features
in 3D canonical space to predict 3D semantic labels. To this end, we introduce
view selection along with a 2D-3D feature aggregation module. Extensive
experiments show the benefit of leveraging features from dense images and
reveal superior robustness to varying point cloud density compared to 3D-only
methods. On the ScanNetV2 benchmark, our MVPNet significantly outperforms prior
point cloud based approaches on the task of 3D Semantic Segmentation. It is
much faster to train than the large networks of the sparse voxel approach. We
provide solid ablation studies to ease the future design of 2D-3D fusion
methods and their extension to other tasks, as we showcase for 3D instance
segmentation.Comment: Geometry Meets Deep Learning Workshop, ICCV 201
Pointwise Convolutional Neural Networks
Deep learning with 3D data such as reconstructed point clouds and CAD models
has received great research interests recently. However, the capability of
using point clouds with convolutional neural network has been so far not fully
explored. In this paper, we present a convolutional neural network for semantic
segmentation and object recognition with 3D point clouds. At the core of our
network is pointwise convolution, a new convolution operator that can be
applied at each point of a point cloud. Our fully convolutional network design,
while being surprisingly simple to implement, can yield competitive accuracy in
both semantic segmentation and object recognition task.Comment: 10 pages, 6 figures, 10 tables. Paper accepted to CVPR 201
SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation
We introduce Similarity Group Proposal Network (SGPN), a simple and intuitive
deep learning framework for 3D object instance segmentation on point clouds.
SGPN uses a single network to predict point grouping proposals and a
corresponding semantic class for each proposal, from which we can directly
extract instance segmentation results. Important to the effectiveness of SGPN
is its novel representation of 3D instance segmentation results in the form of
a similarity matrix that indicates the similarity between each pair of points
in embedded feature space, thus producing an accurate grouping proposal for
each point. To the best of our knowledge, SGPN is the first framework to learn
3D instance-aware semantic segmentation on point clouds. Experimental results
on various 3D scenes show the effectiveness of our method on 3D instance
segmentation, and we also evaluate the capability of SGPN to improve 3D object
detection and semantic segmentation results. We also demonstrate its
flexibility by seamlessly incorporating 2D CNN features into the framework to
boost performance
- …