4,019 research outputs found
Integrating Deep Semantic Segmentation Into 3-D Point Cloud Registration
Point cloud registration is the task of aligning 3D scans of the same environment captured from different poses. When semantic information is available for the points, it can be used as a prior in the search for correspondences to improve registration. Semantic-assisted Normal Distributions Transform (SE-NDT) is a new registration algorithm that reduces the complexity of the problem by using the semantic information to partition the point cloud into a set of normal distributions, which are then registered separately. In this paper we extend the NDT registration pipeline by using PointNet, a deep neural network for segmentation and classification of point clouds, to learn and predict per-point semantic labels. We also present the Iterative Closest Point (ICP) equivalent of the algorithm, a special case of Multichannel Generalized ICP. We evaluate the performance of SE-NDT against the state of the art in point cloud registration on the publicly available classification data set Semantic3d.net. We also test the trained classifier and algorithms on dynamic scenes, using a sequence from the public dataset KITTI. The experiments demonstrate the improvement of the registration in terms of robustness, precision and speed, across a range of initial registration errors, thanks to the inclusion of semantic information
A One Stop 3D Target Reconstruction and multilevel Segmentation Method
3D object reconstruction and multilevel segmentation are fundamental to
computer vision research. Existing algorithms usually perform 3D scene
reconstruction and target objects segmentation independently, and the
performance is not fully guaranteed due to the challenge of the 3D
segmentation. Here we propose an open-source one stop 3D target reconstruction
and multilevel segmentation framework (OSTRA), which performs segmentation on
2D images, tracks multiple instances with segmentation labels in the image
sequence, and then reconstructs labelled 3D objects or multiple parts with
Multi-View Stereo (MVS) or RGBD-based 3D reconstruction methods. We extend
object tracking and 3D reconstruction algorithms to support continuous
segmentation labels to leverage the advances in the 2D image segmentation,
especially the Segment-Anything Model (SAM) which uses the pretrained neural
network without additional training for new scenes, for 3D object segmentation.
OSTRA supports most popular 3D object models including point cloud, mesh and
voxel, and achieves high performance for semantic segmentation, instance
segmentation and part segmentation on several 3D datasets. It even surpasses
the manual segmentation in scenes with complex structures and occlusions. Our
method opens up a new avenue for reconstructing 3D targets embedded with rich
multi-scale segmentation information in complex scenes. OSTRA is available from
https://github.com/ganlab/OSTRA
PADLoC: LiDAR-Based Deep Loop Closure Detection and Registration using Panoptic Attention
A key component of graph-based SLAM systems is the ability to detect loop
closures in a trajectory to reduce the drift accumulated over time from the
odometry. Most LiDAR-based methods achieve this goal by using only the
geometric information, disregarding the semantics of the scene. In this work,
we introduce PADLoC, a LiDAR-based loop closure detection and registration
architecture comprising a shared 3D convolutional feature extraction backbone,
a global descriptor head for loop closure detection, and a novel
transformer-based head for point cloud matching and registration. We present
multiple methods for estimating the point-wise matching confidence based on
diversity indices. Additionally, to improve forward-backward consistency, we
propose the use of two shared matching and registration heads with their source
and target inputs swapped by exploiting that the estimated relative
transformations must be inverse of each other. Furthermore, we leverage
panoptic information during training in the form of a novel loss function that
reframes the matching problem as a classification task in the case of the
semantic labels and as a graph connectivity assignment for the instance labels.
We perform extensive evaluations of PADLoC on multiple real-world datasets
demonstrating that it achieves state-of-the-art performance. The code of our
work is publicly available at http://padloc.cs.uni-freiburg.de
- …