Cross-source point cloud matching by exploring structure property

University of Technology Sydney. Faculty of Engineering and Information Technology.Cross-source point cloud are 3D data coming from heterogeneous sensors. The matching of cross-source point cloud is extremely difficult because they contain mixture of different variations, such as missing data, noise and outliers, different viewpoint, density and spatial transformation. In this thesis, cross-source point cloud matching is solved from three aspects, utilizing of structure information, statistical model and learning representation. Chapter 1 introduces the value of cross-source point cloud registration and summarizes the key challenges of cross-source point cloud registration problem. Chapter 2 reviews the existing registration methods and analyse their limitation in solving the cross-source point cloud registration problem. Chapter 3 proposes two algorithms to discuss how to utilize structure information to solve the cross-source point cloud registration problem. In the first part of this chapter, macro and micro structures are extracted based on 3D point cloud segmentation. Then, these macro and micro structure components are integrated into a graph. With novel descriptors generated, the registration problem is successfully converted into graph matching problem. In the second part, weak region affinity and pixel-wise refinement are proposed to solve the cross-source point cloud. These two components are unified represented into a tensor space and the registration problem is converted into tensor optimization problem. In this method, the tensor space is updated when the transformation matrix is updated to get feedback from the recent transformation estimation step. Chapter 4 discusses how to utilize the statistical distribution of cross-source point cloud to solve matching problem. The goal is to find the potential matching region and estimate the accurate registration relationship. In this chapter, ensemble of shape functions (ESF) is utilized to select potential regions and a novel registration is proposed to solve the matching problem. For the registration, Gaussian mixture models (GMM) is selected as our mathematical tool. However, different to previous GMM-based registration methods, which assume a GMM for each point cloud, the proposed algorithm assumes a virtual GMM and the cross-source point clouds are samples from the virtual GMM. Then, the transformation is optimized to project the samples into a same virtual GMM. When the optimization is convergence, both the parameters of GMM and the transformation matrices are estimated. In Chapter 5, a deep learning method is proposed to represent the local structure information. Because of arbitrary rotation in cross-source point clouds, a rotation-invariant 3D representation method is proposed to robust represent the 3D point cloud although there are arbitrary rotation and translation. Also, there is no robust keypoints in these cross-source point cloud because of they come from heterogenous sensors, train the network is very difficult. A region-based method is proposed to generate regions for each point cloud and synthetic labelled dataset is constructed for training the network. All these algorithms are aimed to solve the cross-source point cloud registration problem. The performance of these algorithms is tested on many datasets, which shows the effective and correctness. These algorithms also provide insightful knowledge for 3D computer vision workers to process 3D point cloud

3D registration and integrated segmentation framework for heterogeneous unmanned robotic systems

The paper proposes a novel framework for registering and segmenting 3D point clouds of large-scale natural terrain and complex environments coming from a multisensor heterogeneous robotics system, consisting of unmanned aerial and ground vehicles. This framework involves data acquisition and pre-processing, 3D heterogeneous registration and integrated multi-sensor based segmentation modules. The first module provides robust and accurate homogeneous registrations of 3D environmental models based on sensors' measurements acquired from the ground (UGV) and aerial (UAV) robots. For 3D UGV registration, we proposed a novel local minima escape ICP (LME-ICP) method, which is based on the well known iterative closest point (ICP) algorithm extending it by the introduction of our local minima estimation and local minima escape mechanisms. It did not require any prior known pose estimation information acquired from sensing systems like odometry, global positioning system (GPS), or inertial measurement units (IMU). The 3D UAV registration has been performed using the Structure from Motion (SfM) approach. In order to improve and speed up the process of outliers removal for large-scale outdoor environments, we introduced the Fast Cluster Statistical Outlier Removal (FCSOR) method. This method was used to filter out the noise and to downsample the input data, which will spare computational and memory resources for further processing steps. Then, we co-registered a point cloud acquired from a laser ranger (UGV) and a point cloud generated from images (UAV) generated by the SfM method. The 3D heterogeneous module consists of a semi-automated 3D scan registration system, developed with the aim to overcome the shortcomings of the existing fully automated 3D registration approaches. This semi-automated registration system is based on the novel Scale Invariant Registration Method (SIRM). The SIRM provides the initial scaling between two heterogenous point clouds and provides an adaptive mechanism for tuning the mean scale, based on the difference between two consecutive estimated point clouds' alignment error values. Once aligned, the resulting homogeneous ground-aerial point cloud is further processed by a segmentation module. For this purpose, we have proposed a system for integrated multi-sensor based segmentation of 3D point clouds. This system followed a two steps sequence: ground-object segmentation and color-based region-growing segmentation. The experimental validation of the proposed 3D heterogeneous registration and integrated segmentation framework was performed on large-scale datasets representing unstructured outdoor environments, demonstrating the potential and benefits of the proposed semi-automated 3D registration system in real-world environments

KSS-ICP: Point Cloud Registration based on Kendall Shape Space

Point cloud registration is a popular topic which has been widely used in 3D model reconstruction, location, and retrieval. In this paper, we propose a new registration method, KSS-ICP, to address the rigid registration task in Kendall shape space (KSS) with Iterative Closest Point (ICP). The KSS is a quotient space that removes influences of translations, scales, and rotations for shape feature-based analysis. Such influences can be concluded as the similarity transformations that do not change the shape feature. The point cloud representation in KSS is invariant to similarity transformations. We utilize such property to design the KSS-ICP for point cloud registration. To tackle the difficulty to achieve the KSS representation in general, the proposed KSS-ICP formulates a practical solution that does not require complex feature analysis, data training, and optimization. With a simple implementation, KSS-ICP achieves more accurate registration from point clouds. It is robust to similarity transformation, non-uniform density, noise, and defective parts. Experiments show that KSS-ICP has better performance than the state of the art.Comment: 13 pages, 20 figure

Robust Building-based Registration of Airborne LiDAR Data and Optical Imagery on Urban Scenes

The motivation of this paper is to address the problem of registering airborne LiDAR data and optical aerial or satellite imagery acquired from different platforms, at different times, with different points of view and levels of detail. In this paper, we present a robust registration method based on building regions, which are extracted from optical images using mean shift segmentation, and from LiDAR data using a 3D point cloud filtering process. The matching of the extracted building segments is then carried out using Graph Transformation Matching (GTM) which allows to determine a common pattern of relative positions of segment centers. Thanks to this registration, the relative shifts between the data sets are significantly reduced, which enables a subsequent fine registration and a resulting high-quality data fusion

Pairwise registration of TLS point clouds by deep multi-scale local features

Abstract(#br)Because of the mechanism of TLS system, noise, outliers, various occlusions, varying cloud densities, etc. inevitably exist in the collection of TLS point clouds. To achieve automatic TLS point cloud registration, many methods, based on the hand-crafted features of keypoints, have been proposed. Despite significant progress, the current methods still face great challenges in accomplishing TLS point cloud registration. In this paper, we propose a multi-scale neural network to learn local shape descriptors for establishing correspondences between pairwise TLS point clouds. To train our model, data augmentation, developed on pairwise semi-synthetic 3D local patches, is to extend our network to be robust to rotation transformation. Then, based on varying local neighborhoods, multi-scale subnetworks are constructed and fused to learn robust local features. Experimental results demonstrate that our proposed method successfully registers two TLS point clouds and outperforms state-of-the-art methods. Besides, our learned descriptors are invariant to translation and tolerant to changes in rotation

Numerical methods for polyline‐to‐point‐cloud registration with applications to patient‐specific stent reconstruction

We present novel numerical methods for polyline‐to‐point‐cloud registration and their application to patient‐specific modeling of deployed coronary artery stents from image data. Patient‐specific coronary stent reconstruction is an important challenge in computational hemodynamics and relevant to the design and improvement of the prostheses. It is an invaluable tool in large‐scale clinical trials that computationally investigate the effect of new generations of stents on hemodynamics and eventually tissue remodeling. Given a point cloud of strut positions, which can be extracted from images, our stent reconstruction method aims at finding a geometrical transformation that aligns a model of the undeployed stent to the point cloud. Mathematically, we describe the undeployed stent as a polyline, which is a piecewise linear object defined by its vertices and edges. We formulate the nonlinear registration as an optimization problem whose objective function consists of a similarity measure, quantifying the distance between the polyline and the point cloud, and a regularization functional, penalizing undesired transformations. Using projections of points onto the polyline structure, we derive novel distance measures. Our formulation supports most commonly used transformation models including very flexible nonlinear deformations. We also propose 2 regularization approaches ensuring the smoothness of the estimated nonlinear transformation. We demonstrate the potential of our methods using an academic 2D example and a real‐life 3D bioabsorbable stent reconstruction problem. Our results show that the registration problem can be solved to sufficient accuracy within seconds using only a few number of Gauss‐Newton iterations.We present novel numerical methods for nonlinear polyline‐to‐point‐cloud registration and their application to patient‐specific modeling of deployed coronary artery stents from image data. We design a general and mathematically sound framework that includes novel (almost everywhere) differentiable distance measures and 2 new regularization approaches to overcome the ill‐posedness and enable robust registration in the presence of outliers. We demonstrate that 3D registration problem arising in stent reconstruction can be solved within seconds using only a small number of Gauss‐Newton iterations.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/142552/1/cnm2934.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/142552/2/cnm2934_am.pd