Deep Semantic Graph Matching for Large-scale Outdoor Point Clouds Registration

Current point cloud registration methods are mainly based on local geometric information and usually ignore the semantic information contained in the scenes. In this paper, we treat the point cloud registration problem as a semantic instance matching and registration task, and propose a deep semantic graph matching method (DeepSGM) for large-scale outdoor point cloud registration. Firstly, the semantic categorical labels of 3D points are obtained using a semantic segmentation network. The adjacent points with the same category labels are then clustered together using the Euclidean clustering algorithm to obtain the semantic instances, which are represented by three kinds of attributes including spatial location information, semantic categorical information, and global geometric shape information. Secondly, the semantic adjacency graph is constructed based on the spatial adjacency relations of semantic instances. To fully explore the topological structures between semantic instances in the same scene and across different scenes, the spatial distribution features and the semantic categorical features are learned with graph convolutional networks, and the global geometric shape features are learned with a PointNet-like network. These three kinds of features are further enhanced with the self-attention and cross-attention mechanisms. Thirdly, the semantic instance matching is formulated as an optimal transport problem, and solved through an optimal matching layer. Finally, the geometric transformation matrix between two point clouds is first estimated by the SVD algorithm and then refined by the ICP algorithm. Experimental results conducted on the KITTI Odometry dataset demonstrate that the proposed method improves the registration performance and outperforms various state-of-the-art methods.Comment: 12 pages, 6 figure

Robotic Assembly Using 3D and 2D Computer Vision

The content of this thesis concerns the development and evaluation of a robotic cell used for automated assembly. The automated assembly is made possible by a combination of an eye-inhand 2D camera and a stationary 3D camera used to automatically detect objects. Computer vision, kinematics and programming is the main topics of the thesis. Possible approaches to object detection has been investigated and evaluated in terms of performance. The kinematic relation between the cameras in the robotic cell and robotic manipulator movements has been described. A functioning solution has been implemented in the robotic cell at the Department of Production and Quality Engineering laboratory. Theory with significant importance to the developed solution is presented. The methods used to achieve each part of the solution is anchored in theory and presented with the decisions and guidelines made throughout the project work in order to achieve the final solution. Each part of the system is presented with associated results. The combination of these results yields a solution which proves that the methods developed to achieve automated assembly works as intended. Limitations, challenges and future possibilities and improvements for the solution is then discussed. The results from the experiments presented in this thesis demonstrates the performance of the developed system. The system fulfills the specifications defined in the problem description and is functioning as intended considering the instrumentation used