Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds

To deal with the exhausting annotations, self-supervised representation learning from unlabeled point clouds has drawn much attention, especially centered on augmentation-based contrastive methods. However, specific augmentations hardly produce sufficient transferability to high-level tasks on different datasets. Besides, augmentations on point clouds may also change underlying semantics. To address the issues, we propose a simple but efficient augmentation fusion contrastive learning framework to combine data augmentations in Euclidean space and feature augmentations in feature space. In particular, we propose a data augmentation method based on sampling and graph generation. Meanwhile, we design a data augmentation network to enable a correspondence of representations by maximizing consistency between augmented graph pairs. We further design a feature augmentation network that encourages the model to learn representations invariant to the perturbations using an encoder perturbation. We comprehensively conduct extensive object classification experiments and object part segmentation experiments to validate the transferability of the proposed framework. Experimental results demonstrate that the proposed framework is effective to learn the point cloud representation in a self-supervised manner, and yields state-of-the-art results in the community. The source code is publicly available at: https://zhiyongsu.github.io/Project/AFSRL.html

Thoracic Cartilage Ultrasound-CT Registration using Dense Skeleton Graph

Autonomous ultrasound (US) imaging has gained increased interest recently, and it has been seen as a potential solution to overcome the limitations of free-hand US examinations, such as inter-operator variations. However, it is still challenging to accurately map planned paths from a generic atlas to individual patients, particularly for thoracic applications with high acoustic-impedance bone structures under the skin. To address this challenge, a graph-based non-rigid registration is proposed to enable transferring planned paths from the atlas to the current setup by explicitly considering subcutaneous bone surface features instead of the skin surface. To this end, the sternum and cartilage branches are segmented using a template matching to assist coarse alignment of US and CT point clouds. Afterward, a directed graph is generated based on the CT template. Then, the self-organizing map using geographical distance is successively performed twice to extract the optimal graph representations for CT and US point clouds, individually. To evaluate the proposed approach, five cartilage point clouds from distinct patients are employed. The results demonstrate that the proposed graph-based registration can effectively map trajectories from CT to the current setup for displaying US views through limited intercostal space. The non-rigid registration results in terms of Hausdorff distance (Mean$\pm$SD) is 9.48$\pm$0.27 mm and the path transferring error in terms of Euclidean distance is 2.21$\pm$1.11 mm.Comment: Accepted by IROS2

Robust Kernel-based Feature Representation for 3D Point Cloud Analysis via Circular Graph Convolutional Network

Feature descriptors of point clouds are used in several applications, such as registration and part segmentation of 3D point clouds. Learning discriminative representations of local geometric features is unquestionably the most important task for accurate point cloud analyses. However, it is challenging to develop rotation or scale-invariant descriptors. Most previous studies have either ignored rotations or empirically studied optimal scale parameters, which hinders the applicability of the methods for real-world datasets. In this paper, we present a new local feature description method that is robust to rotation, density, and scale variations. Moreover, to improve representations of the local descriptors, we propose a global aggregation method. First, we place kernels aligned around each point in the normal direction. To avoid the sign problem of the normal vector, we use a symmetric kernel point distribution in the tangential plane. From each kernel point, we first projected the points from the spatial space to the feature space, which is robust to multiple scales and rotation, based on angles and distances. Subsequently, we perform graph convolutions by considering local kernel point structures and long-range global context, obtained by a global aggregation method. We experimented with our proposed descriptors on benchmark datasets (i.e., ModelNet40 and ShapeNetPart) to evaluate the performance of registration, classification, and part segmentation on 3D point clouds. Our method showed superior performances when compared to the state-of-the-art methods by reducing 70$\%$ of the rotation and translation errors in the registration task. Our method also showed comparable performance in the classification and part-segmentation tasks with simple and low-dimensional architectures.Comment: 10 pages, 9 figure

Detection of 3D Object in Point Cloud: Cloud Semantic Segmentation in Lane Marking

Managing a city efficiently and effectively is more important than ever as growing population and economic strain put a strain on infrastructure like transportation and public services like keeping urban green areas clean and maintained. For effective administration, knowledge of the urban setting is essential. Both portable and stationary laser scanners generate 3D point clouds that accurately depict the environment. These data points may be used to infer the state of the roads, buildings, trees, and other important elements involved in this decision-making process. Perhaps they would support "smart" or "smarter" cities in general. Unfortunately, the point clouds do not immediately supply this sort of data. It must be eliminated. This extraction is done either by human specialists or by sophisticated computer programmes that can identify objects. Because the point clouds might represent such large locations, relying on specialists to identify the things may be an unproductive use of time (streets or even whole cities). Automatic or nearly automatic discovery and recognition of essential objects is now possible with the help of object identification software. In this research, In this paper, we describe a unique approach to semantic segmentation of point clouds, based on the usage of contextual point representations to take use of both local and global features within the point cloud. We improve the accuracy of the point's representation by performing a single innovative gated fusion on the point and its neighbours, which incorporates the knowledge from both sets of data and enhances the representation of the point. Following this, we offer a new graph point net module that further develops the improved representation by composing and updating each point's representation inside the local point cloud structure using the graph attention block in real time. Finally, we make advantage of the global structure of the point cloud by using spatial- and channel-wise attention techniques to construct the ensuing semantic label for each point

Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraph

Large datasets represented by multidimensional data point clouds often possess non-trivial distributions with branching trajectories and excluded regions, with the recent single-cell transcriptomic studies of developing embryo being notable examples. Reducing the complexity and producing compact and interpretable representations of such data remains a challenging task. Most of the existing computational methods are based on exploring the local data point neighbourhood relations, a step that can perform poorly in the case of multidimensional and noisy data. Here we present ElPiGraph, a scalable and robust method for approximation of datasets with complex structures which does not require computing the complete data distance matrix or the data point neighbourhood graph. This method is able to withstand high levels of noise and is capable of approximating complex topologies via principal graph ensembles that can be combined into a consensus principal graph. ElPiGraph deals efficiently with large and complex datasets in various fields from biology, where it can be used to infer gene dynamics from single-cell RNA-Seq, to astronomy, where it can be used to explore complex structures in the distribution of galaxies.Comment: 32 pages, 14 figure