Spherical Transformer: Adapting Spherical Signal to CNNs

Convolutional neural networks (CNNs) have been widely used in various vision tasks, e.g. image classification, semantic segmentation, etc. Unfortunately, standard 2D CNNs are not well suited for spherical signals such as panorama images or spherical projections, as the sphere is an unstructured grid. In this paper, we present Spherical Transformer which can transform spherical signals into vectors that can be directly processed by standard CNNs such that many well-designed CNNs architectures can be reused across tasks and datasets by pretraining. To this end, the proposed method first uses locally structured sampling methods such as HEALPix to construct a transformer grid by using the information of spherical points and its adjacent points, and then transforms the spherical signals to the vectors through the grid. By building the Spherical Transformer module, we can use multiple CNN architectures directly. We evaluate our approach on the tasks of spherical MNIST recognition, 3D object classification and omnidirectional image semantic segmentation. For 3D object classification, we further propose a rendering-based projection method to improve the performance and a rotational-equivariant model to improve the anti-rotation ability. Experimental results on three tasks show that our approach achieves superior performance over state-of-the-art methods

3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review

Three-dimensional (3D) point cloud analysis has become one of the attractive subjects in realistic imaging and machine visions due to its simplicity, flexibility and powerful capacity of visualization. Actually, the representation of scenes and buildings using 3D shapes and formats leveraged many applications among which automatic driving, scenes and objects reconstruction, etc. Nevertheless, working with this emerging type of data has been a challenging task for objects representation, scenes recognition, segmentation, and reconstruction. In this regard, a significant effort has recently been devoted to developing novel strategies, using different techniques such as deep learning models. To that end, we present in this paper a comprehensive review of existing tasks on 3D point cloud: a well-defined taxonomy of existing techniques is performed based on the nature of the adopted algorithms, application scenarios, and main objectives. Various tasks performed on 3D point could data are investigated, including objects and scenes detection, recognition, segmentation and reconstruction. In addition, we introduce a list of used datasets, we discuss respective evaluation metrics and we compare the performance of existing solutions to better inform the state-of-the-art and identify their limitations and strengths. Lastly, we elaborate on current challenges facing the subject of technology and future trends attracting considerable interest, which could be a starting point for upcoming research studie

Potential applications of deep learning in automatic rock joint trace mapping in a rock mass

In blasted rock slopes and underground openings, rock joints are visible in different forms. Rock joints are often exposed as planes confining rock blocks and visible as traces on a well-blasted, smooth rock mass surface. A realistic rock joint model should include both visual forms of joints in a rock mass: i.e., both joint traces and joint planes. Imaged-based 2D semantic segmentation using deep learning via the Convolutional Neural Network (CNN) has shown promising results in extracting joint traces in a rock mass. In 3D analysis, research studies using deep learning have demonstrated outperforming results in automatically extracting joint planes from an unstructured 3D point cloud compared to state-of-the-art methods. We discuss a pilot study using 3D true colour point cloud and their source and derived 2D images in this paper. In the study, we aim to implement and compare various CNN-based networks found in the literature for automatic extraction of joint traces from laser scanning and photogrammetry data. Extracted joint traces can then be clustered and connected to potential joint planes as joint objects in a discrete joint model. This can contribute to a more accurate estimation of rock joint persistence. The goal of the study is to compare the efficiency and accuracy between using 2D images and 3D point cloud as input data. Data are collected from two infrastructure projects with blasted rock slopes and tunnels in Norway.Potential applications of deep learning in automatic rock joint trace mapping in a rock masspublishedVersio

Rotation Invariant Convolutions for 3D Point Clouds Deep Learning

Recent progresses in 3D deep learning has shown that it is possible to design special convolution operators to consume point cloud data. However, a typical drawback is that rotation invariance is often not guaranteed, resulting in networks being trained with data augmented with rotations. In this paper, we introduce a novel convolution operator for point clouds that achieves rotation invariance. Our core idea is to use low-level rotation invariant geometric features such as distances and angles to design a convolution operator for point cloud learning. The well-known point ordering problem is also addressed by a binning approach seamlessly built into the convolution. This convolution operator then serves as the basic building block of a neural network that is robust to point clouds under 6DoF transformations such as translation and rotation. Our experiment shows that our method performs with high accuracy in common scene understanding tasks such as object classification and segmentation. Compared to previous works, most importantly, our method is able to generalize and achieve consistent results across different scenarios in which training and testing can contain arbitrary rotations.Comment: International Conference on 3D Vision (3DV) 201