SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving

In this paper, we introduce a deep encoder-decoder network, named SalsaNet, for efficient semantic segmentation of 3D LiDAR point clouds. SalsaNet segments the road, i.e. drivable free-space, and vehicles in the scene by employing the Bird-Eye-View (BEV) image projection of the point cloud. To overcome the lack of annotated point cloud data, in particular for the road segments, we introduce an auto-labeling process which transfers automatically generated labels from the camera to LiDAR. We also explore the role of imagelike projection of LiDAR data in semantic segmentation by comparing BEV with spherical-front-view projection and show that SalsaNet is projection-agnostic. We perform quantitative and qualitative evaluations on the KITTI dataset, which demonstrate that the proposed SalsaNet outperforms other state-of-the-art semantic segmentation networks in terms of accuracy and computation time. Our code and data are publicly available at https://gitlab.com/aksoyeren/salsanet.git

Improving 3D Semantic Segmentation withTwin-Representation Networks

The growing importance of 3d scene understanding and interpretation is inher-ently connected to the rise of autonomous driving and robotics. Semanticsegmentation of 3d point clouds is a key enabler for this task, providing geo-metric information enhanced with semantics. To use Convolutional NeuralNetworks, a proper representation of the point clouds must be chosen. Variousrepresentations have been proposed, with different advantages and disadvantages.In this work, we present a twin-representation architecture, which is composedof a 3d point-based and a 2d range image branch, to efficiently extract and refinepoint-wise features, supported by strong context information. Additionally, afeature propagation strategy is proposed to connect both branches. The approachis evaluated on the challenging SemanticKITTI dataset [2] and considerablyoutperforms the baseline overall as well as for every individual class. Especiallythe predictions for distant points are significantly improved

Multimodal 3D Semantic Segmentation

Understanding and interpreting a scene is a key task of environment perception for autonomous driving, which is why autonomous vehicles are equipped with a wide range of different sensors. Semantic Segmentation of sensor data provides valuable information for this task and is often seen as key enabler. In this report, we’re presenting a deep learning approach for 3D semantic segmentation of lidar point clouds. The proposed architecture uses the lidar’s native range view and additionally exploits camera features to increase accuracy and robustness. Lidar and camera feature maps of different scales are fused iteratively inside the network architecture. We evaluate our deep fusion approach on a large benchmark dataset and demonstrate its benefits compared to other state-of-the-art approaches, which rely only on lidar

Potential applications of deep learning in automatic rock joint trace mapping in a rock mass

In blasted rock slopes and underground openings, rock joints are visible in different forms. Rock joints are often exposed as planes confining rock blocks and visible as traces on a well-blasted, smooth rock mass surface. A realistic rock joint model should include both visual forms of joints in a rock mass: i.e., both joint traces and joint planes. Imaged-based 2D semantic segmentation using deep learning via the Convolutional Neural Network (CNN) has shown promising results in extracting joint traces in a rock mass. In 3D analysis, research studies using deep learning have demonstrated outperforming results in automatically extracting joint planes from an unstructured 3D point cloud compared to state-of-the-art methods. We discuss a pilot study using 3D true colour point cloud and their source and derived 2D images in this paper. In the study, we aim to implement and compare various CNN-based networks found in the literature for automatic extraction of joint traces from laser scanning and photogrammetry data. Extracted joint traces can then be clustered and connected to potential joint planes as joint objects in a discrete joint model. This can contribute to a more accurate estimation of rock joint persistence. The goal of the study is to compare the efficiency and accuracy between using 2D images and 3D point cloud as input data. Data are collected from two infrastructure projects with blasted rock slopes and tunnels in Norway.Potential applications of deep learning in automatic rock joint trace mapping in a rock masspublishedVersio