EXTRACTION OF ROAD MARKINGS FROM MLS DATA: A REVIEW

Nowadays, Mobile Laser Scanning (MLS) systems are more and more used to realize extended topographic surveys of roads. Most of them provide for each measured point an attribute corresponding to a return signal strength, so called intensity value. This value enables to easily understand uncolored MLS as it helps to differentiate materials based on their albedo. In a road context, this intensity information allows to distinguish, among others, the main subject of this paper, i.e. road markings. However, this task is challenging. Road marking detection from dense MLS point cloud is widely studied by the research community. It might concern road management and diagnosis, intelligent traffic systems, high-definition maps, location and navigation services. Dense MLS point clouds provided by surveyors are not processed online, they are thus not directly applicable to autonomous driving, but those dense and precise data can be for instance used for the generation of HD reference maps. This paper presents a review of the different processing chains published in the literature. It underlines their contributions and highlights their potential limitations. Finally, a discussion and some suggestions of improvement are given

Road Information Extraction from Mobile LiDAR Point Clouds using Deep Neural Networks

Urban roads, as one of the essential transportation infrastructures, provide considerable motivations for rapid urban sprawl and bring notable economic and social benefits. Accurate and efficient extraction of road information plays a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Mobile laser scanning (MLS) systems have been widely used for many transportation-related studies and applications in road inventory, including road object detection, pavement inspection, road marking segmentation and classification, and road boundary extraction, benefiting from their large-scale data coverage, high surveying flexibility, high measurement accuracy, and reduced weather sensitivity. Road information from MLS point clouds is significant for road infrastructure planning and maintenance, and have an important impact on transportation-related policymaking, driving behaviour regulation, and traffic efficiency enhancement. Compared to the existing threshold-based and rule-based road information extraction methods, deep learning methods have demonstrated superior performance in 3D road object segmentation and classification tasks. However, three main challenges remain that impede deep learning methods for precisely and robustly extracting road information from MLS point clouds. (1) Point clouds obtained from MLS systems are always in large-volume and irregular formats, which has presented significant challenges for managing and processing such massive unstructured points. (2) Variations in point density and intensity are inevitable because of the profiling scanning mechanism of MLS systems. (3) Due to occlusions and the limited scanning range of onboard sensors, some road objects are incomplete, which considerably degrades the performance of threshold-based methods to extract road information. To deal with these challenges, this doctoral thesis proposes several deep neural networks that encode inherent point cloud features and extract road information. These novel deep learning models have been tested by several datasets to deliver robust and accurate road information extraction results compared to state-of-the-art deep learning methods in complex urban environments. First, an end-to-end feature extraction framework for 3D point cloud segmentation is proposed using dynamic point-wise convolutional operations at multiple scales. This framework is less sensitive to data distribution and computational power. Second, a capsule-based deep learning framework to extract and classify road markings is developed to update road information and support HD maps. It demonstrates the practical application of combining capsule networks with hierarchical feature encodings of georeferenced feature images. Third, a novel deep learning framework for road boundary completion is developed using MLS point clouds and satellite imagery, based on the U-shaped network and the conditional deep convolutional generative adversarial network (c-DCGAN). Empirical evidence obtained from experiments compared with state-of-the-art methods demonstrates the superior performance of the proposed models in road object semantic segmentation, road marking extraction and classification, and road boundary completion tasks

An Efficient Point Cloud Processing Framework for Terrestrial and Mobile Lidar Data via Reconstructing the Scan Pattern Grid

Lidar (LIght Detection And Ranging) is a remote sensing technology using light in the form of a pulsed laser, which enables efficient, accurate, 3-D data acquisition of a scene. Depending on the mounting platform, lidar data acquisition can be categorized into Airborne Laser Scanning (ALS, or airborne lidar), Terrestrial Laser Scanning (TLS, or terrestrial lidar), and Mobile Laser Scanning (MLS, or mobile lidar). The lidar technique has been widely used for a plethora of applications including topographic mapping, bathymetric mapping, utility mapping, engineering surveying, agriculture, forestry, geology, architecture, industrial facilities, cultural heritage, asset management, construction, and so forth. However, efficiently processing the dense datasets produced by lidar still remains challenging given the large data volume. In addition, because of the scan pattern, range, view angle, and other factors, the point density for terrestrial and mobile lidar data can vary dramatically across the scene, which raises different challenges in developing robust processing methods compared with an ALS point cloud, which tends to be more evenly distributed. To overcome the challenges in processing TLS and MLS data, in this research, the point cloud is structured into a 2-D grid structure called the scan pattern grid, which represents the way that a scanner collects data. This dissertation, comprising four manuscripts, explores the possibilities and performance improvements of exploiting this scan pattern grid to process point cloud data. This first manuscript presents an efficient ground filtering method for TLS data via a Scanline Density Analysis. Ground filtering is a common procedure in lidar data processing, which separates the point cloud data into two classes: ground points and non-ground points. The proposed process first separates the ground candidates, density features, and unidentified points based on an analysis of point density within each scanline. Second, a region growth using the scan pattern clusters the ground candidates using the density features as boundaries and further refines the ground points. Both stages process and analyze the TLS data in each scan separately, exploiting the scan pattern grid for efficiency. The next two manuscripts develop a novel point cloud segmentation with an approach that links the scan pattern grids from multiple scans during the analysis. Point cloud segmentation groups points with similar attributes with respect to geometric, colormetric radiometric, and/or other information to help with object extraction and interpreting the point cloud. The proposed segmentation method only requires the basic geometric information and consists of two main steps. First, a novel feature extraction approach, NORmal VAriation ANAlysis (Norvana), eliminates some noise points and extracts edge points without requiring a general (and error prone) normal estimation at each point. Second, region growing groups the points on a smooth surface using the edge points as boundaries to obtain the segmentation result. Unlike TLS data that can be directly structured from a structured format (e.g., ASTM E57), Mobile lidar data is usually stored in an unorganized manner (e.g., ASPRS LAS). The final manuscript presents an efficient mobile lidar data processing framework including an approach to reconstruct the scanner trajectory such that an unorganized point cloud can be structured into the scan pattern grid based on the order of acquisition and revolutions of the scanner. Then the concept of Norvana for edge detection, normal estimation, feature extraction, and segmentation, is extended to be suitable for processing mobile lidar data and is named Mo norvana. Additionally, the proposed framework also introduces an efficient data visualization scheme exploiting the scan pattern grid. All of the proposed methods implement parallel processing to obtain a higher computational performance. The effectiveness, efficiency, robustness, and versatility are demonstrated both qualitatively and quantitatively by testing multiple terrestrial and mobile lidar datasets collected by different scanners with different spatial scales, resolutions, and scene types. The key contribution of this research is a generalized point cloud processing framework that can efficiently support a wide range of refinements, processes, and analysis for a variety of applications