Simulation of multi-platform LiDAR for assessing total leaf area in tree crowns

LiDAR (Light Detection and Ranging) technology has been increasingly implemented to assess the biophysical attributes of forest canopies. However, LiDAR-based estimation of tree biophysical attributes remains difficult mainly due to the occlusion of vegetative elements in multi-layered tree crowns. In this study, we developed a new algorithm along with a multiple-scan methodology to analyse the impact of occlusion on LiDAR-based estimates of tree leaf area. We reconstructed five virtual tree models using a computer graphic-based approach based on in situ measurements from multiple tree crowns, for which the position, size, orientation and area of all leaves were measured. Multi-platform LiDAR simulations were performed on these 3D tree models through a point-line intersection algorithm. An approach based on the Delaunay triangulation algorithm with automatic adaptive threshold selection was proposed to construct the scanned leaf surface from the simulated discrete LiDAR point clouds. In addition, the leaf area covered by laser beams in each layer was assessed in combination with the ratio and number of the scanned points. Quantitative comparisons of LiDAR scanning for the occlusion effects among various scanning approaches, including fixed-position scanning, multiple terrestrial LiDAR scanning and airborne-terrestrial LiDAR cross-scanning, were assessed on different target trees. The results showed that one simulated terrestrial LiDAR scan alongside the model tree captured only 25–38% of the leaf area of the tree crown. When scanned data were acquired from three simulated terrestrial LiDAR scans around one tree, the accuracy of the leaf area recovery rate reached 60–73% depending on the leaf area index, tree crown volume and leaf area density. When a supplementary airborne LiDAR scanning was included, occlusion was reduced and the leaf area recovery rate increased to 72–90%. Our study provides an approach for the measurement of total leaf area in tree crowns from simulated multi-platform LiDAR data and enables a quantitative assessment of occlusion metrics for various tree crown attributes under different scanning strategies

Semi-automated Generation of Road Transition Lines Using Mobile Laser Scanning Data

Recent advances in autonomous vehicles (AVs) are exponential. Prominent car manufacturers, academic institutions, and corresponding governmental departments around the world are taking active roles in the AV industry. Although the attempts to integrate AV technology into smart roads and smart cities have been in the works for more than half a century, the High Definition Road Maps (HDRMs) that assists full self-driving autonomous vehicles did not yet exist. Mobile Laser Scanning (MLS) has enormous potential in the construction of HDRMs due to its flexibility in collecting wide coverage of street scenes and 3D information on scanned targets. However, without proper and efficient execution, it is difficult to generate HDRMs from MLS point clouds. This study recognizes the research gaps and difficulties in generating transition lines (the paths that pass through a road intersection) in road intersections from MLS point clouds. The proposed method contains three modules: road surface detection, lane marking extraction, and transition line generation. Firstly, the points covering road surface are extracted using the voxel- based upward-growing and the improved region growing. Then, lane markings are extracted and identified according to the multi-thresholding and the geometric filtering. Finally, transition lines are generated through a combination of the lane node structure generation algorithm and the cubic Catmull-Rom spline algorithm. The experimental results demonstrate that transition lines can be successfully generated for both T- and cross-intersections with promising accuracy. In the validation of lane marking extraction using the manually interpreted lane marking points, the method can achieve 90.80% precision, 92.07% recall, and 91.43% F1-score, respectively. The success rate of transition line generation is 96.5%. Furthermore, the Buffer-overlay-statistics (BOS) method validates that the proposed method can generate lane centerlines and transition lines within 20 cm-level localization accuracy from MLS point clouds. In addition, a comparative study is conducted to indicate the better performance of the proposed road marking extraction method than that of three other existing methods. In conclusion, this study makes a considerable contribution to the research on generating transition lines for HDRMs, which further contributes to the research of AVs

Toward knowledge-based automatic 3D spatial topological modeling from LiDAR point clouds for urban areas

Le traitement d'un très grand nombre de données LiDAR demeure très coûteux et nécessite des approches de modélisation 3D automatisée. De plus, les nuages de points incomplets causés par l'occlusion et la densité ainsi que les incertitudes liées au traitement des données LiDAR compliquent la création automatique de modèles 3D enrichis sémantiquement. Ce travail de recherche vise à développer de nouvelles solutions pour la création automatique de modèles géométriques 3D complets avec des étiquettes sémantiques à partir de nuages de points incomplets. Un cadre intégrant la connaissance des objets à la modélisation 3D est proposé pour améliorer la complétude des modèles géométriques 3D en utilisant un raisonnement qualitatif basé sur les informations sémantiques des objets et de leurs composants, leurs relations géométriques et spatiales. De plus, nous visons à tirer parti de la connaissance qualitative des objets en reconnaissance automatique des objets et à la création de modèles géométriques 3D complets à partir de nuages de points incomplets. Pour atteindre cet objectif, plusieurs solutions sont proposées pour la segmentation automatique, l'identification des relations topologiques entre les composants de l'objet, la reconnaissance des caractéristiques et la création de modèles géométriques 3D complets. (1) Des solutions d'apprentissage automatique ont été proposées pour la segmentation sémantique automatique et la segmentation de type CAO afin de segmenter des objets aux structures complexes. (2) Nous avons proposé un algorithme pour identifier efficacement les relations topologiques entre les composants d'objet extraits des nuages de points afin d'assembler un modèle de Représentation Frontière. (3) L'intégration des connaissances sur les objets et la reconnaissance des caractéristiques a été développée pour inférer automatiquement les étiquettes sémantiques des objets et de leurs composants. Afin de traiter les informations incertitudes, une solution de raisonnement automatique incertain, basée sur des règles représentant la connaissance, a été développée pour reconnaître les composants du bâtiment à partir d'informations incertaines extraites des nuages de points. (4) Une méthode heuristique pour la création de modèles géométriques 3D complets a été conçue en utilisant les connaissances relatives aux bâtiments, les informations géométriques et topologiques des composants du bâtiment et les informations sémantiques obtenues à partir de la reconnaissance des caractéristiques. Enfin, le cadre proposé pour améliorer la modélisation 3D automatique à partir de nuages de points de zones urbaines a été validé par une étude de cas visant à créer un modèle de bâtiment 3D complet. L'expérimentation démontre que l'intégration des connaissances dans les étapes de la modélisation 3D est efficace pour créer un modèle de construction complet à partir de nuages de points incomplets.The processing of a very large set of LiDAR data is very costly and necessitates automatic 3D modeling approaches. In addition, incomplete point clouds caused by occlusion and uneven density and the uncertainties in the processing of LiDAR data make it difficult to automatic creation of semantically enriched 3D models. This research work aims at developing new solutions for the automatic creation of complete 3D geometric models with semantic labels from incomplete point clouds. A framework integrating knowledge about objects in urban scenes into 3D modeling is proposed for improving the completeness of 3D geometric models using qualitative reasoning based on semantic information of objects and their components, their geometric and spatial relations. Moreover, we aim at taking advantage of the qualitative knowledge of objects in automatic feature recognition and further in the creation of complete 3D geometric models from incomplete point clouds. To achieve this goal, several algorithms are proposed for automatic segmentation, the identification of the topological relations between object components, feature recognition and the creation of complete 3D geometric models. (1) Machine learning solutions have been proposed for automatic semantic segmentation and CAD-like segmentation to segment objects with complex structures. (2) We proposed an algorithm to efficiently identify topological relationships between object components extracted from point clouds to assemble a Boundary Representation model. (3) The integration of object knowledge and feature recognition has been developed to automatically obtain semantic labels of objects and their components. In order to deal with uncertain information, a rule-based automatic uncertain reasoning solution was developed to recognize building components from uncertain information extracted from point clouds. (4) A heuristic method for creating complete 3D geometric models was designed using building knowledge, geometric and topological relations of building components, and semantic information obtained from feature recognition. Finally, the proposed framework for improving automatic 3D modeling from point clouds of urban areas has been validated by a case study aimed at creating a complete 3D building model. Experiments demonstrate that the integration of knowledge into the steps of 3D modeling is effective in creating a complete building model from incomplete point clouds

地図作成効率化のためのMMS点群の構造化と道路地物抽出

電気通信大学202

Road Information Extraction from Mobile LiDAR Point Clouds using Deep Neural Networks

Urban roads, as one of the essential transportation infrastructures, provide considerable motivations for rapid urban sprawl and bring notable economic and social benefits. Accurate and efficient extraction of road information plays a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Mobile laser scanning (MLS) systems have been widely used for many transportation-related studies and applications in road inventory, including road object detection, pavement inspection, road marking segmentation and classification, and road boundary extraction, benefiting from their large-scale data coverage, high surveying flexibility, high measurement accuracy, and reduced weather sensitivity. Road information from MLS point clouds is significant for road infrastructure planning and maintenance, and have an important impact on transportation-related policymaking, driving behaviour regulation, and traffic efficiency enhancement. Compared to the existing threshold-based and rule-based road information extraction methods, deep learning methods have demonstrated superior performance in 3D road object segmentation and classification tasks. However, three main challenges remain that impede deep learning methods for precisely and robustly extracting road information from MLS point clouds. (1) Point clouds obtained from MLS systems are always in large-volume and irregular formats, which has presented significant challenges for managing and processing such massive unstructured points. (2) Variations in point density and intensity are inevitable because of the profiling scanning mechanism of MLS systems. (3) Due to occlusions and the limited scanning range of onboard sensors, some road objects are incomplete, which considerably degrades the performance of threshold-based methods to extract road information. To deal with these challenges, this doctoral thesis proposes several deep neural networks that encode inherent point cloud features and extract road information. These novel deep learning models have been tested by several datasets to deliver robust and accurate road information extraction results compared to state-of-the-art deep learning methods in complex urban environments. First, an end-to-end feature extraction framework for 3D point cloud segmentation is proposed using dynamic point-wise convolutional operations at multiple scales. This framework is less sensitive to data distribution and computational power. Second, a capsule-based deep learning framework to extract and classify road markings is developed to update road information and support HD maps. It demonstrates the practical application of combining capsule networks with hierarchical feature encodings of georeferenced feature images. Third, a novel deep learning framework for road boundary completion is developed using MLS point clouds and satellite imagery, based on the U-shaped network and the conditional deep convolutional generative adversarial network (c-DCGAN). Empirical evidence obtained from experiments compared with state-of-the-art methods demonstrates the superior performance of the proposed models in road object semantic segmentation, road marking extraction and classification, and road boundary completion tasks

Métodos de classificação de nuvens de pontos recolhidas por sistemas LiDAR móveis, para a geração de modelos digitais de terreno, a grandes escalas

A tecnologia LiDAR (LIght Detection And Ranging) tem-se revelado nos últimos anos como sendo uma técnica bastante eficaz na aquisição de dados geoespaciais. A instalação destes sistemas em plataformas sobre veículos terrestres permite uma elevada rapidez na recolha de nuvens de pontos, por vezes limitada apenas pela velocidade do próprio veículo. A obtenção de dados de base (pontos de cota e linhas de quebra), para a geração de MDT (Modelos Digitais do Terreno) a grandes escalas é normalmente um processo bastante moroso e consequentemente dispendioso. A utilização das nuvens de pontos recolhidas por sistemas LiDAR moveis terrestre surge assim, naturalmente, como uma possível solução eficiente para a obtenção desse tipo de dados. No entanto, as nuvens de pontos recolhidas por estes sistemas, são não-seletivas, sendo necessário efetuar a classificação e segmentação desses dados. A classificação dos pontos da nuvem que representam a superfície do terreno e a sua segmentação de forma a identificar e restituir as linhas de quebra é um desafio em aberto que continua a despertar o interesse dos investigadores. Ao longo deste trabalho pretende-se contribuir para a resposta a esse desafio, propondo e testando diversos métodos inovadores para a classificação e extração de pontos das nuvens recolhidas por sistemas LiDAR móveis terrestres, com o objetivo de geração de MDT a grandes escalas. Ao contrário da maioria dos algoritmos existentes na literatura, em que apenas são utilizadas as coordenadas tridimensionais dos pontos, a maioria dos algoritmos aqui propostos tiraram partido dos princípios de funcionamento dos sistemas e dos dados associados a cada um dos pontos da nuvem. Os algoritmos propostos, têm ainda em consideração a eficiência na obtenção dos dados mínimos e suficientes para a representação da forma do terreno a uma determinada escala. Sendo mantido o paradigma estabelecido pela maioria dos utilizadores e produtores de informação geográfica, na utilização de pontos de cota e linhas de quebra, para a geração de MDT a grandes escalas. É ainda proposto um método para a extração de linhas tridimensionais a partir de nuvens de pontos obtidas ao longo de infraestruturas ferroviárias. Finalmente, tendo em conta que, os perfis transversais a grandes escalas representam o atual paradigma para a modelação do terreno como base para projetos de execução de vias lineares, nomeadamente, rodoviárias e ferroviárias. É apresentado um estudo comparativo de várias estratégias propostas para o agrupamento dos pontos das nuvens com vista à criação desses perfis transversais.LiDAR (LIght Detection And Ranging) technology, revealed in recent years, to be a very effective technique in the acquisition of geospatial information. The installation of these systems on land vehicles allows a high information collection speed of point clouds, often limited only by the speed of the vehicle itself. Obtaining basic information (height points and break lines) for digital terrain models generation, is usually a very time-consuming task and consequently an expensive process. The use of point clouds collected by terrestrial mobile LiDAR systems, naturally emerges as a possible efficient solution to obtain this type of information. However, the point clouds collected by these systems are non-selective, and it is necessary to classify and segment this information. The classification of cloud points representing the surface of the terrain and the segmentation of information in order to identify and restore their break lines is an open challenge that continues to stimulate the researcher’s interest. Throughout this work, its intended to contribute to answer to this challenge, by proposing and testing several innovative methods, for the classification and extraction of information from point clouds collected by terrestrial mobile LiDAR systems, for digital terrain models creation Most of the proposed algorithms take advantage of the principles of system operation and the information stored for each cloud point. Instead of most existing algorithms in the literature, where only the points three-dimensional coordinates are used. The proposed algorithms also consider the efficiency in obtain, the minimum and necessary information, to represent the terrain shape at a given scale. Being maintained the paradigm established by the majority of users and producers of geographic information, in the use of height points and break lines, for digital terrain models representation. A method is also presented for the particular case of the use of these systems on railway lines. Finally, considering, that the use of cross sections represents the terrain modelling, current paradigm, for linear infrastructures projects, namely roads and railways. A comparative study of several strategies proposed for cloud points group, is presented, in way to create these cross sections