Road Information Extraction from Mobile LiDAR Point Clouds using Deep Neural Networks

Urban roads, as one of the essential transportation infrastructures, provide considerable motivations for rapid urban sprawl and bring notable economic and social benefits. Accurate and efficient extraction of road information plays a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Mobile laser scanning (MLS) systems have been widely used for many transportation-related studies and applications in road inventory, including road object detection, pavement inspection, road marking segmentation and classification, and road boundary extraction, benefiting from their large-scale data coverage, high surveying flexibility, high measurement accuracy, and reduced weather sensitivity. Road information from MLS point clouds is significant for road infrastructure planning and maintenance, and have an important impact on transportation-related policymaking, driving behaviour regulation, and traffic efficiency enhancement. Compared to the existing threshold-based and rule-based road information extraction methods, deep learning methods have demonstrated superior performance in 3D road object segmentation and classification tasks. However, three main challenges remain that impede deep learning methods for precisely and robustly extracting road information from MLS point clouds. (1) Point clouds obtained from MLS systems are always in large-volume and irregular formats, which has presented significant challenges for managing and processing such massive unstructured points. (2) Variations in point density and intensity are inevitable because of the profiling scanning mechanism of MLS systems. (3) Due to occlusions and the limited scanning range of onboard sensors, some road objects are incomplete, which considerably degrades the performance of threshold-based methods to extract road information. To deal with these challenges, this doctoral thesis proposes several deep neural networks that encode inherent point cloud features and extract road information. These novel deep learning models have been tested by several datasets to deliver robust and accurate road information extraction results compared to state-of-the-art deep learning methods in complex urban environments. First, an end-to-end feature extraction framework for 3D point cloud segmentation is proposed using dynamic point-wise convolutional operations at multiple scales. This framework is less sensitive to data distribution and computational power. Second, a capsule-based deep learning framework to extract and classify road markings is developed to update road information and support HD maps. It demonstrates the practical application of combining capsule networks with hierarchical feature encodings of georeferenced feature images. Third, a novel deep learning framework for road boundary completion is developed using MLS point clouds and satellite imagery, based on the U-shaped network and the conditional deep convolutional generative adversarial network (c-DCGAN). Empirical evidence obtained from experiments compared with state-of-the-art methods demonstrates the superior performance of the proposed models in road object semantic segmentation, road marking extraction and classification, and road boundary completion tasks

A Comprehensive Review on Computer Vision Analysis of Aerial Data

With the emergence of new technologies in the field of airborne platforms and imaging sensors, aerial data analysis is becoming very popular, capitalizing on its advantages over land data. This paper presents a comprehensive review of the computer vision tasks within the domain of aerial data analysis. While addressing fundamental aspects such as object detection and tracking, the primary focus is on pivotal tasks like change detection, object segmentation, and scene-level analysis. The paper provides the comparison of various hyper parameters employed across diverse architectures and tasks. A substantial section is dedicated to an in-depth discussion on libraries, their categorization, and their relevance to different domain expertise. The paper encompasses aerial datasets, the architectural nuances adopted, and the evaluation metrics associated with all the tasks in aerial data analysis. Applications of computer vision tasks in aerial data across different domains are explored, with case studies providing further insights. The paper thoroughly examines the challenges inherent in aerial data analysis, offering practical solutions. Additionally, unresolved issues of significance are identified, paving the way for future research directions in the field of aerial data analysis.Comment: 112 page

Cloud Simulation Based Bridge Damage Identification Enhanced by Computer Vision and Augmented Reality

Bedeutung im Kontext von Schäden erweitert. Es wird eine systematische Methodik für die Schadensidentifikation entwickelt, die die Datenerfassung, Schadensbewertung und Schadensdatenverwaltung umfasst. Für die Erfassung von sichtbaren Schadensdaten wird die semantische Segmentierung verwendet. Zahlreiche Strategien und innovative Convolutional Neural Networks wurden entwickelt, um herkömmliche Netzwerke im Kontext der semantischen Segmentierung zu verbessern. Allerdings wurde ein umfassender Vergleich dieser Netzwerke selten durchgeführt. Für zwei Strategien, der Attention Mechanismen und der Generative Adversarial Networks, wird eine vergleichende Studie durchgeführt, um die semantische Segmentierung zu verbessern. Basierend auf dem U-Net werden neuartige Verteilungstypen für beide Strategien mit verschiedenen Diskriminatoren entwickelt und verglichen. Die am besten abschneidenden Netzwerke werden dann einem Validierungsprozess unterzogen, und auch die kombinierten Effekte der beiden Strategien werden vertieft untersucht. Die Cloud Simulation wird zur numerischen Bewertung von Brückenschäden und der Identifikation von verdeckten Schäden angewendet. Zwei Ansätze, nämlich der Single Variation Approach (SVA) und der Dual Variation Approach (DVA), werden vorgestellt. Beide Ansätze werden auf unterschiedliche Szenarien angewendet, um den Einfluss verschiedener Lastfälle und Überwachungspunkte zu studieren. Der effektivere DVA-Ansatz wird in einem Prototyp implementiert, der Funktionalitäten wie Datenkonvertierung, Visualisierung, Generierung von Modellvariationen und Ergebnisanalyse umfasst. Zur Validierung wird eine Stahlbetonbrücke analysiert. Qualitative und quantitative Bewertungen für die Schadensrehabilitation werden in eine Wissensbasis integriert, die automatische Vorschläge für die praktische Schadensrehabilitation für die infizierten Schäden liefert. Augmented Reality wird zur Verbesserung des Visualisierungsergebnisses bei der Schadensinformationsverwaltung für die Vor-Ort-Inspektion und die Rehabilitationsinformationen eingesetzt und in einer Baustellenumgebung validiert. Den Abschluss bildet eine Marketingperspektive der neugewonnenen Ergebnisse.The entire research work is dedicated to reinterpreting the concept of system identification by exploring its connotation and expanding its denotation in the context of damage. The work focuses on bridges as representative structures and proposes a systematic methodology for damage identification, encompassing damage data acquisition, damage assessment, and damage data management.  Semantic segmentation is employed for viewable damage data identification. Numerous strategies and innovative convolutional neural networks have been developed to enhance traditional networks in the context of semantic segmentation. However, a comprehensive comparison of these networks has been rarely conducted. Two strategies, namely attention mechanisms and generative adversarial networks, are examined in order to enhance semantic segmentation. Based on the U-net, novel distribution types of attention mechanisms and generative adversarial networks with different discriminators are compared in a lightweight test. The best performed networks are then implemented in the validation process, and in addition the combined effects of the attention mechanism and discriminator are investigated. Cloud simulation is applied for quantitative evaluation and identification of non-viewable damage. Two approaches, namely the Single Variation Approach (SVA) and the Dual Variation Approach (DVA), are introduced and applied to different scenarios to account for various load cases and monitoring points as variables. A prototype is developed to implement the more effective DVA approach, incorporating functionalities such as data conversion, visualization, model variation generation and result analysis. A monitored concrete bridge is employed for validation of the assessment of the effectiveness and reliability of the method. Qualitative and quantitative assessments are incorporated into a knowledge base for damage rehabilitation, which automatically provide practical suggestion for the specific identified damage.  Augmented reality is utilized to enhance the visualization experience for on-site inspection providing rehabilitation information and a prototype in a construction setting. The conclusion presents a marketing perspective on the findings

Spatial-Temporal Data Mining for Ocean Science: Data, Methodologies, and Opportunities

With the increasing amount of spatial-temporal~(ST) ocean data, numerous spatial-temporal data mining (STDM) studies have been conducted to address various oceanic issues, e.g., climate forecasting and disaster warning. Compared with typical ST data (e.g., traffic data), ST ocean data is more complicated with some unique characteristics, e.g., diverse regionality and high sparsity. These characteristics make it difficult to design and train STDM models. Unfortunately, an overview of these studies is still missing, hindering computer scientists to identify the research issues in ocean while discouraging researchers in ocean science from applying advanced STDM techniques. To remedy this situation, we provide a comprehensive survey to summarize existing STDM studies in ocean. Concretely, we first summarize the widely-used ST ocean datasets and identify their unique characteristics. Then, typical ST ocean data quality enhancement techniques are discussed. Next, we classify existing STDM studies for ocean into four types of tasks, i.e., prediction, event detection, pattern mining, and anomaly detection, and elaborate the techniques for these tasks. Finally, promising research opportunities are highlighted. This survey will help scientists from the fields of both computer science and ocean science have a better understanding of the fundamental concepts, key techniques, and open challenges of STDM in ocean

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Networks, Communication, and Computing Vol. 2

Networks, communications, and computing have become ubiquitous and inseparable parts of everyday life. This book is based on a Special Issue of the Algorithms journal, and it is devoted to the exploration of the many-faceted relationship of networks, communications, and computing. The included papers explore the current state-of-the-art research in these areas, with a particular interest in the interactions among the fields