Gamification for road asset inspection from Mobile Mapping System data

Gamification techniques have been proven effective in various fields such as education and industry. In this paper, we introduce a novel approach that applies gamification techniques to the identification of road assets in Mobile Laser Scanning (MLS) data. Our method utilises three gamification techniques: avatar (vehicle), point cloud segmentation into levels, and scoring. We implemented these techniques in Unreal Engine and evaluated their performance using three real-world case studies. We also compared two ways of point cloud visualisation: mesh-based and point-based. Our results demonstrate that our gamification approach improves the handling and visualisation of point clouds when compared to other free software such as Cloud Compare. Specifically, the point-based visualisation method provides a more accurate representation of the road environment and the input point cloud and is easier to import into Unreal Engine. However, this method requires more computational resources for visualisation. On the other hand, level segmentation ensures a constant frame rate of 60 frames per second. Furthermore, our gamification approach enhances the experience of road asset identification, making it more enjoyable for the user. However, we acknowledge that the quality of the point cloud remains the primary factor affecting the accuracy of asset identification, regardless of the software used. Overall, our proposed gamification approach offers a promising solution for improving the identification of road assets in MLS data and has the potential to be applied to other fields beyond road asset identification.Xunta de Galicia | Ref. ED481B-2019-061Agencia Estatal de Investigación | Ref. PID2019-105221RB-C43Agencia Estatal de Investigación | Ref. RYC2021‐033560‐

Desenvolvemento de modelos de información de infraestructuras segundo estándares abertos e parametrización automática a partir de datos xeomáticos.

It seeks to develop procedures that allow generating information models of these structures, created from the relevant information of the point clouds obtained with these systems. For this purpose, the BIM standards for civil engineering structures, both currently available and those that will be published for the duration of the thesis, will be exploited and adopted. Information modeling techniques will be used in these standards, with the aim of obtaining a system that allows modeling the structures automatically. The models will also be made compatible with other methodologies designed for BIM, whose purpose is to take full advantage of the information available for management and maintenance tasks. Meeting these objectives, an automatic modeling system will be developed according to the BIM standards for transport infrastructures, suitable for automatic feeding from geomatic data and remote sensing, which is in turn integrable into management and maintenance systems for these types of structures of civil engineering.Esta tesis busca el desarrollo de metodologías para la exportación de la información geomática de infraestructuras de transporte, particularmente estructuras ferroviarias y carreteras, obtenida mediante tecnologías de mapeado móvil. Se busca desarrollar procedimientos que permitan generar modelos de información de estas estructuras, creados a partir de la información relevante de las nubes de puntos obtenidas con estos sistemas. Con este propósito, se explotarán y adoptarán los estándares BIM para estructuras de ingeniería civil, tanto los actualmente disponibles como aquellos que serán publicados durante la duración de la tesis. Se utilizarán técnicas de modelado de información en estos estándares, con objetivo de obtener un sistema que permita realizar un modelado de las estructuras de manera automática. Se llevará a cabo también la compatibilización los modelos con otras metodologías diseñadas para BIM, cuyo propósito es el aprovechamiento total de la información disponible para tareas de gestión y mantenimiento. Cumpliendo estos objetivos se desarrollará un sistema automático de modelado según los estándares BIM para infraestructuras de transporte, apto para su alimentación automática a partir de datos geomáticos y teledetección, el cual es a su vez integrable en sistemas de gestión y mantenimiento para este tipo de estructuras de ingeniería civil.Esta tese busca o desenvolvemento de metodoloxías para a exportación da información xeomática de infraestruturas de transporte, particularmente estruturas ferroviarias e estradas, obtida mediante tecnoloxías de mapeado móbil. A tese busca o desenvolvemento de procedementos que permitan xerar modelos de información destas estruturas, creados a partir da información relevante das nubes de puntos obtidas con estes sistemas. Con este propósito, se explotarán e adoptarán os estándares BIM para estruturas de enxeñería civil, tanto os actualmente dispoñibles como aqueles que serán publicados durante a duración da tese. Utilizaranse técnicas de modelado de información nestes estándares, con obxectivo de obter un sistema que permita realizar un modelado das estruturas de maneira automática. Levarase a cabo tamén a compatibilización dos modelos con outras metodoloxías diseñadas para BIM, cuxo propósito é o aproveitamento total da información dispoñible para tarefas de xestión e mantemento. Cumplindo estes obxectivos se desenvolverá un sistema automático de modelado segundo os estándares BIM para infraestruturas de transporte, apto para a súa alimentación automática a partir de datos xeomáticos e teledetección, o cal é a súa vez integrable en sistemas de xestión e mantemento para este tipo de estruturas de enxeñería civil

Multimodal deep learning for point cloud panoptic segmentation of railway environments

The demand for transportation asset digitalisation has significantly increased over the years. For this purpose, mobile mapping systems (MMSs) are among the most popular technologies that allow capturing high precision three-dimensional point clouds of the infrastructure. In this paper, a multimodal deep learning methodology is presented for panoptic segmentation of the railway infrastructure. The methodology takes advantage of image rasterisation of the point clouds to perform a rough segmentation and discard more than 80% of points that are not relevant to the infrastructure. With this approach, the computational requirements for processing the remaining point cloud are highly reduced, allowing the process of dense point clouds in short periods of time. A 90 km-long railway scenario was used for training and testing. The proposed methodology is two times faster than the current state-of-the-art for the same point cloud density, and pole-like object segmentation metrics are improved.Fundación BBVAAgencia Estatal de Investigación | Ref. PID2019-108816RB-I00Ministerio de Universidades | Ref. FPU20/01024Universidade de Vigo/CISU

Automatic manhole extraction from MMS data to update basemaps

Basemaps are the main resource used in urban planning, building and infrastructure asset management. Therefore, they must be accurate and up to date to better serve citizens, contractors, property owners and town planning departments. Traditionally, they have been updated by aerial photogrammetry, but this is not always possible and alternatives need to be sought. In such cases, a useful option for large scales is the mobile mapping system (MMS). However, automatic extraction from MMS point clouds is limited by the complexity of the urban environment. Therefore, the influence of the urban pattern is analysed in three zones with varied urban characteristics: areas with high buildings, open areas, and areas with a low level of urbanization. In these areas, the capture and automatic extraction of 3D urban elements is performed using commercial software, which is useful for some elements but not for manholes. The objective of this study is to establish a methodology for extracting manholes automatically and completing hidden buildings' corners, in order to update urban basemaps. Shape and intensity are the main detection parameters for manholes, whereas additional information from satellite image Quickbird is used to complete the buildings. The worst rate of detection for all the extracted urban elements was found in areas of high buildings. Finally, the article analyses the computing cost for manhole extraction, and the economic cost and time consume of the entire process, including the proposed methodolgy using an MMS point cloud and the traditional survey in this case.Peer ReviewedPostprint (updated version

Transport infrastructure management based on LiDAR synthetic data: a deep learning approach with a ROADSENSE simulator

In the realm of transportation system management, various remote sensing techniques have proven instrumental in enhancing safety, mobility, and overall resilience. Among these techniques, Light Detection and Ranging (LiDAR) has emerged as a prevalent method for object detection, facilitating the comprehensive monitoring of environmental and infrastructure assets in transportation environments. Currently, the application of Artificial Intelligence (AI)-based methods, particularly in the domain of semantic segmentation of 3D LiDAR point clouds by Deep Learning (DL) models, is a powerful method for supporting the management of both infrastructure and vegetation in road environments. In this context, there is a lack of open labeled datasets that are suitable for training Deep Neural Networks (DNNs) in transportation scenarios, so, to fill this gap, we introduce ROADSENSE (Road and Scenic Environment Simulation), an open-access 3D scene simulator that generates synthetic datasets with labeled point clouds. We assess its functionality by adapting and training a state-of-the-art DL-based semantic classifier, PointNet++, with synthetic data generated by both ROADSENSE and the well-known HELIOS++ (HEildelberg LiDAR Operations Simulator). To evaluate the resulting trained models, we apply both DNNs on real point clouds and demonstrate their effectiveness in both roadway and forest environments. While the differences are minor, the best mean intersection over union (MIoU) values for highway and national roads are over 77%, which are obtained with the DNN trained on HELIOS++ point clouds, and the best classification performance in forested areas is over 92%, which is obtained with the model trained on ROADSENSE point clouds. This work contributes information on a valuable tool for advancing DL applications in transportation scenarios, offering insights and solutions for improved road and roadside management.Agencia Estatal de Investigación | Ref. PID2022-140662OB-I00Agencia Estatal de Investigación | Ref. PCI2020-120705-2-I0

Técnicas de inteligencia artificial aplicadas a sistemas de detección y clasificación de señales de tráfico.

Esta tesis, presentada como conjunto de artículos de investigación, estudia y analiza soluciones para los sistemas de detección y clasificación de señales de tráfico que suponen un reto en aplicaciones de la actualidad, como son la seguridad y asistencia en carretera a conductores, los coches autónomos, el mantenimiento de señalización vertical, o el análisis de escenas de tráfico. Las señales de tráfico constituyen un activo fundamental dentro de la red decarreteras porque su objetivo es ser fácilmente perceptible por los peatones y conductores para advertirles y guiarlos tanto de día como de noche. El hecho de que las señales estén diseñadas para ser únicas y tener características distinguibles, como formas simples y colores uniformes, implica que su detección y reconocimiento sea un problema limitado. Sin embargo, el desarrollo de un sistema de reconocimiento de señales en tiempo real aún presenta desafíos debido a los tiempos de respuesta, los cuales son cruciales para tomar decisiones en el entorno, y la variabilidad que presentan las imágenes de escenas de tráfico, que pueden incluir imágenes a distintas escalas, puntos de vista complicados, oclusiones, y diferentes condiciones de luz. Cualquier sistema de detección y clasificación de señales de tráfico debe hacer frente a estos retos. En este trabajo, se presenta un sistema de clasificación de señales de tráfico basado en aprendizaje profundo (Deep Learning). Concretamente, los principales componentes de la red neuronal profunda (Deep Neural Network) propuesta, son capas convolucionales y redes de transformaciones espaciales (Spatial Transformer Networks). Dicha red es alimentada con imágenes RGB de señales de tráfico de distintos países como Alemania, Bélgica o España. En el caso de las señales de Alemania, que pertenecen al dataset denominado German Traffic Sign Recognition Benchmark (GTSRB), la arquitectura de red y los parámetros de optimización propuestos obtienen un 99.71% de precisión, mejorando tanto al sistema visual humano como a todos los resultados previos del estado del arte, siendo además más eficiente en términos de requisitos de memoria. En el momento de redactar esta tesis, nuestro método se encuentra en la primera posición de la clasificación a nivel mundial. Por otro lado, respecto a la problemática de la detección de señales de tráfico, se analizan varios sistemas de detección de objetos propuestos en el estado del arte, que son específicamente modificados y adaptados al dominio del problema que nos ocupa para aplicar la transferencia de conocimiento en redes neuronales (transfer learning). También se estudian múltiples parámetros de rendimiento para cada uno de los modelos de detección con el fin de ofrecer al lector cuál sería el mejor detector de señales teniendo en cuenta restricciones del entorno donde se desplegará la solución, como la precisión, el consumo de memoria o la velocidad de ejecución. Nuestro estudio muestra que el modelo Faster R-CNN Inception Resnet V2 obtiene la mejor precisión (95.77% mAP), mientras que R-FCN Resnet 101 alcanza el mejor equilibrio entre tiempo de ejecución (85.45 ms por imagen) y precisión (95.15% mAP)

Coarse-to-fine classification of road infrastructure elements from mobile point clouds using symmetric ensemble point network and euclidean cluster extraction

Classifying point clouds obtained from mobile laser scanning of road environments is a fundamental yet challenging problem for road asset management and unmanned vehicle navigation. Deep learning networks need no prior knowledge to classify multiple objects, but often generate a certain amount of false predictions. However, traditional clustering methods often involve leveraging a priori knowledge, but may lack generalisability compared to deep learning networks. This paper presents a classification method that coarsely classifies multiple objects of road infrastructure with a symmetric ensemble point (SEP) network and then refines the results with a Euclidean cluster extraction (ECE) algorithm. The SEP network applies a symmetric function to capture relevant structural features at different scales and select optimal sub-samples using an ensemble method. The ECE subsequently adjusts points that have been predicted incorrectly by the first step. The experimental results indicate that this method effectively extracts six types of road infrastructure elements: road surfaces, buildings, walls, traffic signs, trees and streetlights. The overall accuracy of the SEP-ECE method improves by 3.97% with respect to PointNet. The achieved average classification accuracy is approximately 99.74%, which is suitable for practical use in transportation network management

Comparing Mobile and Aerial Laser Scanner point cloud data sets for automating the detection and delimitation procedure of safety-critical near-road slopes

An inappropriately maintained road cut-slope is likely to fail, resulting in landslides or falling rocks that compromise road safety. Thus, road managers need to know the location of dangerous slopes along the road in order to prevent these events from happening. In this article, we compare two different approaches for conducting the digitization of the road environment and the automatic detection and delimitation of road slopes: Mobile Laser Scanners (MLS) and Aerial Laser Scanners (ALS). The point clouds obtained using the first kind of devices are dense, rich in detail and generated from a ground perspective; the second type of scanners produce less dense clouds from a zenithal perspective. We explore what is the effect of the point cloud density and scanner point of view over the slope detection procedure. Two road segments from the Spanish A55 and A52 highways were used as study zones, and a total of 28.61 km were analyzed. Better detection and delimitation results were achieved when using the ALS data and its corresponding algorithm. It was observed that the higher point density and detail of the MLS clouds were not an advantage for the slope detection task, and that measuring the road from a terrestrial perspective affected in a negative way during the detection process: the crest of the slopes often remained unmeasured, hidden behind vegetation or man-made elements, thus resulting in the slopes not being complete in the MLS clouds. Meanwhile, the whole slope structure is scanned when the scene is measured from an aerial perspective, henceforth obtaining better detection rates despite the relatively low resolution. The findings of this study provide valuable information in the field of road asset management, and help road managers make decisions when choosing what technology to use for the data gathering process.Agencia Estatal de Investigación | Ref. PID2022-140662OB-I00Universidade de Vigo/CISU