Multimodal deep learning for point cloud panoptic segmentation of railway environments

The demand for transportation asset digitalisation has significantly increased over the years. For this purpose, mobile mapping systems (MMSs) are among the most popular technologies that allow capturing high precision three-dimensional point clouds of the infrastructure. In this paper, a multimodal deep learning methodology is presented for panoptic segmentation of the railway infrastructure. The methodology takes advantage of image rasterisation of the point clouds to perform a rough segmentation and discard more than 80% of points that are not relevant to the infrastructure. With this approach, the computational requirements for processing the remaining point cloud are highly reduced, allowing the process of dense point clouds in short periods of time. A 90 km-long railway scenario was used for training and testing. The proposed methodology is two times faster than the current state-of-the-art for the same point cloud density, and pole-like object segmentation metrics are improved.Fundación BBVAAgencia Estatal de Investigación | Ref. PID2019-108816RB-I00Ministerio de Universidades | Ref. FPU20/01024Universidade de Vigo/CISU

Santiago urban dataset SUD: Combination of Handheld and Mobile Laser Scanning point clouds

Santiago Urban Dataset SUD is a real dataset that combines Mobile Laser Scanning (MLS) and Handheld Mobile Laser Scanning (HMLS) point clouds. The data is composed by 2 km of streets, sited in Santiago de Compostela (Spain). Point clouds undergo a manual labelling process supported by both heuristic and Deep Learning methods, resulting in the classification of eight specific classes: road, sidewalk, curb, buildings, vehicles, vegetation, poles, and others. Three PointNet++ models were trained; the first one using MLS point clouds, the second one with HMLS point clouds and the third one with both H&MLS point clouds. In order to ascertain the quality and efficacy of each Deep Learning model, various metrics were employed, including confusion matrices, precision, recall, F1-score, and IoU. The results are consistent with other state-of-the-art works and indicate that SUD is valid for comparing point cloud semantic segmentation works. Furthermore, the survey's extensive coverage and the limited occlusions indicate the potential utility of SUD in urban mobility research.Agencia Estatal de Investigación | Ref. PID2019-105221RB-C43Xunta de Galicia | Ref. ED481B-2019-061Xunta de Galicia | Ref. ED431C 2020/01Universidade de Vigo/CISU

Point cloud semantic segmentation of complex railway environments using deep learning

Financiado para publicación en acceso aberto: Universidade de Vigo/CISUGSafety of transportation networks is of utmost importance for our society. With the emergency of digitalization, the railway sector is accelerating the automation in inventory and inspection procedures. Mobile mapping systems allow capturing three-dimensional point clouds of the infrastructure in short periods of time. In this paper, a deep learning methodology for semantic segmentation of railway infrastructures is presented. The methodology segments both linear and punctual elements from railway infrastructure, and it is tested in four scenarios: i) 90 km-long railway; ii) 2 km-long low-quality point clouds; iii) 400 m-long high-quality point clouds; iv) 1.4 km-long railway recoded with aerial mapping system. The longest one is used for training and testing, obtaining mean accuracy greater than 90%. The other scenarios are used only for testing, and qualitative results are discussed, proving that the method can be applied to new scenarios that significantly differ in terms of data quality and resolution.Agencia Estatal de Investigación | Ref. PID2019-108816RB-I00Ministerio de Universidades | Ref. FPU20/0102

Automatic point cloud semantic segmentation of complex railway environments

The growing development of data digitalisation methods has increased their demand and applications in the transportation infrastructure field. Currently, mobile mapping systems (MMSs) are one of the most popular technologies for the acquisition of infrastructure data, with three-dimensional (3D) point clouds as their main product. In this work, a heuristic-based workflow for semantic segmentation of complex railway environments is presented, in which their most relevant elements are classified, namely, rails, masts, wiring, droppers, traffic lights, and signals. This method takes advantage of existing methodologies in the field for point cloud processing and segmentation, taking into account the geometry and spatial context of each classified element in the railway environment. This method is applied to a 90-kilometre-long railway lane and validated against a manual reference on random sections of the case study data. The results are presented and discussed at the object level, differentiating the type of the element. The indicators F1 scores obtained for each element are superior to 85%, being higher than 99% in rails, the most significant element of the infrastructure. These metrics showcase the quality of the algorithm, which proves that this method is efficient for the classification of long and variable railway sections, and for the assisted labelling of point cloud data for future applications based on training supervised learning models.Agencia Estatal de Investigación | Ref. RTI2018-095893-B-C21Agencia Estatal de Investigación | Ref. FJC2018-035550-