Multi-Scale Hierarchical Conditional Random Field for Railway Electrification Scene Classification Using Mobile Laser Scanning Data

With the recent rapid development of high-speed railway in many countries, precise inspection for railway electrification systems has become more significant to ensure safe railway operation. However, this time-consuming manual inspection is not satisfactory for the high-demanding inspection task, thus a safe, fast and automatic inspection method is required. With LiDAR (Light Detection and Ranging) data becoming more available, the accurate railway electrification scene understanding using LiDAR data becomes feasible towards automatic 3D precise inspection. This thesis presents a supervised learning method to classify railway electrification objects from Mobile Laser Scanning (MLS) data. First, a multi-range Conditional Random Field (CRF), which characterizes not only labeling homogeneity at a short range, but also the layout compatibility between different objects at a middle range in the probabilistic graphical model is implemented and tested. Then, this multi-range CRF model will be extended and improved into a hierarchical CRF model to consider multi-scale layout compatibility at full range. The proposed method is evaluated on a dataset collected in Korea with complex railway electrification systems environment. The experiment shows the effectiveness of proposed model

Autonomous Point Cloud Segmentation for Power Lines Inspection in Smart Grid

LiDAR is currently one of the most utilized sensors to effectively monitor the status of power lines and facilitate the inspection of remote power distribution networks and related infrastructures. To ensure the safe operation of the smart grid, various remote data acquisition strategies, such as Airborne Laser Scanning (ALS), Mobile Laser Scanning (MLS), and Terrestrial Laser Scanning (TSL) have been leveraged to allow continuous monitoring of regional power networks, which are typically surrounded by dense vegetation. In this article, an unsupervised Machine Learning (ML) framework is proposed, to detect, extract and analyze the characteristics of power lines of both high and low voltage, as well as the surrounding vegetation in a Power Line Corridor (PLC) solely from LiDAR data. Initially, the proposed approach eliminates the ground points from higher elevation points based on statistical analysis that applies density criteria and histogram thresholding. After denoising and transforming of the remaining candidate points by applying Principle Component Analysis (PCA) and Kd-tree, power line segmentation is achieved by utilizing a two-stage DBSCAN clustering to identify each power line individually. Finally, all high elevation points in the PLC are identified based on their distance to the newly segmented power lines. Conducted experiments illustrate that the proposed framework is an agnostic method that can efficiently detect the power lines and perform PLC-based hazard analysis.Comment: Accepted in the 22nd World Congress of the International Federation of Automatic Control [IFAC WC 2023

Vegetation Detection and Classification for Power Line Monitoring

Electrical network maintenance inspections must be regularly executed, to provide a continuous distribution of electricity. In forested countries, the electrical network is mostly located within the forest. For this reason, during these inspections, it is also necessary to assure that vegetation growing close to the power line does not potentially endanger it, provoking forest fires or power outages. Several remote sensing techniques have been studied in the last years to replace the labor-intensive and costly traditional approaches, be it field based or airborne surveillance. Besides the previously mentioned disadvantages, these approaches are also prone to error, since they are dependent of a human operator’s interpretation. In recent years, Unmanned Aerial Vehicle (UAV) platform applicability for this purpose has been under debate, due to its flexibility and potential for customisation, as well as the fact it can fly close to the power lines. The present study proposes a vegetation management and power line monitoring method, using a UAV platform. This method starts with the collection of point cloud data in a forest environment composed of power line structures and vegetation growing close to it. Following this process, multiple steps are taken, including: detection of objects in the working environment; classification of said objects into their respective class labels using a feature-based classifier, either vegetation or power line structures; optimisation of the classification results using point cloud filtering or segmentation algorithms. The method is tested using both synthetic and real data of forested areas containing power line structures. The Overall Accuracy of the classification process is about 87% and 97-99% for synthetic and real data, respectively. After the optimisation process, these values were refined to 92% for synthetic data and nearly 100% for real data. A detailed comparison and discussion of results is presented, providing the most important evaluation metrics and a visual representations of the attained results.Manutenções regulares da rede elétrica devem ser realizadas de forma a assegurar uma distribuição contínua de eletricidade. Em países com elevada densidade florestal, a rede elétrica encontra-se localizada maioritariamente no interior das florestas. Por isso, durante estas inspeções, é necessário assegurar também que a vegetação próxima da rede elétrica não a coloca em risco, provocando incêndios ou falhas elétricas. Diversas técnicas de deteção remota foram estudadas nos últimos anos para substituir as tradicionais abordagens dispendiosas com mão-de-obra intensiva, sejam elas através de vigilância terrestre ou aérea. Além das desvantagens mencionadas anteriormente, estas abordagens estão também sujeitas a erros, pois estão dependentes da interpretação de um operador humano. Recentemente, a aplicabilidade de plataformas com Unmanned Aerial Vehicles (UAV) tem sido debatida, devido à sua flexibilidade e potencial personalização, assim como o facto de conseguirem voar mais próximas das linhas elétricas. O presente estudo propõe um método para a gestão da vegetação e monitorização da rede elétrica, utilizando uma plataforma UAV. Este método começa pela recolha de dados point cloud num ambiente florestal composto por estruturas da rede elétrica e vegetação em crescimento próximo da mesma. Em seguida,múltiplos passos são seguidos, incluindo: deteção de objetos no ambiente; classificação destes objetos com as respetivas etiquetas de classe através de um classificador baseado em features, vegetação ou estruturas da rede elétrica; otimização dos resultados da classificação utilizando algoritmos de filtragem ou segmentação de point cloud. Este método é testado usando dados sintéticos e reais de áreas florestais com estruturas elétricas. A exatidão do processo de classificação é cerca de 87% e 97-99% para os dados sintéticos e reais, respetivamente. Após o processo de otimização, estes valores aumentam para 92% para os dados sintéticos e cerca de 100% para os dados reais. Uma comparação e discussão de resultados é apresentada, fornecendo as métricas de avaliação mais importantes e uma representação visual dos resultados obtidos

A Survey on Audio-Video based Defect Detection through Deep Learning in Railway Maintenance

Within Artificial Intelligence, Deep Learning (DL) represents a paradigm that has been showing unprecedented performance in image and audio processing by supporting or even replacing humans in defect and anomaly detection. The Railway sector is expected to benefit from DL applications, especially in predictive maintenance applications, where smart audio and video sensors can be leveraged yet kept distinct from safety-critical functions. Such separation is crucial, as it allows for improving system dependability with no impact on its safety certification. This is further supported by the development of DL in other transportation domains, such as automotive and avionics, opening for knowledge transfer opportunities and highlighting the potential of such a paradigm in railways. In order to summarize the recent state-of-the-art while inquiring about future opportunities, this paper reviews DL approaches for the analysis of data generated by acoustic and visual sensors in railway maintenance applications that have been published until August 31st, 2021. In this paper, the current state of the research is investigated and evaluated using a structured and systematic method, in order to highlight promising approaches and successful applications, as well as to identify available datasets, current limitations, open issues, challenges, and recommendations about future research directions

Development of UAV-Based Rail Track Geometry Irregularity Monitoring and Measuring Platform Empowered by Artificial Intelligence

Rail tracks need to be consistently monitored and inspected for problems associated with rust, deformation, and cracks that, at their worst, can cause catastrophic train derailments. Many non-destructive testing approaches have been explored and extensively utilized to help inspect rails’ health, but most of them require intensive human power and/or heavy sensor systems (e.g. total stations, manual/car-mounted trolly, etc.) that are not efficient or convenient to cover a long range of rails and may interfere with the normal operation of trains.In light of the rapid development of unmanned aerial systems/vehicles (UAS’s/UAVs) and high definition photographic and optical distance measuring sensors, this paper proposes a novel UAV-based rail track irregularity monitoring and measuring platform that can remotely inspect the geometry irregularity of tracks at various angles and cover a long distance by only a few personnel. By mounting a light distance and range (LiDAR) scanning sensor and a data acquisition system on the UAV, we can continuously collect 3D point cloud data (PCD) frames that reflect the surfaces of tracks, ground, and other objects. Data points in these PCD frames are manually annotated into two classes: rail tracks and background. Then, annotated PCD frames are pre-processed and fed to train a state-of-the-art machine-learning-based 3D point cloud semantic segmentation network, RandLA-Net, to assign each point into one of the two aforementioned classes, so that point clusters that represent rail tracks can be extracted. The trained model can be deployed for real-time distinction between rails and background. Then, principal component analysis (PCA) and multiple regressions are conducted to identify the top and inner surface of the rails. In the end, various geometry measurement of rails, such as gauge, cross level, etc. can be performed to inspect any irregularities. The geometry measurement obtained by the proposed UAV-LiDAR-based framework is compared against standard official value of each geometry. The evaluation results have confirmed the similar or the more advanced performance of the proposed platform with more terrain flexibilities

3D Reconstruction of 138 KV Power-lines from Airborne LiDAR Data

Due to infrequent and imprecise maintenance inspection in power-line corridors, accidents can be caused by interferences, for instance, surrounding trees. Transmission power-line inspection conventionally relies on the participation of ground personnel and airborne camera to patrol power-lines, and is limited by intensive labour, and difficult working conditions and management. Airborne light detection and ranging (LiDAR) has proven a powerful tool to overcome these limitations to enable more efficient inspection. Active airborne LiDAR systems directly capture the 3D information of power infrastructure and surrounding objects. This study aims at building a semi-automatic 3D reconstruction workflow for power-lines extracted from airborne LiDAR data of 138 kV transmission line corridors (500 m by 340 m) in Nanaimo, BC, Canada. The proposed workflow consists of three components: detection, extraction, and fitting. The power-lines are automatically detected with regular geometric shape using a set of algorithms, including density-based filtering, Hough transform and concatenating algorithm. The complete power-lines are then extracted using a rectangular searching technique. Finally, the 3D power-lines are reconstructed through fitting by a hyperbolic cosine function and least-squares fitting. A case study is carried out to evaluate the proposed workflow for hazard tree detection in the corridor. The results obtained demonstrate that power-lines can be reconstructed in 3D, which are useful in detection of hazard trees to support power-line corridor management

Automatic Crack Segmentation for UAV-assisted Bridge Inspection

Bridges are a critical piece of infrastructure in the network of road and rail transport system. Many of the bridges in Norway (in Europe) are at the end of their lifespan, therefore regular inspection and maintenance are critical to ensure the safety of their operations. However, the traditional inspection procedures and resources required are so time consuming and costly that there exists a significant maintenance backlog. The central thrust of this paper is to demonstrate the significant benefits of adapting a Unmanned Aerial Vehicle (UAV)-assisted inspection to reduce the time and costs of bridge inspection and established the research needs associated with the processing of the (big) data produced by such autonomous technologies. In this regard, a methodology is proposed for analysing the bridge damage that comprises three key stages, (i) data collection and model training, where one performs experiments and trials to perfect drone flights for inspection using case study bridges to inform and provide necessary (big) data for the second key stage, (ii) 3D construction, where one built 3D models that offer a permanent record of element geometry for each bridge asset, which could be used for navigation and control purposes, (iii) damage identification and analysis, where deep learning-based data analytics and modelling are applied for processing and analysing UAV image data and to perform bridge damage performance assessment. The proposed methodology is exemplified via UAV-assisted inspection of Skodsberg bridge, a 140 m prestressed concrete bridge, in the Viken county in eastern Norway.publishedVersio

Machine Learning Tool for Transmission Capacity Forecasting of Overhead Lines based on Distributed Weather Data

Die Erhöhung des Anteils intermittierender erneuerbarer Energiequellen im elektrischen Energiesystem ist eine Herausforderung für die Netzbetreiber. Ein Beispiel ist die Zunahme der Nord-Süd Übertragung von Windenergie in Deutschland, die zu einer Erhöhung der Engpässe in den Freileitungen führt und sich direkt in den Stromkosten der Endverbraucher niederschlägt. Neben dem Ausbau neuer Freileitungen ist ein witterungsabhängiger Freileitungsbetrieb eine Lösung, um die aktuelle Auslastung des Systems zu verbessern. Aus der Analyse in einer Probeleitung in Deutschland wurde gezeigt, dass einen Zuwachs von ca. 28% der Stromtragfähigkeit eine Reduzierung der Kosten für Engpassmaßnahmen um ca. 55% bedeuten kann. Dieser Vorteil kann nur vom Netzbetreiber wahrgenommen werden, wenn eine Belastbarkeitsprognose für die Stromerzeugunsgplanung der konventionellen Kraftwerke zur Verfügung steht. Das in dieser Dissertation vorgestellte System prognostiziert die Belastbarkeit von Freileitungen für 48 Stunden, mit einer Verbesserung der Prognosegenauigkeit im Vergleich zum Stand-der-Technik von 6,13% in Durchschnitt. Der Ansatz passt die meteorologischen Vorhersagen an die lokale Wettersituation entlang der Leitung an. Diese Anpassungen sind aufgrund von Veränderungen der Topographie entlang der Leitungstrasse und Windschatten der umliegenden Bäume notwendig, da durch die meteorologischen Modelle diese nicht beschrieben werden können. Außerdem ist das in dieser Dissertation entwickelte Modell in der Lage die Genauigkeitsabweichungen der Wettervorhersage zwischen Tag und Nacht abzugleichen, was vorteilhaft für die Strombelastbarkeitsprognose ist. Die Zuverlässigkeit und deswegen auch die Effizienz des Stromerzeugungsplans für den nächsten 48 Stunden wurde um 10% gegenüber dem Stand der Technik erhöht. Außerdem wurde in Rahmen dieser Arbeit ein Verfahren für die Positionierung der Wetterstationen entwickelt, um die wichtigsten Stellen entlang der Leitung abzudecken und gleichzeitig die Anzahl der Wetterstationen zu minimieren. Wird ein verteiltes Sensornetzwerk in ganz Deutschland umgesetzt, wird die Einsparung von Redispatchingkosten eine Kapitalrendite von ungefähr drei Jahren bedeuten. Die Durchführung einer transienten Analyse ist im entwickelten System ebenfalls möglich, um Engpassfälle für einige Minuten zu lösen, ohne die maximale Leitertemperatur zu erreichen. Dieses Dokument versucht, die Vorteile der Freileitungsmonitoringssysteme zu verdeutlichen und stellt eine Lösung zur Unterstützung eines flexiblen elektrischen Netzes vor, die für eine erfolgreiche Energiewende erforderlich ist

A review of infrared thermography applications for ice detection and mitigation

Ice accretion on various onshore and offshore infrastructures imparts hazardous effects sometimes beyond repair, which may be life-threatening. Therefore, it has become necessary to look for ways to detect and mitigate ice. Some ice mitigation techniques have been tested or in use in aviation and railway sectors, however, their applicability to other sectors/systems is still in the research phase. To make such systems autonomous, ice protection systems need to be accompanied by reliable ice detection systems, which include electronic, mechatronics, mechanical, and optical techniques. Comparing the benefits and limitations of all available methodologies, Infrared Thermography (IRT) appears to be one of the useful, non-destructive, and emerging techniques as it offers wide area monitoring instead of just point-based ice monitoring. This paper reviews the applications of IRT in the field of icing on various subject areas to provide valuable insights into the existing development of an intelligent and autonomous ice mitigation system for general applications

A review on the prospects of mobile manipulators for smart maintenance of railway track

Inspection and repair interventions play vital roles in the asset management of railways. Autonomous mobile manipulators possess considerable potential to replace humans in many hazardous railway track maintenance tasks with high efficiency. This paper investigates the prospects of the use of mobile manipulators in track maintenance tasks. The current state of railway track inspection and repair technologies is initially reviewed, revealing that very few mobile manipulators are in the railways. Of note, the technologies are analytically scrutinized to ascertain advantages, unique capabilities, and potential use in the deployment of mobile manipulators for inspection and repair tasks across various industries. Most mobile manipulators in maintenance use ground robots, while other applications use aerial, underwater, or space robots. Power transmission lines, the nuclear industry, and space are the most extensive application areas. Clearly, the railways infrastructure managers can benefit from the adaptation of best practices from these diversified designs and their broad deployment, leading to enhanced human safety and optimized asset digitalization. A case study is presented to show the potential use of mobile manipulators in railway track maintenance tasks. Moreover, the benefits of the mobile manipulator are discussed based on previous research. Finally, challenges and requirements are reviewed to provide insights into future research