A Deep Learning-Based Hybrid Approach to Detect Fastener Defects in Real-Time

A fastener is an important component used to fix the rail in railways. Defects in this component cause the rail and ballast to remain unstable. If the defective fasteners are not replaced in time, it is inevitable that the train will derail, and serious accidents will occur. Therefore, this component should be inspected periodically. Conventional image processing-based control systems are affected by noise and different lighting conditions in the real environment. In this study, it is aimed to determine the defects of fasteners with a deep learning-based hybrid approach. The YOLOv4-Tiny method is used for fastener detection and localization. This method gives accurate results, especially for the detection of small objects. After the fastener position is determined, a new lightweight convolutional neural network model is used for defect classification. The proposed convolutional neural network for classification has a small network structure because it uses depth-wise and pointwise convolution layers. When the experimental results are compared with other known transfer learning methods, better results were obtained in terms of training/test time and accuracy

Rail Robot for Rail Track Inspection

Railway transportation requires constant inspections and immediate maintenance to ensure public safety. Traditional manual inspections are not only time consuming, but also expensive. In addition, the accuracy of defect detection is also subjected to human expertise and efficiency at the time of inspection. Computing and Robotics offer automated IoT based solutions where robots could be deployed on rail-tracks and hard to reach areas, and controlled from control rooms to provide faster and low-cost inspection. In this thesis, a novel automated system based on robotics and visual inspection is proposed. The system provides local image processing while inspecting and cloud storage of information that consist of images of the defected railway tracks only. The proposed system utilizes state of the art Machine Learning system and applies it on the images obtained from the tracks in order to classify them as normal or suspicious. Such locations are then marked and more careful inspection can be performed by a dedicated operator with very few locations to inspect (as opposed to the full track)

Importance and applications of robotic and autonomous systems (RAS) in railway maintenance sector: a review

Maintenance, which is critical for safe, reliable, quality, and cost-effective service, plays a dominant role in the railway industry. Therefore, this paper examines the importance and applications of Robotic and Autonomous Systems (RAS) in railway maintenance. More than 70 research publications, which are either in practice or under investigation describing RAS developments in the railway maintenance, are analysed. It has been found that the majority of RAS developed are for rolling-stock maintenance, followed by railway track maintenance. Further, it has been found that there is growing interest and demand for robotics and autonomous systems in the railway maintenance sector, which is largely due to the increased competition, rapid expansion and ever-increasing expense

Comparison of deep learning and analytic image processing methods for autonomous inspection of railway bolts and clips

In this work, different methods are proposed and compared for autonomous inspection of railway bolts and clips. A prototype of an autonomous data acquisition system was developed to automatically obtain information of the state of the railway track using LiDAR and camera sensors. This system was employed in a testing railway track installed in the facilities of the University of Vigo to obtain the images used in this work. Then, the images were further processed using analytic image segmentation algorithms as well as a neural network to detect the bolts and clips. Once these elements are detected, their relative position is computed to evaluate if there is any missing component. Finally, the orientation of the clips is computed to ensure that all the bolts are correctly placed. Four different methods were implemented, and their performance was evaluated using the segmentations provided by the analytical methods and the neural network.Ministerio de Universidades | Ref. FPU21/01176Ministerio de Ciencia e Innovación | Ref. PLEC2021-007940Recovery, Transformation and Resilience Plan of the European Union – NextGenerationEU (University of Vigo) | Ref. 58550

A Survey on Audio-Video based Defect Detection through Deep Learning in Railway Maintenance

Within Artificial Intelligence, Deep Learning (DL) represents a paradigm that has been showing unprecedented performance in image and audio processing by supporting or even replacing humans in defect and anomaly detection. The Railway sector is expected to benefit from DL applications, especially in predictive maintenance applications, where smart audio and video sensors can be leveraged yet kept distinct from safety-critical functions. Such separation is crucial, as it allows for improving system dependability with no impact on its safety certification. This is further supported by the development of DL in other transportation domains, such as automotive and avionics, opening for knowledge transfer opportunities and highlighting the potential of such a paradigm in railways. In order to summarize the recent state-of-the-art while inquiring about future opportunities, this paper reviews DL approaches for the analysis of data generated by acoustic and visual sensors in railway maintenance applications that have been published until August 31st, 2021. In this paper, the current state of the research is investigated and evaluated using a structured and systematic method, in order to highlight promising approaches and successful applications, as well as to identify available datasets, current limitations, open issues, challenges, and recommendations about future research directions

Anomaly Detection in Noisy Images

Finding rare events in multidimensional data is an important detection problem that has applications in many fields, such as risk estimation in insurance industry, finance, flood prediction, medical diagnosis, quality assurance, security, or safety in transportation. The occurrence of such anomalies is so infrequent that there is usually not enough training data to learn an accurate statistical model of the anomaly class. In some cases, such events may have never been observed, so the only information that is available is a set of normal samples and an assumed pairwise similarity function. Such metric may only be known up to a certain number of unspecified parameters, which would either need to be learned from training data, or fixed by a domain expert. Sometimes, the anomalous condition may be formulated algebraically, such as a measure exceeding a predefined threshold, but nuisance variables may complicate the estimation of such a measure. Change detection methods used in time series analysis are not easily extendable to the multidimensional case, where discontinuities are not localized to a single point. On the other hand, in higher dimensions, data exhibits more complex interdependencies, and there is redundancy that could be exploited to adaptively model the normal data. In the first part of this dissertation, we review the theoretical framework for anomaly detection in images and previous anomaly detection work done in the context of crack detection and detection of anomalous components in railway tracks. In the second part, we propose new anomaly detection algorithms. The fact that curvilinear discontinuities in images are sparse with respect to the frame of shearlets, allows us to pose this anomaly detection problem as basis pursuit optimization. Therefore, we pose the problem of detecting curvilinear anomalies in noisy textured images as a blind source separation problem under sparsity constraints, and propose an iterative shrinkage algorithm to solve it. Taking advantage of the parallel nature of this algorithm, we describe how this method can be accelerated using graphical processing units (GPU). Then, we propose a new method for finding defective components on railway tracks using cameras mounted on a train. We describe how to extract features and use a combination of classifiers to solve this problem. Then, we scale anomaly detection to bigger datasets with complex interdependencies. We show that the anomaly detection problem naturally fits in the multitask learning framework. The first task consists of learning a compact representation of the good samples, while the second task consists of learning the anomaly detector. Using deep convolutional neural networks, we show that it is possible to train a deep model with a limited number of anomalous examples. In sequential detection problems, the presence of time-variant nuisance parameters affect the detection performance. In the last part of this dissertation, we present a method for adaptively estimating the threshold of sequential detectors using Extreme Value Theory on a Bayesian framework. Finally, conclusions on the results obtained are provided, followed by a discussion of possible future work

Multi-Scale Hierarchical Conditional Random Field for Railway Electrification Scene Classification Using Mobile Laser Scanning Data

With the recent rapid development of high-speed railway in many countries, precise inspection for railway electrification systems has become more significant to ensure safe railway operation. However, this time-consuming manual inspection is not satisfactory for the high-demanding inspection task, thus a safe, fast and automatic inspection method is required. With LiDAR (Light Detection and Ranging) data becoming more available, the accurate railway electrification scene understanding using LiDAR data becomes feasible towards automatic 3D precise inspection. This thesis presents a supervised learning method to classify railway electrification objects from Mobile Laser Scanning (MLS) data. First, a multi-range Conditional Random Field (CRF), which characterizes not only labeling homogeneity at a short range, but also the layout compatibility between different objects at a middle range in the probabilistic graphical model is implemented and tested. Then, this multi-range CRF model will be extended and improved into a hierarchical CRF model to consider multi-scale layout compatibility at full range. The proposed method is evaluated on a dataset collected in Korea with complex railway electrification systems environment. The experiment shows the effectiveness of proposed model

Machine learning approach to thermite weld defects detection and classification.

Masters Degree. University of KwaZulu- Natal, Durban.The defects formed during the thermite welding process between two sections of rails require the welded joints to be inspected for quality purpose. The commonly used non-destructive method for inspection is Radiography testing. However, the detection and classification of various defects from the generated radiography imagesremains a costly, lengthy and subjective process as it is purely conducted manually by trained experts. It has been shown that most rail breaks occur due to a crack that initiated from the weld joint defect that was not detected. To meet the requirements of the modern technologies, the development of an automated detection and classification model is significantly demanded by the railway industry. This work presents a method based on image processing and machine learning techniques to automatically detect and classify welding defects. Radiography images are first enhanced using the Contrast Limited Adaptive Histogram Equalisation method; thereafter, the Chan-Vese Active Contour Model is applied to the enhanced images to segment and extract the weld joint as the Region of Interest from the image background. A comparative investigation between the Local Binary Patterns descriptor and the Bag of Visual Words approach with Speeded Up Robust Features descriptor was carried out for extracting features in the weld joint images. The effectiveness of the aforementioned feature extractors was evaluated using the Support Vector Machines, K-Nearest Neighbours and Naive Bayes classifiers. This study’s experimental results showed that the Bag of Visual Words approach when used with the Support Vector Machines classifier, achieves the best overall classification accuracy of 94.66%. The proposed method can be expanded in other industries where Radiography testing is used as the inspection tool