A Survey on Audio-Video based Defect Detection through Deep Learning in Railway Maintenance

Within Artificial Intelligence, Deep Learning (DL) represents a paradigm that has been showing unprecedented performance in image and audio processing by supporting or even replacing humans in defect and anomaly detection. The Railway sector is expected to benefit from DL applications, especially in predictive maintenance applications, where smart audio and video sensors can be leveraged yet kept distinct from safety-critical functions. Such separation is crucial, as it allows for improving system dependability with no impact on its safety certification. This is further supported by the development of DL in other transportation domains, such as automotive and avionics, opening for knowledge transfer opportunities and highlighting the potential of such a paradigm in railways. In order to summarize the recent state-of-the-art while inquiring about future opportunities, this paper reviews DL approaches for the analysis of data generated by acoustic and visual sensors in railway maintenance applications that have been published until August 31st, 2021. In this paper, the current state of the research is investigated and evaluated using a structured and systematic method, in order to highlight promising approaches and successful applications, as well as to identify available datasets, current limitations, open issues, challenges, and recommendations about future research directions

Infrastructure Design, Signalling and Security in Railway

Railway transportation has become one of the main technological advances of our society. Since the first railway used to carry coal from a mine in Shropshire (England, 1600), a lot of efforts have been made to improve this transportation concept. One of its milestones was the invention and development of the steam locomotive, but commercial rail travels became practical two hundred years later. From these first attempts, railway infrastructures, signalling and security have evolved and become more complex than those performed in its earlier stages. This book will provide readers a comprehensive technical guide, covering these topics and presenting a brief overview of selected railway systems in the world. The objective of the book is to serve as a valuable reference for students, educators, scientists, faculty members, researchers, and engineers

Modelling safety critical systems with ageing components, with application to underground railway risk and hazards

In this thesis methodologies for modelling risk on ageing systems are developed. In the first stages of the thesis, two systems on an underground railway are used to demonstrate the modelling approach. In the latter stages of this thesis the modelling approach is expanded further, presenting a method for optimisation of a phased maintenance strategy, an inclusion of uncertainty in model outputs and an approach to model size reduction. Initially, a Petri net modelling approach is proposed to predict the derailment caused by component failures on a Switch and Crossing (S&C). A holistic methodology is adopted such that components of the system are divided into subsets of interconnected modules at a system level. Degradation within each module is idealized through a sequence of discrete states of wear until final failure occurs. Monte Carlo analysis is used to numerically evaluate the resulting Petri net. Through this methodology, different maintenance strategies, such as partial replacement, complete replacement, and opportunistic maintenance, are tested, to evaluate their influence on the final risk of derailment and predicted system state over time. This work includes a more in-depth modelling approach for S&C than that available in literature. This improves on the state of the art by removing assumptions of perfect maintenance and inspection. In addition, the approach includes modelling of dependencies between components, that are introduced through shared maintenance actions. Secondly, a Petri net modelling approach is applied to an automatic fire protection system to assess the probability of system failure, throughout the system life. Components are modelled with individual Petri nets, which are connected by a phased asset management strategy. The model is solved numerically via Monte Carlo simulation and component failure probabilities are combined using logic developed through Fault Tree analysis. For each time period, this application gives the probability of detection, deluge and alarm system failure, along with the number of maintenance actions, system tests and false system activations. The key contributions from this work include a detailed model for the interlocking fire protection systems and the application of a phased asset management strategy. This phased strategy allows the modelling of different maintenance approaches that are applied at different times depending on the system age. This approach demonstrates an increased functionality in comparison to modelling approaches currently available for fire protection systems, In addition, the modelling approach is extended further towards an optimal risk-based asset management decision making tool. The model for the fire protection systems is used as an application and is extended to give a measure of risk and whole-life cost. This extended model forms the basis of a two-stage optimisation approach within the framework of a phased asset management strategy. A Simulated Annealing algorithm is combined with a Genetic Algorithm to reduce system level risk and whole-life cost. A method for the incorporation of uncertainty in predicted model outputs is also presented. Novel aspects within this work include: the development of the optimisation approach for a phased asset management strategy and the developed algorithm for quantifying model output uncertainty given uncertain input parameters. The optimization of a phased system shows improvements on current model optimisation examples as it allows different strategies to be applied at different phases of the system lifecycle. It allows these phases to be determined in an automatic manner. The inclusion of uncertainty estimates on model outputs improves current Petri net modelling approaches, where uncertainty in input parameters is not included, as it allows decisions based on modelling outcomes to be more fully informed. Finally, a method is presented that can be applied to large system level Petri net models to produce equivalent model at a reduced computational cost. The method consists of generating a reduced Petri net which approximates the behaviour of its larger counterpart with a shorter simulation time. Parameters in this reduced structure are updated following a combined Approximate Bayesian Computation and Subset Simulation framework. Novel contributions from this work include: the proposed reduction approach, a method for using this reduction approach to improve model optimisation efficiency and the exploration of the reduction approach to justify model structure selection. These improve on approaches for model reduction available in literature, which are commonly rule based and so less flexible. In addition, model choice is typically user defined without quantifiable evidence for the suitability of the selected model structure

Novel Approaches for Structural Health Monitoring

The thirty-plus years of progress in the field of structural health monitoring (SHM) have left a paramount impact on our everyday lives. Be it for the monitoring of fixed- and rotary-wing aircrafts, for the preservation of the cultural and architectural heritage, or for the predictive maintenance of long-span bridges or wind farms, SHM has shaped the framework of many engineering fields. Given the current state of quantitative and principled methodologies, it is nowadays possible to rapidly and consistently evaluate the structural safety of industrial machines, modern concrete buildings, historical masonry complexes, etc., to test their capability and to serve their intended purpose. However, old unsolved problematics as well as new challenges exist. Furthermore, unprecedented conditions, such as stricter safety requirements and ageing civil infrastructure, pose new challenges for confrontation. Therefore, this Special Issue gathers the main contributions of academics and practitioners in civil, aerospace, and mechanical engineering to provide a common ground for structural health monitoring in dealing with old and new aspects of this ever-growing research field

Modelling safety critical systems with ageing components, with application to underground railway risk and hazards

In this thesis methodologies for modelling risk on ageing systems are developed. In the first stages of the thesis, two systems on an underground railway are used to demonstrate the modelling approach. In the latter stages of this thesis the modelling approach is expanded further, presenting a method for optimisation of a phased maintenance strategy, an inclusion of uncertainty in model outputs and an approach to model size reduction. Initially, a Petri net modelling approach is proposed to predict the derailment caused by component failures on a Switch and Crossing (S&C). A holistic methodology is adopted such that components of the system are divided into subsets of interconnected modules at a system level. Degradation within each module is idealized through a sequence of discrete states of wear until final failure occurs. Monte Carlo analysis is used to numerically evaluate the resulting Petri net. Through this methodology, different maintenance strategies, such as partial replacement, complete replacement, and opportunistic maintenance, are tested, to evaluate their influence on the final risk of derailment and predicted system state over time. This work includes a more in-depth modelling approach for S&C than that available in literature. This improves on the state of the art by removing assumptions of perfect maintenance and inspection. In addition, the approach includes modelling of dependencies between components, that are introduced through shared maintenance actions. Secondly, a Petri net modelling approach is applied to an automatic fire protection system to assess the probability of system failure, throughout the system life. Components are modelled with individual Petri nets, which are connected by a phased asset management strategy. The model is solved numerically via Monte Carlo simulation and component failure probabilities are combined using logic developed through Fault Tree analysis. For each time period, this application gives the probability of detection, deluge and alarm system failure, along with the number of maintenance actions, system tests and false system activations. The key contributions from this work include a detailed model for the interlocking fire protection systems and the application of a phased asset management strategy. This phased strategy allows the modelling of different maintenance approaches that are applied at different times depending on the system age. This approach demonstrates an increased functionality in comparison to modelling approaches currently available for fire protection systems, In addition, the modelling approach is extended further towards an optimal risk-based asset management decision making tool. The model for the fire protection systems is used as an application and is extended to give a measure of risk and whole-life cost. This extended model forms the basis of a two-stage optimisation approach within the framework of a phased asset management strategy. A Simulated Annealing algorithm is combined with a Genetic Algorithm to reduce system level risk and whole-life cost. A method for the incorporation of uncertainty in predicted model outputs is also presented. Novel aspects within this work include: the development of the optimisation approach for a phased asset management strategy and the developed algorithm for quantifying model output uncertainty given uncertain input parameters. The optimization of a phased system shows improvements on current model optimisation examples as it allows different strategies to be applied at different phases of the system lifecycle. It allows these phases to be determined in an automatic manner. The inclusion of uncertainty estimates on model outputs improves current Petri net modelling approaches, where uncertainty in input parameters is not included, as it allows decisions based on modelling outcomes to be more fully informed. Finally, a method is presented that can be applied to large system level Petri net models to produce equivalent model at a reduced computational cost. The method consists of generating a reduced Petri net which approximates the behaviour of its larger counterpart with a shorter simulation time. Parameters in this reduced structure are updated following a combined Approximate Bayesian Computation and Subset Simulation framework. Novel contributions from this work include: the proposed reduction approach, a method for using this reduction approach to improve model optimisation efficiency and the exploration of the reduction approach to justify model structure selection. These improve on approaches for model reduction available in literature, which are commonly rule based and so less flexible. In addition, model choice is typically user defined without quantifiable evidence for the suitability of the selected model structure

Micro-CT Scanning in the Investigation of Squat Defects in Rail Steel

This thesis contains the results from an investigation into Squats, a discrete rail steel defect. Micro-computed tomography (micro-CT) scanning was used to scan the entire structure of four out of five defects removed from railway track. The fifth was not scanned as it shelled in track. The difficulty turning these raw X-ray images into a segmented 3D model were overcome by developing a new technique. Isolating separate regions of the scans created areas where voxel value variation was at a minimum (i.e. the histogram became one narrow peak rather than multiple broad peaks). This allowed the automatic crack segregation module to work fairly well, and then enhanced using the region growing modules within the software. These scan results were then verified to be accurate using metallographic sample preparation, optical and electron microscopy. Micro CT scanning was performed on a custom 450KeV scanner, allowing the capture of the first entire Squat crack network morphology. Full defect imaging allowed the different defects to be compared to each other, highlighting differences and similarities. The defects came from metro, mixed and high-speed railways and one was found within an aluminothermic weld. The verification process and investigations of the scan volumes yielded further information about the defect’s origins. These origins were used to determine that, for the five defects investigated, there were four different causes. The two that shared a cause were from the same track section. Based on the causes, the defects were identified as a Stud, two Grinding Induced Squats (GIS), a Squat (caused by the legacy issue of MnS inclusions) and possibly a Squat or Stud in a slightly contaminated weld. None of the defects were considered to be a classic Squat, which is caused by Rolling Contact Fatigue (RCF), because there were other factors in their initiation. One of the defects contained a transverse defect, which is a crack that grows down through the railhead and can break the rail. This transverse defect was ~9mm deep into the rail when it was removed, meaning it would not have returned to the surface to shell. A high-resolution volume of the transverse defect region was created and its origin gives an important insight into the potential causes of rail-breaking defects. The origin of this transverse defect was a cluster of debris-filled voids that had formed due to the corrosion and cyclic loading (fretting) of a crack branch. These voids aligned with a deep grinding mark on the surface of the rail, which acted as a stress raiser. Because corrosion is a factor in this transverse defect case, the age of a rail and its environment are factors for defect development as well as traffic volume, given the correlation of corrosion with time. Results of this work highlight both the importance of a good surface finish and the diversity of causes found within the term “Squat”. Thus the identification of the Stud variant may be the beginning of a more comprehensive group of Squat type defects being established. This refining of the category could lead to fruitful big data analyses of the Squat type defect occurrences. The CT volumes of the defects created in this work can easily be stored for comparison in future investigations. The virtual nature of the volumes allows the sharing of defect information more readily than physical and sectioned defects, which deteriorate with time and require physical storage and transport

Novel methods of object recognition and fault detection applied to non-destructive testing of rail’s surface during production

A series of rail image inspection algorithms have been developed for Tata Steels Scunthorpe rail production line. The following thesis describes the contributions made by the author in the design and application of these algorithms. A fully automated rail inspection system that has never been implemented before in any such company or setup has been developed. An industrial computer vision system (JLI) already exists for the image acquisition of rails during production at a rail manufacturing plant in Scunthorpe. An automated inspection system using the same JLI vision system has been developed for the detection of rail‟s surface defects during manufacturing process. This is to complement the human factor by developing a fully automated image processing based system to recognize the faults with an improved efficiency and to allow an exhaustive detection on the entire rail in production. A set of bespoke algorithms has been developed from a plethora of available image processing techniques to extract and identify components in an image of rail in order to detect abnormalities. This has been achieved through offline processing of the rail images using the blended use of different object recognition and image processing techniques, in particular, variation of standard image processing techniques. Several edge detection methods as well as adapted well known Artificial Neural Network and Principal Component Analysis techniques for fault detection on rail have been developed. A combination of customised existing image algorithms and newly developed algorithms have been put together to perform the efficient defect detection. The developed system is fast, reliable and efficient for detection of unique artefacts occurring on the rail surface during production followed by fault classification on the rail imaging system. Extensive testing shows that the defect detection techniques developed for automated rail inspection is capable of detecting more than 90% of the defects present in the available data set of rail images, which has more than 100,000 images under investigation. This demonstrates the efficiency and accuracy of the algorithms developed in this work

Development of a new approach for predicting tram track degradation based on passenger ride/comfort data

These days tram as a type of the public transport system has become popular because of its attractive features such as road usage efficiency, low emission of pollutants, reduction in traffic congestion and efficiency in capital costs and maintenance expenses compared to private cars. For the case study, the Melbourne tram network, which is the longest tram network in the world, has been targeted. Melbourne tram system consists of 493 trams, 24 routes, and 1,763 tram stops. According to the operator of the Melbourne tram network, the total number of patronage in 2017-2018 was 206.3 million. In parallel with the annual increase in tram demand and patronage, tram infrastructure systems need to bear more stresses and traffic pressure. Track degradation is a common problem in the area of tram track infrastructure. One of the main aspects of track degradation is the presence of irregularity in track geometric parameters. In order to deal with degradation problems, tram track infrastructure maintenance management systems have been developed for design and implementation of maintenance works and renewal activities. Track degradation prediction models are the core and the main part of these management systems. Without accurately predicting the future condition of tram tracks, designing and providing preventive maintenance strategies are not feasible. In this research, the collected data which cover six sequential years (2010 to 2015) have been analysed and influencing parameters in tram track degradation have been identified. Gauge and twist were identified as the influencing track geometry parameters in the tram track degradation. Besides that, track surface and rail support as structural parameters were identified as significant parameters in prediction of future track geometry parameters and consequently tram track degradation. In order to develop tram track degradation prediction models and according to the successful experience of the previous studies, three types of prediction models including Artificial Neural Network (ANN), Support Vector Machine (SVM) and Random Forest Regression (RFR) models have been created. According to the results, RFR models provide better predictions in terms of the performance indicators including the coefficient of determination and Root Mean Squared Error (RMSE) compared to the ANN and SVM models. In this research, based on the Melbourne tram track dataset, a new track degradation index has been proposed. Track degradation indices can be used as an indicator of rail condition concerning the risk of damage or failure over a period of time. The index can be applied in establishing a sustainable tram track maintenance management system. The new index composed of two main parts including the mean value of the geometry deviation and the average differential geometry deviation. The proposed index has been compared with three major track geometry degradation indices. For this purpose, the predictability performance of the indices has been considered. In this regard, the Pearson correlation analysis was applied to previous and current values of the indices. According to the results, the correlation coefficient of the proposed index was higher than the other indices. The finding of the evaluation presented that the proposed index can be used as an effective measure for the assessment of the geometric condition of tram tracks. In this research, a new approach has been proposed to predict the tram track degradation were which is cost-effective and can be carried out repeatedly without imposing delay to tram services. Conventional approaches are mainly based on the previous track geometry parameters which have been discussed in this research. In the new approach, passenger ride comfort data or acceleration data has been used to predict the future condition of track geometry parameters which has been represented by the tram track degradation index. For developing the degradation prediction models, the previous models which have been used to predict the degradation based on the track geometry parameters were applied. The future degradation index has been targeted as the target variable and acceleration parameter besides the structural parameters have been used as the explanatory variables. According to the results of the evaluation, the RFR model can predict the future degradation index with approximately 10 percent higher R2 and 9 percent lower prediction error compared to other developed models. In this research two methods for predicting the future tram track degradation index, first was the method based on the previous track geometry parameters and the second was the method based on the acceleration data, have been presented. According to the results of the degradation index prediction based on the previous track geometry parameters, RMSE was 0.35 and R2 value was 0.95. On the other hand, for the prediction based on the acceleration data, RMSE was 1.04 and R2 value was 0.74. The comparison of these methods shows that although the prediction error has been increased and R2 value has been decreased in the latest method, the values of the performance indicators are still in acceptable ranges. These results imply that the prediction of tram track degradation based on the acceleration data can be considered as a reliable method along with conventional tram track degradation prediction method for maintaining tram tracks. The proposed method can provide more predictions of potential future faults by reducing inspection costs and inspection intervals

SynerCrete’18: interdisciplinary approaches for cement-based materials and structural concrete: synergizing expertise and bridging scales of space and time, vol. 2

info:eu-repo/semantics/publishedVersio

Aeronautical Engineering: A continuing bibliography with indexes

This bibliography lists 512 reports, articles and other documents introduced into the NASA scientific and technical information system in April 1982