Uncertainty Quantification for Deep Learning in Ultrasonic Crack Characterization

Deep learning for nondestructive evaluation (NDE) has received a lot of attention in recent years for its potential ability to provide human level data analysis. However, little research into quantifying the uncertainty of its predictions has been done. Uncertainty quantification (UQ) is essential for qualifying NDE inspections and building trust in their predictions. Therefore, this article aims to demonstrate how UQ can best be achieved for deep learning in the context of crack sizing for inline pipe inspection. A convolutional neural network architecture is used to size surface breaking defects from plane wave imaging (PWI) images with two modern UQ methods: deep ensembles and Monte Carlo dropout. The network is trained using PWI images of surface breaking defects simulated with a hybrid finite element / ray-based model. Successful UQ is judged by calibration and anomaly detection, which refer to whether in-domain model error is proportional to uncertainty and if out of training domain data is assigned high uncertainty. Calibration is tested using simulated and experimental images of surface breaking cracks, while anomaly detection is tested using experimental side-drilled holes and simulated embedded cracks. Monte Carlo dropout demonstrates poor uncertainty quantification with little separation between in and out-of-distribution data and a weak linear fit ( R=0.84 ) between experimental root-mean-square-error and uncertainty. Deep ensembles improve upon Monte Carlo dropout in both calibration ( R=0.95 ) and anomaly detection. Adding spectral normalization and residual connections to deep ensembles slightly improves calibration ( R=0.98 ) and significantly improves the reliability of assigning high uncertainty to out-of-distribution samples

Deep learning in automated ultrasonic NDE -- developments, axioms and opportunities

The analysis of ultrasonic NDE data has traditionally been addressed by a trained operator manually interpreting data with the support of rudimentary automation tools. Recently, many demonstrations of deep learning (DL) techniques that address individual NDE tasks (data pre-processing, defect detection, defect characterisation, and property measurement) have started to emerge in the research community. These methods have the potential to offer high flexibility, efficiency, and accuracy subject to the availability of sufficient training data. Moreover, they enable the automation of complex processes that span one or more NDE steps (e.g. detection, characterisation, and sizing). There is, however, a lack of consensus on the direction and requirements that these new methods should follow. These elements are critical to help achieve automation of ultrasonic NDE driven by artificial intelligence such that the research community, industry, and regulatory bodies embrace it. This paper reviews the state-of-the-art of autonomous ultrasonic NDE enabled by DL methodologies. The review is organised by the NDE tasks that are addressed by means of DL approaches. Key remaining challenges for each task are noted. Basic axiomatic principles for DL methods in NDE are identified based on the literature review, relevant international regulations, and current industrial needs. By placing DL methods in the context of general NDE automation levels, this paper aims to provide a roadmap for future research and development in the area.Comment: Accepted version to be published in NDT & E Internationa

Roadmap on signal processing for next generation measurement systems

Signal processing is a fundamental component of almost any sensor-enabled system, with a wide range of applications across different scientific disciplines. Time series data, images, and video sequences comprise representative forms of signals that can be enhanced and analysed for information extraction and quantification. The recent advances in artificial intelligence and machine learning are shifting the research attention towards intelligent, data-driven, signal processing. This roadmap presents a critical overview of the state-of-the-art methods and applications aiming to highlight future challenges and research opportunities towards next generation measurement systems. It covers a broad spectrum of topics ranging from basic to industrial research, organized in concise thematic sections that reflect the trends and the impacts of current and future developments per research field. Furthermore, it offers guidance to researchers and funding agencies in identifying new prospects.AerodynamicsMicrowave Sensing, Signals & System

Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines

Technology Readiness Levels for Machine Learning Systems

The development and deployment of machine learning (ML) systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. The lack of diligence can lead to technical debt, scope creep and misaligned objectives, model misuse and failures, and expensive consequences. Engineering systems, on the other hand, follow well-defined processes and testing standards to streamline development for high-quality, reliable results. The extreme is spacecraft systems, where mission critical measures and robustness are ingrained in the development process. Drawing on experience in both spacecraft engineering and ML (from research through product across domain areas), we have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" (MLTRL) framework defines a principled process to ensure robust, reliable, and responsible systems while being streamlined for ML workflows, including key distinctions from traditional software engineering. Even more, MLTRL defines a lingua franca for people across teams and organizations to work collaboratively on artificial intelligence and machine learning technologies. Here we describe the framework and elucidate it with several real world use-cases of developing ML methods from basic research through productization and deployment, in areas such as medical diagnostics, consumer computer vision, satellite imagery, and particle physics

Automated damage diagnosis of concrete jack arch beam using optimized deep stacked autoencoders and multi-sensor fusion

A novel hybrid framework of optimized deep learning models combined with multi-sensor fusion is developed for condition diagnosis of concrete arch beam. The vibration responses of structure are first processed by principal component analysis for dimensionality reduction and noise elimination. Then, the deep network based on stacked autoencoders (SAE) is established at each sensor for initial condition diagnosis, where extracted principal components and corresponding condition categories are inputs and output, respectively. To enhance diagnostic accuracy of proposed deep SAE, an enhanced whale optimization algorithm is proposed to optimize network meta-parameters. Eventually, Dempster-Shafer fusion algorithm is employed to combine initial diagnosis results from each sensor to make a final diagnosis. A miniature structural component of Sydney Harbour Bridge with artificial multiple progressive damages is tested in laboratory. The results demonstrate that the proposed method can detect structural damage accurately, even under the condition of limited sensors and high levels of uncertainties