Defect detection in infrared thermography by deep learning algorithms

L'évaluation non destructive (END) est un domaine permettant d'identifier tous les types de dommages structurels dans un objet d'intérêt sans appliquer de dommages et de modifications permanents. Ce domaine fait l'objet de recherches intensives depuis de nombreuses années. La thermographie infrarouge (IR) est l'une des technologies d'évaluation non destructive qui permet d'inspecter, de caractériser et d'analyser les défauts sur la base d'images infrarouges (séquences) provenant de l'enregistrement de l'émission et de la réflexion de la lumière infrarouge afin d'évaluer les objets non autochauffants pour le contrôle de la qualité et l'assurance de la sécurité. Ces dernières années, le domaine de l'apprentissage profond de l'intelligence artificielle a fait des progrès remarquables dans les applications de traitement d'images. Ce domaine a montré sa capacité à surmonter la plupart des inconvénients des autres approches existantes auparavant dans un grand nombre d'applications. Cependant, en raison de l'insuffisance des données d'entraînement, les algorithmes d'apprentissage profond restent encore inexplorés, et seules quelques publications font état de leur application à l'évaluation non destructive de la thermographie (TNDE). Les algorithmes d'apprentissage profond intelligents et hautement automatisés pourraient être couplés à la thermographie infrarouge pour identifier les défauts (dommages) dans les composites, l'acier, etc. avec une confiance et une précision élevée. Parmi les sujets du domaine de recherche TNDE, les techniques d'apprentissage automatique supervisées et non supervisées sont les tâches les plus innovantes et les plus difficiles pour l'analyse de la détection des défauts. Dans ce projet, nous construisons des cadres intégrés pour le traitement des données brutes de la thermographie infrarouge à l'aide d'algorithmes d'apprentissage profond et les points forts des méthodologies proposées sont les suivants: 1. Identification et segmentation automatique des défauts par des algorithmes d'apprentissage profond en thermographie infrarouge. Les réseaux neuronaux convolutifs (CNN) pré-entraînés sont introduits pour capturer les caractéristiques des défauts dans les images thermiques infrarouges afin de mettre en œuvre des modèles basés sur les CNN pour la détection des défauts structurels dans les échantillons composés de matériaux composites (diagnostic des défauts). Plusieurs alternatives de CNNs profonds pour la détection de défauts dans la thermographie infrarouge. Les comparaisons de performance de la détection et de la segmentation automatique des défauts dans la thermographie infrarouge en utilisant différentes méthodes de détection par apprentissage profond : (i) segmentation d'instance (Center-mask ; Mask-RCNN) ; (ii) détection d’objet (Yolo-v3 ; Faster-RCNN) ; (iii) segmentation sémantique (Unet ; Res-unet); 2. Technique d'augmentation des données par la génération de données synthétiques pour réduire le coût des dépenses élevées associées à la collecte de données infrarouges originales dans les composites (composants d'aéronefs.) afin d'enrichir les données de formation pour l'apprentissage des caractéristiques dans TNDE; 3. Le réseau antagoniste génératif (GAN convolutif profond et GAN de Wasserstein) est introduit dans la thermographie infrarouge associée à la thermographie partielle des moindres carrés (PLST) (réseau PLS-GANs) pour l'extraction des caractéristiques visibles des défauts et l'amélioration de la visibilité des défauts pour éliminer le bruit dans la thermographie pulsée; 4. Estimation automatique de la profondeur des défauts (question de la caractérisation) à partir de données infrarouges simulées en utilisant un réseau neuronal récurrent simplifié : Gate Recurrent Unit (GRU) à travers l'apprentissage supervisé par régression.Non-destructive evaluation (NDE) is a field to identify all types of structural damage in an object of interest without applying any permanent damage and modification. This field has been intensively investigated for many years. The infrared thermography (IR) is one of NDE technology through inspecting, characterize and analyzing defects based on the infrared images (sequences) from the recordation of infrared light emission and reflection to evaluate non-self-heating objects for quality control and safety assurance. In recent years, the deep learning field of artificial intelligence has made remarkable progress in image processing applications. This field has shown its ability to overcome most of the disadvantages in other approaches existing previously in a great number of applications. Whereas due to the insufficient training data, deep learning algorithms still remain unexplored, and only few publications involving the application of it for thermography nondestructive evaluation (TNDE). The intelligent and highly automated deep learning algorithms could be coupled with infrared thermography to identify the defect (damages) in composites, steel, etc. with high confidence and accuracy. Among the topics in the TNDE research field, the supervised and unsupervised machine learning techniques both are the most innovative and challenging tasks for defect detection analysis. In this project, we construct integrated frameworks for processing raw data from infrared thermography using deep learning algorithms and highlight of the methodologies proposed include the following: 1. Automatic defect identification and segmentation by deep learning algorithms in infrared thermography. The pre-trained convolutional neural networks (CNNs) are introduced to capture defect feature in infrared thermal images to implement CNNs based models for the detection of structural defects in samples made of composite materials (fault diagnosis). Several alternatives of deep CNNs for the detection of defects in the Infrared thermography. The comparisons of performance of the automatic defect detection and segmentation in infrared thermography using different deep learning detection methods: (i) instance segmentation (Center-mask; Mask-RCNN); (ii) objective location (Yolo-v3; Faster-RCNN); (iii) semantic segmentation (Unet; Res-unet); 2. Data augmentation technique through synthetic data generation to reduce the cost of high expense associated with the collection of original infrared data in the composites (aircraft components.) to enrich training data for feature learning in TNDE; 3. The generative adversarial network (Deep convolutional GAN and Wasserstein GAN) is introduced to the infrared thermography associated with partial least square thermography (PLST) (PLS-GANs network) for visible feature extraction of defects and enhancement of the visibility of defects to remove noise in Pulsed thermography; 4. Automatic defect depth estimation (Characterization issue) from simulated infrared data using a simplified recurrent neural network: Gate Recurrent Unit (GRU) through the regression supervised learning

Ensemble Joint Sparse Low Rank Matrix Decomposition for Thermography Diagnosis System

Composite is widely used in the aircraft industry and it is essential for manufacturers to monitor its health and quality. The most commonly found defects of composite are debonds and delamination. Different inner defects with complex irregular shape is difficult to be diagnosed by using conventional thermal imaging methods. In this paper, an ensemble joint sparse low rank matrix decomposition (EJSLRMD) algorithm is proposed by applying the optical pulse thermography (OPT) diagnosis system. The proposed algorithm jointly models the low rank and sparse pattern by using concatenated feature space. In particular, the weak defects information can be separated from strong noise and the resolution contrast of the defects has significantly been improved. Ensemble iterative sparse modelling are conducted to further enhance the weak information as well as reducing the computational cost. In order to show the robustness and efficacy of the model, experiments are conducted to detect the inner debond on multiple carbon fiber reinforced polymer (CFRP) composites. A comparative analysis is presented with general OPT algorithms. Not withstand above, the proposed model has been evaluated on synthetic data and compared with other low rank and sparse matrix decomposition algorithms

Detection and quantification of delamination in concrete via time-lapse thermography with machine learning

This study developed a framework to automatically extract sub-surface defects from time-lapse thermography (TLT) images of reinforced concrete bridge components. Traditional approaches for processing TLT data typically require manual interventions that are not easily scaled to a large network of concrete bridges. A backbone of robust algorithms for detecting and analyzing deep sub-surface defects in concrete is needed to support condition assessment of concrete structures such as bridges. The current study leverages advances in adaptive signal and image processing to develop a fully automated TLT data processing pipeline that is capable of efficiently detecting defects at different depths in concrete. The methodology decomposes raw TLT datasets into narrow band time-frequency domains via a multiscale data analysis approach called a Wavelet Transform. The resulting decomposed modes are mined to extract defect information using thermal contrast enhancement routines. An objective measure of effectiveness based on signal-to-noise ratio was developed and used to compare the current framework with traditional approaches for processing TLT data. Active contour models were also designed to automatically extract the boundary location and geometric properties of the sub-surface defects. The results of this study show that the detection of deeper defects (3 in. and beyond) can be improved by analyzing the time-frequency response of surface temperature variations over a period of time. Compared to traditional lock-in algorithms and conventional infrared thermography images, the proposed framework is more effective at removing noisy information and produces images with greater contrast between intact and defective areas of concrete. Furthermore, a new process has been established to predict depths of delamination in reinforced concrete bridge components. For previous works, traditional approaches were adopted to quantify depths in active thermography, which mainly depend on estimated models as a function of time, frequency, phase contrast, material properties of specimens. This work deals with the passive thermography that is affected by several environmental parameters such as solar heating, daytime or nighttime, wind speed, clouds, shadow. The current work has employed the Machine Learning (ML) technology to estimate defect depths in concrete block. Features, such as phases, amplitudes, frequencies, have been extracted by utilizing the Fast Fourier Transform (FFT) in a stage of analysis. Furthermore, additional subfeatures, minor features, have been added to the ML analysis, for instance average and/or subtraction values between the maxima and minima features, to attain an acceptable learning performance. Support vector machine (SVM) and k-Nearest Neighbor (KNN) classifiers have been trained by using crossvalidation with different folds and hold validations. The predicted models have achieved an improved accuracy in estimating delamination depths in the concrete specimens with a good agreement.Includes bibliographical references

Deep learning with filtering for defect characterization in pulsed thermography based non-destructive testing

Pulsed thermography is widely used for non-destructive testing of various materials. The temperature profile obtained after pulse heating is used to characterize the underlying defects in an object. In this paper, the automation of the process of defect visualization and depth quantification in pulsed thermography through various deep learning algorithms is reported. Stainless steel plate with artificial defects is considered for analysis. The raw temperature data is smoothed using moving average, Savitzky-Golay and quadratic regression filters to reduce noise. Thermal signal reconstruction, the conventional method to eliminate noise, is also used for generating filtered datasets. Defect visualization refers to identifying and locating the defects in an image sample and Mask region convolutional neural network (Mask R-CNN) is considered for not just detecting the defects but also locating them on the image. The located defects are utilized for depth estimation using the following networks-multi-layer perceptron (MLP), long short-term memory (LSTM) and gated recurrent units (GRU). The input to the networks is the temperature contrast characteristics which symbolizes the difference in temperature over defective and non-defective areas measured over 250 time points and output of the networks is the estimated depth. The study shows that LSTM based approach provides the least percentage error of 5.5% and is a very suitable approach for automation of defect characterization in pulsed thermography

Step heating thermography supported by machine learning and simulation for internal defect size measurement in additive manufacturing

A methodology based on step-heating thermography for predicting the length dimension of small defects in additive manufacturing from temperature data measured on thermal images is proposed. Regression learners were applied with different configurations to predict the length of the defects. These algorithms were trained using large datasets generated with Finite Element Method simulations. The different predictive methods obtained were optimized using Bayesian inference. Using predictive methods generated and based on intrinsic performance results, knowing the material characteristics, the defect length can be predicted from single temperature data in defect and non-defect zone. Thus, the developed algorithms were implemented in a laboratory set-up carried out on ad-hoc manufactured parts of Nylon and polylactic acid which include induced defects with different sizes and thicknesses. Using the trained algorithm, the deviation of the predicted results for the defect size varied between 13% and 37% for PLA and between 13% and 36% for Nylon.This research has been funded by Ministry of Science and Innovation (Government of Spain) through the research project titled Fusion of nondestructive technologies and numerical simulation methods for the inspection and monitoring of joints in new materials and additive manufacturing processes (FaTIMA) with code RTI2018-099850-B-I00

Thermographic Imaging in Cultural Heritage: A Short Review

Over the recent period, there has been an increasing interest in the use of pulsed infrared thermography (PT) for the non-destructive evaluation of Cultural Heritage (CH). Unlike other techniques that are commonly employed in the same field, PT enables the depth-resolved detection of different kinds of subsurface features, thus providing helpful information for both scholars and restorers. Due to this reason, several research activities are currently underway to further improve the PT effectiveness. In this manuscript, the specific use of PT for the analysis of three different types of CH, namely documentary materials, panel paintings–marquetery, and mosaics, will be reviewed. In the latter case, i.e., mosaics, passive thermography combined with ground penetrating radar (GPR) and digital microscopy (DM) have also been deepened, considering their suitability in the open field. Such items have been selected because they are characterized by quite distinct physical and structural properties and, therefore, different PT (and, in some cases, verification) approaches have been employed for their investigations

Non-Destructive Techniques for the Condition and Structural Health Monitoring of Wind Turbines: A Literature Review of the Last 20 Years

A complete surveillance strategy for wind turbines requires both the condition monitoring (CM) of their mechanical components and the structural health monitoring (SHM) of their load-bearing structural elements (foundations, tower, and blades). Therefore, it spans both the civil and mechanical engineering fields. Several traditional and advanced non-destructive techniques (NDTs) have been proposed for both areas of application throughout the last years. These include visual inspection (VI), acoustic emissions (AEs), ultrasonic testing (UT), infrared thermography (IRT), radiographic testing (RT), electromagnetic testing (ET), oil monitoring, and many other methods. These NDTs can be performed by human personnel, robots, or unmanned aerial vehicles (UAVs); they can also be applied both for isolated wind turbines or systematically for whole onshore or offshore wind farms. These non-destructive approaches have been extensively reviewed here; more than 300 scientific articles, technical reports, and other documents are included in this review, encompassing all the main aspects of these survey strategies. Particular attention was dedicated to the latest developments in the last two decades (2000–2021). Highly influential research works, which received major attention from the scientific community, are highlighted and commented upon. Furthermore, for each strategy, a selection of relevant applications is reported by way of example, including newer and less developed strategies as well

Artificial vision by thermography : calving prediction and defect detection in carbon fiber reinforced polymer

La vision par ordinateur est un domaine qui consiste à extraire ou identifier une ou plusieurs informations à partir d’une ou plusieurs images dans le but soit d’automatiser une tache, soit de fournir une aide à la décision. Avec l’augmentation de la capacité de calcul des ordinateurs, la vulgarisation et la diversification des moyens d’imagerie tant dans la vie quotidienne, que dans le milieu industriel,ce domaine a subi une évolution rapide lors de dernières décennies. Parmi les différentes modalités d’imagerie pour lesquels il est possible d’utiliser la vision artificielle cette thèse se concentre sur l’imagerie infrarouge. Plus particulièrement sur l’imagerie infrarouge pour les longueurs d’ondes comprises dans les bandes moyennes et longues. Cette thèse se porte sur deux applications industrielles radicalement différentes. Dans la première partie de cette thèse, nous présentons une application de la vision artificielle pour la détection du moment de vêlage en milieux industriel pour des vaches Holstein. Plus précisément l’objectif de cette recherche est de déterminer le moment de vêlage en n’utilisant que des données comportementales de l’animal. À cette fin, nous avons acquis des données en continu sur différents animaux pendant plusieurs mois. Parmi les nombreux défis présentés par cette application l’un d’entre eux concerne l’acquisition des données. En effet, les caméras que nous avons utilisées sont basées sur des capteurs bolométriques, lesquels sont sensibles à un grand nombre de variables. Ces variables peuvent être classées en quatre catégories : intrinsèque, environnemental, radiométrique et géométrique. Un autre défit important de cette recherche concerne le traitement des données. Outre le fait que les données acquises utilisent une dynamique plus élevée que les images naturelles ce qui complique le traitement des données ; l’identification de schéma récurrent dans les images et la reconnaissance automatique de ces derniers grâce à l’apprentissage automatique représente aussi un défi majeur. Nous avons proposé une solution à ce problème. Dans le reste de cette thèse nous nous sommes penchés sur la problématique de la détection de défaut dans les matériaux, en utilisant la technique de la thermographie pulsée. La thermographie pulsée est une méthode très populaire grâce à sa simplicité, la possibilité d’être utilisée avec un grand nombre de matériaux, ainsi que son faible coût. Néanmoins, cette méthode est connue pour produire des données bruitées. La cause principale de cette réputation vient des diverses sources de distorsion auquel les cameras thermiques sont sensibles. Dans cette thèse, nous avons choisi d’explorer deux axes. Le premier concerne l’amélioration des méthodes de traitement de données existantes. Dans le second axe, nous proposons plusieurs méthodes pour améliorer la détection de défauts. Chaque méthode est comparée à plusieurs méthodes constituant l’état de l’art du domaine.Abstract Computer vision is a field which consists in extracting or identifying one or more information from one or more images in order either to automate a task or to provide decision support. With the increase in the computing capacity of computers, the popularization and diversification of imaging means, both in industry, as well as in everyone’s life, this field has undergone a rapid development in recent decades. Among the different imaging modalities for which it is possible to use artificial vision, this thesis focuses on infrared imaging. More particularly on infrared imagery for wavelengths included in the medium and long bands. This thesis focuses on two radically different industrial applications. In the first part of this thesis, we present an application of artificial vision for the detection of the calving moment in industrial environments for Holstein cows. More precisely, the objective of this research is to determine the time of calving using only physiological data from the animal. To this end, we continuously acquired data on different animals over several days. Among the many challenges presented by this application, one of them concerns data acquisition. Indeed, the cameras we used are based on bolometric sensors, which are sensitive to a large number of variables. These variables can be classified into four categories: intrinsic, environmental, radiometric and geometric. Another important challenge in this research concerns the processing of data. Besides the fact that the acquired data uses a higher dynamic range than the natural images which complicates the processing of the data; Identifying recurring patterns in images and automatically recognizing them through machine learning is a major challenge. We have proposed a solution to this problem. In the rest of this thesis we have focused on the problem of defect detection in materials, using the technique of pulsed thermography. Pulse thermography is a very popular method due toits simplicity, the possibility of being used with a large number of materials, as well as its low cost. However, this method is known to produce noisy data. The main cause of this reputation comes from the various sources of distortion to which thermal cameras are sensitive. In this thesis, we have chosen to explore two axes. The first concerns the improvement of existing data processing methods. In the second axis, we propose several methods to improve fault detection. Each method is compared to several methods constituting the state of the art in the field

Machine Learning Methods for Rapid Inspection of Automated Fiber Placement Manufactured Composite Structures

The advanced manufacturing capabilities provided through the automated fiber placement (AFP) system has allowed for faster layup time and more consistent production across a number of different geometries. This contributes to the modern production of large composite structures and the widespread adaptation of composites in industry in general and aerospace in particular. However, the automation introduced in this process increases the difficulty of quality assurance efforts and inspection. The AFP process can induce a number of manufacturing defects including wrinkles, twists, gaps, and overlaps. The manual identification of these defects is often laborious and requires a measure of expert knowledge. A software package for the assistance of the inspection process has been used in conjunction with automated inspection hardware for the automated inspection, identification, and characterization of AFP manufacturing defects. Image analysis algorithms were developed and demonstrated on a number of defect types. Defects are identified in scan images and exact size and shape characteristics are extracted for export

Prospect of using machine learning-based microwave nondestructive testing technique for corrosion under insulation: A review

Corrosion under insulations is described as localized corrosion that forms because of moisture penetration through the insulation materials or due to contaminants’ presence within the insulation material. The traditional non-destructive inspection techniques operating at a low frequency require removing insulation material to enable inspection, due to poor signal penetration. Several high-frequency inspection techniques such as the microwave technique have shown successful inspection in detecting the defect under insulations, without removing the insulations. However, the microwave technique faces several challenges such as poor spatial imaging, large errors in terms of defect size and depth owing to stand-off distance variations, optimal frequency point selection, and the presence of the outlier in microwave measurement data. The microwave technique in conjunction with machine learning approaches has tremendous potential and viability for assessing corrosion under insulation. This paper provides an in-depth review of non-destructive techniques for assessing corrosion under insulation, as well as the possibility of using machine learning approaches in microwave techniques in comparison to other conventional techniques