Artificial Intelligence in Civil Infrastructure Health Monitoring—historical Perspectives, Current Trends, and Future Visions

Over the past 2 decades, the use of artificial intelligence (AI) has exponentially increased toward complete automation of structural inspection and assessment tasks. This trend will continue to rise in image processing as unmanned aerial systems (UAS) and the internet of things (IoT) markets are expected to expand at a compound annual growth rate of 57.5% and 26%, respectively, from 2021 to 2028. This paper aims to catalog the milestone development work, summarize the current research trends, and envision a few future research directions in the innovative application of AI in civil infrastructure health monitoring. A blow-by-blow account of the major technology progression in this research field is provided in a chronological order. Detailed applications, key contributions, and performance measures of each milestone publication are presented. Representative technologies are detailed to demonstrate current research trends. A road map for future research is outlined to address contemporary issues such as explainable and physics-informed AI. This paper will provide readers with a lucid memoir of the historical progress, a good sense of the current trends, and a clear vision for future research

In-line quality control for Zero Defect Manufacturing: design, development and uncertainty analysis of vision-based instruments for dimensional measurements at different scales

Lo scopo di questo progetto di dottorato industriale finanziato attraverso una borsa di studio della Regione Marche è stato quello di sviluppare ricerca con potenziale impatto su un settore industriale, promuovere il coinvolgimento delle fabbriche e delle imprese locali nella ricerca e innovazione svolta in collaborazione con l'università e produrre ricerca in linea con le esigenze dell'ambiente industriale, non solo a livello regionale. Quindi, attraverso la collaborazione con una torneria locale (Zannini SpA) e una piccola azienda high-tech focalizzata sull'introduzione dell'innovazione meccatronica nel settore della tornitura (Z4Tec srl), e anche grazie a una collaborazione internazionale con l'Università di Anversa, abbiamo progettato e sviluppato nuovi strumenti per il controllo qualità in linea, basati su tecnologie senza contatto, in particolare tecnologie elettro-ottiche. Portando anche l'attenzione sull'importanza di prendere in considerazione l'incertezza, poiché è fondamentale nel processo decisionale basato sui dati che sono alla base di una strategia di Zero Defect Manufacturing. Infatti, la scarsa qualità delle misure può pregiudicare la qualità dei dati. In particolare, questo lavoro presenta due strumenti di misura che sono stati progettati e sviluppati con lo scopo di effettuare controllo qualità in linea di produzione e l’incertezza di misura di ogni strumento è stata analizzata in confronto ad altri strumenti presenti sul mercato. Nella parte finale di questo lavoro si è valutata l’incertezza di un profilometro a triangolazione di linea laser. Pertanto, la ricerca condotta in questa tesi può essere organizzata in due obiettivi principali: lo sviluppo di nuovi sistemi di misura dimensionale basati sulla visione da implementare in linea di produzione e l'analisi dell'incertezza di questi strumenti di misura. Per il primo obiettivo ci siamo concentrati su due tipi di misure dimensionali imposte dall'industria manifatturiera: macroscopiche (misure in mm) e microscopiche (misure in µm). Per le misure macroscopiche l'obiettivo era il controllo in linea della qualità dimensionale di pezzi torniti attraverso la profilometria ottica telecentrica. Il campione da ispezionare è stato posto tra l'illuminatore e l'obiettivo per ottenere la proiezione dell'ombra del campione. Le misure sono state eseguite mediante analisi grafica dell'immagine. Abbiamo discusso le disposizioni meccaniche mirate a ottimizzare le immagini acquisite e i problemi che eventuali disallineamenti meccanici dei componenti potrebbero introdurre nella qualità delle immagini. Per le misure microscopiche abbiamo progettato un sistema di misurazione della rugosità superficiale basato sulla visione retroilluminata, con l'obiettivo di determinare le condizioni ottimali di imaging utilizzando la modulation transfer function e l'uso di una electrically tunable lens. Un campione tornito (un cilindro) è posto di fronte a una telecamera ed è retroilluminato da una sorgente di luce collimata; tale configurazione ottica fornisce l'immagine del bordo del campione. Per testare la sensibilità del sistema di misura è stata utilizzata una serie di campioni di acciaio torniti con diverse rugosità superficiali. Per il secondo obiettivo, le tecniche di valutazione dell'incertezza di misura utilizzate in questo lavoro sono state un'analisi dell'incertezza statistica di tipo A e un'analisi Gage R&R. Nel caso del profilometro telecentrico, l'analisi è stata eseguita in confronto con altri dispositivi presenti sul mercato con un'analisi di tipo A e una Gage R&R. L'incertezza di misura del profilometro si è rivelata sufficiente per ottenere risultati nell'intervallo di tolleranza richiesto. Per il sistema di visione retroilluminato, il confronto dei risultati è stato effettuato con altri strumenti allo stato dell'arte, con un'analisi di Tipo A. Il confronto ha mostrato che le prestazioni dello strumento retroilluminato dipendono dai valori di rugosità superficiale considerati; mentre a valori maggiori di rugosità l'offset aumenta, per valori inferiori di rugosità i risultati sono compatibili con quelli dello strumento di riferimento (a stilo). Infine, sono state valutate la ripetibilità e la riproducibilità di un profilometro a triangolazione di linea laser, attraverso uno studio Gage R&R. Ogni punto di misura è stato ispezionato da tre operatori e l'insieme dei dati è stato elaborato con un'analisi dell'incertezza di Tipo A. Successivamente, uno studio Gage R&R ha contribuito a indagare la ripetibilità, la riproducibilità e la variabilità del sistema. Questa analisi ha dimostrato un'incertezza accettabile.The purpose of this industrial PhD project financed through a scholarship from the Regione Marche was to develop research with potential impact on an industrial sector, to promote the involvement of local factories and companies in research and innovation performed jointly with the university and to produce research in line with the needs of the industrial environment, not only at regional level. Hence, through collaborating with a local turning factory (Zannini SpA) and a small high-tech company focused on introducing mechatronic innovation in the turning sector (Z4Tec srl), and also thanks to an international collaboration with the University of Antwerp, we designed and developed new instruments for in-line quality control, based on non-contact technologies, specifically electro-optical technologies. While also bringing attention to the importance of taking uncertainty into consideration, since it is pivotal in data-based decision making which are at the base of a Zero Defect Manufacturing strategy. This means that poor quality of measurements can prejudice the quality of the data. In particular, this work presents two measurement instruments that were designed and developed for the purpose of in-line quality control and the uncertainty of each of the two instruments was evaluated and analyzed in comparison with instruments already present on the market. In the last part of this work, the uncertainty of a hand-held laser-line triangulation profilometer is estimated. Hence, the research conducted in this thesis can be organized in two main objectives: the development of new vision-based dimensional measurement systems to be implemented in production line and the uncertainty analysis of these measurement instruments. For the first objective we focused on two types of dimensional measurements imposed by the manufacturing industry: macroscopic (measuring dimensions in mm) and microscopic (measuring roughness in µm). For macroscopic measurements the target was the in-production dimensional quality control of turned parts through telecentric optical profilometry. The sample to be inspected was placed between illuminator and objective in order to obtain the projection of the shadow of the sample over a white background. Dimensional measurements were then performed by means of image processing over the image obtained. We discussed the mechanical arrangements targeted to optimize images acquired as well as the main issues that eventual mechanical misalignments of components might introduce in the quality of images. For microscopic measurements we designed a backlit vision-based surface roughness measurement system with a focus on smart behaviors such as determining the optimal imaging conditions using the modulation transfer function and the use of an electrically tunable lens. A turned sample (a cylinder) is placed in front of a camera and it is backlit by a collimated source of light; such optical configuration provides the image of the edge of the sample. A set of turned steel samples with different surface roughness was used to test the sensitivity of the measurement system. For the second objective, the measurement uncertainty evaluation techniques used in this work were a Type A statistical uncertainty analysis and a Gage R&R analysis. In the case of the telecentric profilometer, the analysis was performed in comparison with other on-the-market devices with a Type A analysis and a Gage R&R analysis. The measurement uncertainty of the profilometer proved to be sufficient to obtain results within the tolerance interval required. For the backlit vision system, the comparison of the results was made with other state-of-the-art instruments, with a Type A analysis. The comparison showed that the performance of the backlit instrument depends on the values of surface roughness considered; while at larger values of roughness the offset increases, the results are compatible with the ones of the reference instrument (stylus-based) at lower values of roughness. Lastly, the repeatability and reproducibility of a laser-line triangulation profilometer were assessed, through a Gage R&R study. Each measuring point was inspected by three different operators and the data set has been, at first, processed by a Type A uncertainty analysis. Then, a Gage R&R study helped investigate repeatability, reproducibility and the system variability. This analysis showed that the presented laser-line triangulation system has an acceptable uncertainty

Deep learning–based nondestructive evaluation of reinforcement bars using ground-penetrating radar and electromagnetic induction data

Funding Information: The research was funded by the National Natural Science Foundation of China (41974165, 42111530126) and Hubei Key Laboratory of Intelligent Geo‐Information Processing (KLIGIP‐2018A2). The authors thank Zhiwei Duan and Xuefeng Yin for their contributions in the initial stage of the work, and the editor and anonymous reviewers for their constructive comments and suggestions to improve the quality of the paper.Peer reviewedPostprin

Deep Learning Based Concrete Distress Detection System for Civil Infrastructure

In most civil concrete structures, the inspection of structural health is essential. A periodical inspection process ensures that the infrastructure will meet the functional requirements properly or not. To avoid hazardous situations in civil infrastructure, proper maintenance of concrete structures is necessary. The manual visual examination process might provide erroneous results while exploring critical parts of concrete surfaces. As a result, an accurate, safe, and dependable automated process is required for detecting concrete distress. Spalling is a critical distress that can damage concrete surfaces in civil infrastructure. Severe and harmful spalling needs to be taken care of to avoid life-threatening incidents by identifying the location of the distress. Aside from determining the location of the spalling, the severity level of the spalling must also be determined. These severity levels help determine how adverse the situation is and prioritize the process of fixing the spalling. Due to the impact of concrete distress, detecting surface defects like spallings caught the attention of researchers. In this thesis, we have presented approaches to detecting the location of spalling according to its severity level. The proposed methods use deep learning-based approaches and multi-class semantic segmentation. Our approaches have explored two major criteria to detect the spalling and categorize its severity level. Furthermore, we have conducted qualitative and quantitative analyses to demonstrate the performance achieved by the proposed methodologies

Non-contact method to assess the surface roughness of metal castings by 3D laser scanning

This paper defines a methodology to estimate the surface roughness of metal castings by 3D laser scanning. The proposed method applies Principal Component Analysis (PCA) which transforms the point cloud of the casting surface into an orthogonal coordinate system. Using this coordinate system, the Root Mean Square (RMS) deviation of the surface peaks and valleys is estimated. This method is used to analyze the factors affecting point cloud generation and evaluate the technique used to obtain a consistent roughness parameter. A correlation curve was then established by plotting the roughness parameters obtained by PCA method against the corresponding root-mean square (RMS) readings on the cast micro finish comparator. Surface roughness measurements is performed on SCRATA ‘A’ plates and independent casting surfaces; whose roughness is previously unknown; is measured and the results are found to be consistent with the roughness values of the known cast micro finish comparator. The results from the surface comparators and areas of the scanned castings are also validated using a laser interferometer. The proposed method provides a fast, accurate and automated way of calculating surface roughness from the point cloud data. Its repeatability and versatility compares favorably with existing methods and would aid process control and standard interpretation