A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment

InsPLAD: A Dataset and Benchmark for Power Line Asset Inspection in UAV Images

Power line maintenance and inspection are essential to avoid power supply interruptions, reducing its high social and financial impacts yearly. Automating power line visual inspections remains a relevant open problem for the industry due to the lack of public real-world datasets of power line components and their various defects to foster new research. This paper introduces InsPLAD, a Power Line Asset Inspection Dataset and Benchmark containing 10,607 high-resolution Unmanned Aerial Vehicles colour images. The dataset contains seventeen unique power line assets captured from real-world operating power lines. Additionally, five of those assets present six defects: four of which are corrosion, one is a broken component, and one is a bird's nest presence. All assets were labelled according to their condition, whether normal or the defect name found on an image level. We thoroughly evaluate state-of-the-art and popular methods for three image-level computer vision tasks covered by InsPLAD: object detection, through the AP metric; defect classification, through Balanced Accuracy; and anomaly detection, through the AUROC metric. InsPLAD offers various vision challenges from uncontrolled environments, such as multi-scale objects, multi-size class instances, multiple objects per image, intra-class variation, cluttered background, distinct point-of-views, perspective distortion, occlusion, and varied lighting conditions. To the best of our knowledge, InsPLAD is the first large real-world dataset and benchmark for power line asset inspection with multiple components and defects for various computer vision tasks, with a potential impact to improve state-of-the-art methods in the field. It will be publicly available in its integrity on a repository with a thorough description. It can be found at https://github.com/andreluizbvs/InsPLAD.Comment: This is an original manuscript of an article published by Taylor & Francis in the International Journal of Remote Sensing on 29 Nov 2023, available online: https://doi.org/10.1080/01431161.2023.228390

Detection of Deficiencies and Data Analysis of Bridge Members with Deep Convolutional Neural Networks

Concrete cracks and structural steel corrosion are two of the most common defects in bridges. Quantifying and classifying these defects provide bridge inspectors and engineers with valuable data for assessing deterioration levels. However, the bridge inspection process is typically a subjective, time intensive, and tedious task, as defects can be overlooked or in locations not easily accessible. Previous studies have investigated deep learning-based inspection methods, implementing popular models such as Mask R-CNN and U-Net. The architectures of these models offer certain advantages depending on the required task. This thesis aims to evaluate and compare Mask R-CNN and U-Net regarding their crack detection and segmentation performances. Both models utilize identical datasets for training and validation. These datasets are compiled from publicly available sources and from previous data collection efforts by a research team at the University of Nebraska-Lincoln. Detection accuracy is evaluated utilizing a labeled orthomosaic image of a pedestrian bridge located in Lincoln, Nebraska. Analyzing the effectiveness of the deep convolutional neural networks on a large-scale image, rather than local images, is more akin to how traditional inspections are completed. In addition, both models are evaluated for their capabilities through surface strain analysis on the same pedestrian bridge. The crack predictions of the models are comprised of binary pixel-wise classifications (crack or non-crack) where the Euclidean distance and centerline-to-centerline spacing were measured. These measurements were gathered to compute relative crack width and crack spacing for strain calculation. Average strain values for each of the three spans of the bridge were then calculated. The average strains obtained from U-Net mask prediction measurements for each span fell within the valid service stage, encompassing the range from cracking to yield stress. In contrast, Mask R-CNN yielded strain results indicated stress levels exceeding yield in those spans. One of the largest publicly available structural steel corrosion datasets was also published as part of this thesis [at https://github.com/bennett-jackson/steel-corrosion-dataset]. Corrosion detection and evaluation has not been researched as extensively as concrete crack detection due to limitations in dataset accessibility. The published dataset consists of 3,423 images containing various levels of corrosion. These images were categorized into five condition states (good, fair, poor, severe, and weathering) based upon an existing corrosion annotation guideline. Classifying images in this manner aims to provide an organized dataset for future research. Advisor: Chungwook Si

Low-cost deep learning UAV and Raspberry Pi solution to real time pavement condition assessment

In this thesis, a real-time and low-cost solution to the autonomous condition assessment of pavement is proposed using deep learning, Unmanned Aerial Vehicle (UAV) and Raspberry Pi tiny computer technologies, which makes roads maintenance and renovation management more efficient and cost effective. A comparison study was conducted to compare the performance of seven different combinations of meta-architectures for pavement distress classification. It was observed that real-time object detection architecture SSD with MobileNet feature extractor is the best combination for real-time defect detection to be used by tiny computers. A low-cost Raspberry Pi smart defect detector camera was configured using the trained SSD MobileNet v1, which can be deployed with UAV for real-time and remote pavement condition assessment. The preliminary results show that the smart pavement detector camera achieves an accuracy of 60% at 1.2 frames per second in raspberry pi and 96% at 13.8 frames per second in CPU-based computer

Detection of exposed steel rebars based on deep-learning techniques and unmanned aerial vehicles

In recent years deep-learning techniques have been developed and applied to inspect cracks in RC structures. The accuracy of these techniques leads to believe that they may also be applied to the identification of other pathologies. This article proposes a technique for automated detection of exposed steel rebars. The tools developed rely on convolutional neural networks (CNNs) based on transfer-learning using AlexNet. Experiments were conducted in large-scale structures to assess the efficiency of the method. To circumvent limitations on the proximity access to structures as large as the ones used in the experiments, as well as increase cost efficiency, the image capture was performed using an unmanned aerial system (UAS). The final goal of the proposed methodology is to generate orthomosaic maps of the pathologies or structure 3D models with superimposed pathologies. The results obtained are promising, confirming the high adaptability of CNN based methodologies for structural inspection.This work was financially supported by: Base Funding - UIDB/04708/2020 and Programmatic Funding - UIDP/04708/2020 of the CONSTRUCT - Instituto de I&D em Estruturas e Construções funded by national funds through the FCT/MCTES (PIDDAC). Additionally, the author Rafael Cabral acknowledges the support provided by the doctoral grant UI/BD/150970/2021 - Portuguese Science Foundation, FCT/MCTES.info:eu-repo/semantics/publishedVersio

Hypertuned-YOLO for interpretable distribution power grid fault location based on EigenCAM

Ensuring the reliability of electrical distribution networks is a pressing concern, especially given the power outages due to surface contamination on insulating components. Surface contamination can elevate surface conductivity, thereby resulting in failures that can lead to power shutdowns. Addressing this challenge, this paper proposes an approach for real-time monitoring of electrical distribution grids to prevent such incidents. A hypertuned version of the you only look once (YOLO) model is tailored for this application. We refine the model's hyperparameters by integrating a genetic algorithm to maximize its detection performance. The EigenCAM technique enhances the visual interpretability of the model's outcomes, providing operators with actionable insights for maintenance and monitoring tasks. Benchmark tests reveal that the proposed Hypertuned-YOLO outperforms Detectron (Masked R-CNN), YOLOv5, and YOLOv7 models. The Hypertuned-YOLO achieves an F1-score of 0.867 and a [email protected] of 0.922, validating its robustness and efficacy

Enhancing Road Infrastructure Monitoring: Integrating Drones for Weather-Aware Pothole Detection

The abstract outlines the research proposal focused on the utilization of Unmanned Aerial Vehicles (UAVs) for monitoring potholes in road infrastructure affected by various weather conditions. The study aims to investigate how different materials used to fill potholes, such as water, grass, sand, and snow-ice, are impacted by seasonal weather changes, ultimately affecting the performance of pavement structures. By integrating weather-aware monitoring techniques, the research seeks to enhance the rigidity and resilience of road surfaces, thereby contributing to more effective pavement management systems. The proposed methodology involves UAV image-based monitoring combined with advanced super-resolution algorithms to improve image refinement, particularly at high flight altitudes. Through case studies and experimental analysis, the study aims to assess the geometric precision of 3D models generated from aerial images, with a specific focus on road pavement distress monitoring. Overall, the research aims to address the challenges of traditional road failure detection methods by exploring cost-effective 3D detection techniques using UAV technology, thereby ensuring safer roadways for all users