Low-cost deep learning UAV and Raspberry Pi solution to real time pavement condition assessment

In this thesis, a real-time and low-cost solution to the autonomous condition assessment of pavement is proposed using deep learning, Unmanned Aerial Vehicle (UAV) and Raspberry Pi tiny computer technologies, which makes roads maintenance and renovation management more efficient and cost effective. A comparison study was conducted to compare the performance of seven different combinations of meta-architectures for pavement distress classification. It was observed that real-time object detection architecture SSD with MobileNet feature extractor is the best combination for real-time defect detection to be used by tiny computers. A low-cost Raspberry Pi smart defect detector camera was configured using the trained SSD MobileNet v1, which can be deployed with UAV for real-time and remote pavement condition assessment. The preliminary results show that the smart pavement detector camera achieves an accuracy of 60% at 1.2 frames per second in raspberry pi and 96% at 13.8 frames per second in CPU-based computer

Interleaved Deep Artifacts-Aware Attention Mechanism for Concrete Structural Defect Classification.

Automatic machine classification of concrete structural defects in images poses significant challenges because of multitude of problems arising from the surface texture, such as presence of stains, holes, colors, poster remains, graffiti, marking and painting, along with uncontrolled weather conditions and illuminations. In this paper, we propose an interleaved deep artifacts-aware attention mechanism (iDAAM) to classify multi-target multi-class and single-class defects from structural defect images. Our novel architecture is composed of interleaved fine-grained dense modules (FGDM) and concurrent dual attention modules (CDAM) to extract local discriminative features from concrete defect images. FGDM helps to aggregate multi-layer robust information with wide range of scales to describe visually-similar overlapping defects. On the other hand, CDAM selects multiple representations of highly localized overlapping defect features and encodes the crucial spatial regions from discriminative channels to address variations in texture, viewing angle, shape and size of overlapping defect classes. Within iDAAM, FGDM and CDAM are interleaved to extract salient discriminative features from multiple scales by constructing an end-to-end trainable network without any preprocessing steps, making the process fully automatic. Experimental results and extensive ablation studies on three publicly available large concrete defect datasets show that our proposed approach outperforms the current state-of-the-art methodologies

Artificial Intelligence in Civil Infrastructure Health Monitoring—historical Perspectives, Current Trends, and Future Visions

Over the past 2 decades, the use of artificial intelligence (AI) has exponentially increased toward complete automation of structural inspection and assessment tasks. This trend will continue to rise in image processing as unmanned aerial systems (UAS) and the internet of things (IoT) markets are expected to expand at a compound annual growth rate of 57.5% and 26%, respectively, from 2021 to 2028. This paper aims to catalog the milestone development work, summarize the current research trends, and envision a few future research directions in the innovative application of AI in civil infrastructure health monitoring. A blow-by-blow account of the major technology progression in this research field is provided in a chronological order. Detailed applications, key contributions, and performance measures of each milestone publication are presented. Representative technologies are detailed to demonstrate current research trends. A road map for future research is outlined to address contemporary issues such as explainable and physics-informed AI. This paper will provide readers with a lucid memoir of the historical progress, a good sense of the current trends, and a clear vision for future research

Few-shot learning for image-based bridge damage detection

Autonomous bridge visual inspection is a real-world challenge due to various materials, surface coatings, and changing light and weather conditions. Traditional supervised learning relies on massive annotated data to establish a robust model, which requires a time-consuming data acquisition process. This work proposes a few-shot learning (FSL) approach based on improved ProtoNet for damage detection with just a few labeled examples. Feature embedding is achieved through cross-domain transfer learning from ImageNet instead of episodic training. The ProtoNet is improved with embedding normalization to enhance transduction performance based on Euclidean distance and a linear classifier for classification. The approach is explored on a public dataset through different ablation experiments and achieves over 94% mean accuracy for 2-way 5-shot classification via the pre-trained GoogleNet after fine-tuning. Moreover, the proposed fine-tuning methods based on a fully connected layer (FCN) and Hadamard product are demonstrated with better performance than the previous method. Finally, the approach is validated using real bridge inspection images, demonstrating its capability of fast implementation for practical damage inspection with weakly supervised information

dacl10k: Benchmark for Semantic Bridge Damage Segmentation

Reliably identifying reinforced concrete defects (RCDs)plays a crucial role in assessing the structural integrity, traffic safety, and long-term durability of concrete bridges, which represent the most common bridge type worldwide. Nevertheless, available datasets for the recognition of RCDs are small in terms of size and class variety, which questions their usability in real-world scenarios and their role as a benchmark. Our contribution to this problem is "dacl10k", an exceptionally diverse RCD dataset for multi-label semantic segmentation comprising 9,920 images deriving from real-world bridge inspections. dacl10k distinguishes 12 damage classes as well as 6 bridge components that play a key role in the building assessment and recommending actions, such as restoration works, traffic load limitations or bridge closures. In addition, we examine baseline models for dacl10k which are subsequently evaluated. The best model achieves a mean intersection-over-union of 0.42 on the test set. dacl10k, along with our baselines, will be openly accessible to researchers and practitioners, representing the currently biggest dataset regarding number of images and class diversity for semantic segmentation in the bridge inspection domain.Comment: 23 pages, 6 figure

Human-machine knowledge hybrid augmentation method for surface defect detection based few-data learning

Visual-based defect detection is a crucial but challenging task in industrial quality control. Most mainstream methods rely on large amounts of existing or related domain data as auxiliary information. However, in actual industrial production, there are often multi-batch, low-volume manufacturing scenarios with rapidly changing task demands, making it difficult to obtain sufficient and diverse defect data. This paper proposes a parallel solution that uses a human-machine knowledge hybrid augmentation method to help the model extract unknown important features. Specifically, by incorporating experts' knowledge of abnormality to create data with rich features, positions, sizes, and backgrounds, we can quickly accumulate an amount of data from scratch and provide it to the model as prior knowledge for few-data learning. The proposed method was evaluated on the magnetic tile dataset and achieved F1-scores of 60.73%, 70.82%, 77.09%, and 82.81% when using 2, 5, 10, and 15 training images, respectively. Compared to the traditional augmentation method's F1-score of 64.59%, the proposed method achieved an 18.22% increase in the best result, demonstrating its feasibility and effectiveness in few-data industrial defect detection.Comment: 24 pages, 15 figure

An Integrated Method for Optimizing Bridge Maintenance Plans

Bridges are one of the vital civil infrastructure assets, essential for economic developments and public welfare. Their large numbers, deteriorating condition, public demands for safe and efficient transportation networks and limited maintenance and intervention budgets pose a challenge, particularly when coupled with the need to respect environmental constraints. This state of affairs creates a wide gap between critical needs for intervention actions, and tight maintenance and rehabilitation funds. In an effort to meet this challenge, a newly developed integrated method for optimized maintenance and intervention plans for reinforced concrete bridge decks is introduced. The method encompasses development of five models: surface defects evaluation, corrosion severities evaluation, deterioration modeling, integrated condition assessment, and optimized maintenance plans. These models were automated in a set of standalone computer applications, coded using C#.net in Matlab environment. These computer applications were subsequently combined to form an integrated method for optimized maintenance and intervention plans. Four bridges and a dataset of bridge images were used in testing and validating the developed optimization method and its five models. The developed models have unique features and demonstrated noticeable performance and accuracy over methods used in practice and those reported in the literature. For example, the accuracy of the surface defects detection and evaluation model outperforms those of widely-recognized machine leaning and deep learning models; reducing detection, recognition and evaluation of surface defects error by 56.08%, 20.2% and 64.23%, respectively. The corrosion evaluation model comprises design of a standardized amplitude rating system that circumvents limitations of numerical amplitude-based corrosion maps. In the integrated condition, it was inferred that the developed model accomplished consistent improvement over the visual inspection procedures in-use by the Ministry of Transportation in Quebec. Similarly, the deterioration model displayed average enhancement in the prediction accuracies by 60% when compared against the most commonly-utilized weibull distribution. The performance of the developed multi-objective optimization model yielded 49% and 25% improvement over that of genetic algorithm in a five-year study period and a twenty five-year study period, respectively. At the level of thirty five-year study period, unlike the developed model, classical meta-heuristics failed to find feasible solutions within the assigned constraints. The developed integrated platform is expected to provide an efficient tool that enables decision makers to formulate sustainable maintenance plans that optimize budget allocations and ensure efficient utilization of resources