Optimized deep encoder-decoder methods for crack segmentation

Continuous maintenance of concrete infrastructure is an important task which is needed to continue safe operations of these structures. One kind of defect that occurs on surfaces in these structures are cracks. Automatic detection of those cracks poses a challenging computer vision task as background, shape, colour and size of cracks vary. In this work we propose optimized deep encoder-decoder methods consisting of a combination of techniques which yield an increase in crack segmentation performance. Specifically, we propose a new design for the decoder-part in encoder-decoder based deep learning architectures for semantic segmentation. We study its composition and how to achieve increased performance by exploring components such as deep supervision and upsampling strategies. Then we examine the optimal encoder to go in conjunction with this decoder and determine that pretrained encoders lead to an increase in performance. We propose a data augmentation strategy to increase the amount of available training data and carry out the performance evaluation of the designed architecture on four publicly available crack segmentation datasets. Additionally, we introduce two techniques into the field of surface crack segmentation, previously not used there: Generating results using test-time-augmentation and performing a statistical result analysis over multiple training runs. The former approach generally yields increased performance results, whereas the latter allows for more reproducible and better representability of a methods results. Using those aforementioned strategies with our proposed encoder-decoder architecture we are able to achieve new state of the art results in all datasets

Developing an Efficient Real-Time Terrestrial Infrastructure Inspection System Using Autonomous Drones and Deep Learning

Unmanned aerial vehicles (UAV), commonly referred to as drones (Dynamic Remotely Operated Navigation Equipment), show promise for deploying regular, automated structural inspections remotely. Deep learning has shown great potential for robustly detecting structural faults from collected images, through convolutional neural networks (CNN). However, running computationally demanding tasks (such as deep learning algorithms) on-board drones is difficult due to on-board memory and processing constraints. Moreover, the potential for fully automating drone navigation for structural data collection while optimizing deep learning models deployed to computationally constrained on-board processing units has yet to be realized for infrastructure inspection. Thus, an efficient, fully autonomous drone infrastructure inspection system is introduced. Using inertial sensors, mounted time-of-flight (ToF) and optical sensors to calculate distance readings for obstacle avoidance, a drone can autonomously track around structures. The drone can localize and extract faults in real-time on low-power processing units, through pixel-wise segmentation of faults from structural images collected by an on-board digital camera. Furthermore, proposed modifications to a CNN-based U-Net architecture show notable improvements to the baseline U-Net, in terms of pixel-wise segmentation accuracy and efficiency on computationally constrained on-board devices. After fault segmentation, the fault points corresponding to the predicted fault pixels are passed into a custom fault tracking algorithm; based on a robust line estimation technique, modifications are proposed using a quadtree data structure and a smart sampling approach. Using this approach, the drone is capable of following along faults robustly and efficiently during inspection to better gauge the extent of the spread of the faults

A Convolutional-Transformer Network for Crack Segmentation with Boundary Awareness

Cracks play a crucial role in assessing the safety and durability of manufactured buildings. However, the long and sharp topological features and complex background of cracks make the task of crack segmentation extremely challenging. In this paper, we propose a novel convolutional-transformer network based on encoder-decoder architecture to solve this challenge. Particularly, we designed a Dilated Residual Block (DRB) and a Boundary Awareness Module (BAM). The DRB pays attention to the local detail of cracks and adjusts the feature dimension for other blocks as needed. And the BAM learns the boundary features from the dilated crack label. Furthermore, the DRB is combined with a lightweight transformer that captures global information to serve as an effective encoder. Experimental results show that the proposed network performs better than state-of-the-art algorithms on two typical datasets. Datasets, code, and trained models are available for research at https://github.com/HqiTao/CT-crackseg

What's cracking? A review and analysis of deep learning methods for structural crack segmentation, detection and quantification

Surface cracks are a very common indicator of potential structural faults. Their early detection and monitoring is an important factor in structural health monitoring. Left untreated, they can grow in size over time and require expensive repairs or maintenance. With recent advances in computer vision and deep learning algorithms, the automatic detection and segmentation of cracks for this monitoring process have become a major topic of interest. This review aims to give researchers an overview of the published work within the field of crack analysis algorithms that make use of deep learning. It outlines the various tasks that are solved through applying computer vision algorithms to surface cracks in a structural health monitoring setting and also provides in-depth reviews of recent fully, semi and unsupervised approaches that perform crack classification, detection, segmentation and quantification. Additionally, this review also highlights popular datasets used for cracks and the metrics that are used to evaluate the performance of those algorithms. Finally, potential research gaps are outlined and further research directions are provided

Ensemble of deep convolutional neural networks for automatic pavement crack detection and measurement

Automated pavement crack detection and measurement are important road issues. Agencies have to guarantee the improvement of road safety. Conventional crack detection and measurement algorithms can be extremely time-consuming and low efficiency. Therefore, recently, innovative algorithms have received increased attention from researchers. In this paper, we propose an ensemble of convolutional neural networks (without a pooling layer) based on probability fusion for automated pavement crack detection and measurement. Specifically, an ensemble of convolutional neural networks was employed to identify the structure of small cracks with raw images. Secondly, outputs of the individual convolutional neural network model for the ensemble were averaged to produce the final crack probability value of each pixel, which can obtain a predicted probability map. Finally, the predicted morphological features of the cracks were measured by using the skeleton extraction algorithm. To validate the proposed method, some experiments were performed on two public crack databases (CFD and AigleRN) and the results of the different state-of-the-art methods were compared. To evaluate the efficiency of crack detection methods, three parameters were considered: precision (Pr), recall (Re) and F1 score (F1). For the two public databases of pavement images, the proposed method obtained the highest values of the three evaluation parameters: for the CFD database, Pr = 0.9552, Re = 0.9521 and F1 = 0.9533 (which reach values up to 0.5175 higher than the values obtained on the same database with the other methods), for the AigleRN database, Pr = 0.9302, Re = 0.9166 and F1 = 0.9238 (which reach values up to 0.7313 higher than the values obtained on the same database with the other methods). The experimental results show that the proposed method outperforms the other methods. For crack measurement, the crack length and width can be measure based on different crack types (complex, common, thin, and intersecting cracks.). The results show that the proposed algorithm can be effectively applied for crack measurement

Weakly-supervised surface crack segmentation by generating pseudo-labels using localization with a classifier and thresholding

Surface cracks are a common sight on public infrastructure nowadays. Recent work has been addressing this problem by supporting structural maintenance measures using machine learning methods. Those methods are used to segment surface cracks from their background, making them easier to localize. However, a common issue is that to create a well-functioning algorithm, the training data needs to have detailed annotations of pixels that belong to cracks. Our work proposes a weakly supervised approach that leverages a CNN classifier in a novel way to create surface crack pseudo labels. First, we use the classifier to create a rough crack localization map by using its class activation maps and a patch based classification approach and fuse this with a thresholding based approach to segment the mostly darker crack pixels. The classifier assists in suppressing noise from the background regions, which commonly are incorrectly highlighted as cracks by standard thresholding methods. Then, the pseudo labels can be used in an end-to-end approach when training a standard CNN for surface crack segmentation. Our method is shown to yield sufficiently accurate pseudo labels. Those labels, incorporated into segmentation CNN training using multiple recent crack segmentation architectures, achieve comparable performance to fully supervised methods on four popular crack segmentation datasets.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl