Analytical study of computer vision-based pavement crack quantification using machine learning techniques

Image-based techniques are a promising non-destructive approach for road pavement condition evaluation. The main objective of this study is to extract, quantify and evaluate important surface defects, such as cracks, using an automated computer vision-based system to provide a better understanding of the pavement deterioration process. To achieve this objective, an automated crack-recognition software was developed, employing a series of image processing algorithms of crack extraction, crack grouping, and crack detection. Bottom-hat morphological technique was used to remove the random background of pavement images and extract cracks, selectively based on their shapes, sizes, and intensities using a relatively small number of user-defined parameters. A technical challenge with crack extraction algorithms, including the Bottom-hat transform, is that extracted crack pixels are usually fragmented along crack paths. For de-fragmenting those crack pixels, a novel crack-grouping algorithm is proposed as an image segmentation method, so called MorphLink-C. Statistical validation of this method using flexible pavement images indicated that MorphLink-C not only improves crack-detection accuracy but also reduces crack detection time. Crack characterization was performed by analysing imagerial features of the extracted crack image components. A comprehensive statistical analysis was conducted using filter feature subset selection (FSS) methods, including Fischer score, Gini index, information gain, ReliefF, mRmR, and FCBF to understand the statistical characteristics of cracks in different deterioration stages. Statistical significance of crack features was ranked based on their relevancy and redundancy. The statistical method used in this study can be employed to avoid subjective crack rating based on human visual inspection. Moreover, the statistical information can be used as fundamental data to justify rehabilitation policies in pavement maintenance. Finally, the application of four classification algorithms, including Artificial Neural Network (ANN), Decision Tree (DT), k-Nearest Neighbours (kNN) and Adaptive Neuro-Fuzzy Inference System (ANFIS) is investigated for the crack detection framework. The classifiers were evaluated in the following five criteria: 1) prediction performance, 2) computation time, 3) stability of results for highly imbalanced datasets in which, the number of crack objects are significantly smaller than the number of non-crack objects, 4) stability of the classifiers performance for pavements in different deterioration stages, and 5) interpretability of results and clarity of the procedure. Comparison results indicate the advantages of white-box classification methods for computer vision based pavement evaluation. Although black-box methods, such as ANN provide superior classification performance, white-box methods, such as ANFIS, provide useful information about the logic of classification and the effect of feature values on detection results. Such information can provide further insight for the image-based pavement crack detection application

Adaptive Road Crack Detection System by Pavement Classification

This paper presents a road distress detection system involving the phases needed to properly deal with fully automatic road distress assessment. A vehicle equipped with line scan cameras, laser illumination and acquisition HW-SW is used to storage the digital images that will be further processed to identify road cracks. Pre-processing is firstly carried out to both smooth the texture and enhance the linear features. Non-crack features detection is then applied to mask areas of the images with joints, sealed cracks and white painting, that usually generate false positive cracking. A seed-based approach is proposed to deal with road crack detection, combining Multiple Directional Non-Minimum Suppression (MDNMS) with a symmetry check. Seeds are linked by computing the paths with the lowest cost that meet the symmetry restrictions. The whole detection process involves the use of several parameters. A correct setting becomes essential to get optimal results without manual intervention. A fully automatic approach by means of a linear SVM-based classifier ensemble able to distinguish between up to 10 different types of pavement that appear in the Spanish roads is proposed. The optimal feature vector includes different texture-based features. The parameters are then tuned depending on the output provided by the classifier. Regarding non-crack features detection, results show that the introduction of such module reduces the impact of false positives due to non-crack features up to a factor of 2. In addition, the observed performance of the crack detection system is significantly boosted by adapting the parameters to the type of pavement

Machine learning algorithms for monitoring pavement performance

ABSTRACT: This work introduces the need to develop competitive, low-cost and applicable technologies to real roads to detect the asphalt condition by means of Machine Learning (ML) algorithms. Specifically, the most recent studies are described according to the data collection methods: images, ground penetrating radar (GPR), laser and optic fiber. The main models that are presented for such state-of-the-art studies are Support Vector Machine, Random Forest, Naïve Bayes, Artificial neural networks or Convolutional Neural Networks. For these analyses, the methodology, type of problem, data source, computational resources, discussion and future research are highlighted. Open data sources, programming frameworks, model comparisons and data collection technologies are illustrated to allow the research community to initiate future investigation. There is indeed research on ML-based pavement evaluation but there is not a widely used applicability by pavement management entities yet, so it is mandatory to work on the refinement of models and data collection methods

Deep Learning Approaches in Pavement Distress Identification: A Review

This paper presents a comprehensive review of recent advancements in image processing and deep learning techniques for pavement distress detection and classification, a critical aspect in modern pavement management systems. The conventional manual inspection process conducted by human experts is gradually being superseded by automated solutions, leveraging machine learning and deep learning algorithms to enhance efficiency and accuracy. The ability of these algorithms to discern patterns and make predictions based on extensive datasets has revolutionized the domain of pavement distress identification. The paper investigates the integration of unmanned aerial vehicles (UAVs) for data collection, offering unique advantages such as aerial perspectives and efficient coverage of large areas. By capturing high-resolution images, UAVs provide valuable data that can be processed using deep learning algorithms to detect and classify various pavement distresses effectively. While the primary focus is on 2D image processing, the paper also acknowledges the challenges associated with 3D images, such as sensor limitations and computational requirements. Understanding these challenges is crucial for further advancements in the field. The findings of this review significantly contribute to the evolution of pavement distress detection, fostering the development of efficient pavement management systems. As automated approaches continue to mature, the implementation of deep learning techniques holds great promise in ensuring safer and more durable road infrastructure for the benefit of society

Automatic detection of concrete cracks from images using Adam-SqueezeNet deep learning model

Cracks on concrete surface are typically clear warning signs of a potential threat to the integrity and serviceability of structure. The techniques based on image processing can effectively detect the cracks from images. These techniques, however, are generally susceptible to user-driven heuristic thresholds and extraneous distractors. Inspired by recent success of artificial intelligence, a deep learning based automated crack detection system called CrackSN was developed. An image dataset of concrete surface is collected by smartphone and carefully prepared in order to develop and train the CrackSN system. This proposed deep learning model, built on the Adam-SqueezeNet architecture, automatically learns the discriminative feature directly from the labeled and augmented patches. Hyperparameters of SqueezeNet are tuned with Adam optimization additive through the training and validation procedures. The fine-tuned CrackSN model outperforms state-of-the-art models in recent literature by correctly classifying 97.3% of the cracked patches in the image dataset. The success of CrackSN model demonstrated with light network design and outstanding performance provides a key step toward automated damage inspection and health evaluation for infrastructure. &nbsp

An Exploration of Recent Intelligent Image Analysis Techniques for Visual Pavement Surface Condition Assessment.

Road pavement condition assessment is essential for maintenance, asset management, and budgeting for pavement infrastructure. Countries allocate a substantial annual budget to maintain and improve local, regional, and national highways. Pavement condition is assessed by measuring several pavement characteristics such as roughness, surface skid resistance, pavement strength, deflection, and visual surface distresses. Visual inspection identifies and quantifies surface distresses, and the condition is assessed using standard rating scales. This paper critically analyzes the research trends in the academic literature, professional practices and current commercial solutions for surface condition ratings by civil authorities. We observe that various surface condition rating systems exist, and each uses its own defined subset of pavement characteristics to evaluate pavement conditions. It is noted that automated visual sensing systems using intelligent algorithms can help reduce the cost and time required for assessing the condition of pavement infrastructure, especially for local and regional road networks. However, environmental factors, pavement types, and image collection devices are significant in this domain and lead to challenging variations. Commercial solutions for automatic pavement assessment with certain limitations exist. The topic is also a focus of academic research. More recently, academic research has pivoted toward deep learning, given that image data is now available in some form. However, research to automate pavement distress assessment often focuses on the regional pavement condition assessment standard that a country or state follows. We observe that the criteria a region adopts to make the evaluation depends on factors such as pavement construction type, type of road network in the area, flow and traffic, environmental conditions, and region\u27s economic situation. We summarized a list of publicly available datasets for distress detection and pavement condition assessment. We listed approaches focusing on crack segmentation and methods concentrating on distress detection and identification using object detection and classification. We segregated the recent academic literature in terms of the camera\u27s view and the dataset used, the year and country in which the work was published, the F1 score, and the architecture type. It is observed that the literature tends to focus more on distress identification ( presence/absence detection) but less on distress quantification, which is essential for developing approaches for automated pavement rating

What's cracking? A review and analysis of deep learning methods for structural crack segmentation, detection and quantification

Surface cracks are a very common indicator of potential structural faults. Their early detection and monitoring is an important factor in structural health monitoring. Left untreated, they can grow in size over time and require expensive repairs or maintenance. With recent advances in computer vision and deep learning algorithms, the automatic detection and segmentation of cracks for this monitoring process have become a major topic of interest. This review aims to give researchers an overview of the published work within the field of crack analysis algorithms that make use of deep learning. It outlines the various tasks that are solved through applying computer vision algorithms to surface cracks in a structural health monitoring setting and also provides in-depth reviews of recent fully, semi and unsupervised approaches that perform crack classification, detection, segmentation and quantification. Additionally, this review also highlights popular datasets used for cracks and the metrics that are used to evaluate the performance of those algorithms. Finally, potential research gaps are outlined and further research directions are provided

A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment

Identification of Top-down, Bottom-up, and Cement-Treated Reflective Cracks Using Convolutional Neural Network and Artificial Neural Network

The objective of this study was to formulate a Convolutional Neural Networks (CNN) model and to develop a decision-making tool using Artificial Neural Networks (ANN) to identify top-down, bottom-up, and cement treated (CT) reflective cracking in in-service flexible pavements. The CNN’s architecture consisted of five convolutional layers with three max-pooling layers and three fully connected layers. Input variables for the ANN model were pavement age, asphalt concrete (AC) thickness, annual average daily traffic (AADT), type of base, crack orientation, and crack location. The ANN network architecture consisted of an input layer of six neurons, a hidden layer of ten neurons, and a target layer of three neurons. The developed CNN model was found to achieve an accuracy of 93.8% and 91.0% in the testing and validation phases, respectively. The ANN based decision-making tool achieved an overall accuracy of 92% indicating its effectiveness in crack identification and classification. In the second phase of the study, the flexible pavement responses under a dual tire assembly were analyzed to identify the critical stress mechanisms for bottom-up and top-down cracking. Higher tensile strains were observed to occur underneath the tire ribs than away from them supporting the argument that both surface initiated and bottom-up fatigue cracking develop in or near the wheel paths. The incorporation of surface transverse tangential stresses increased the surface tensile strains near the tire ribs by approximately 68%, 63%, and 53% respectively for low, medium, and high volume flexible pavements indicating an increased potential for the initiation and development of top-down cracking when tangential stresses are considered. In contrast, this effect was observed to be minimal for the tensile strains at the bottom of the asphalt layer, which are the main pavement responses used in the prediction of fatigue cracking. Shrinkage cracking in cement treated base (CTB) was also modeled in finite element using displacement boundary conditions. The tensile stresses due to shrinkage strains in the cement treated base were observed to be comparable to the tensile strength of CTB at 7 days and higher at 56 days indicating the potential development of shrinkage cracks

A neutral network approach to automated classification of cracks in images of highway pavement using vectorized data

The collection and interpretation of highway pavement distress data is an expensive process. This process, in its traditional sense, required many man hours, as technicians have to survey the highway. This process is also quite dangerous and subject to traffic and weather conditions. Much research has been performed in order to automate the collection and interpretation phases. This research focuses on an alternative methodology for classification of highway pavement distresses. The Learning Vector Quantizer, an artificial neural network architecture, was implemented to classify images of highway pavement according to the longitudinal, transverse, fatigue and block crack types. Data was recorded from vectorized crack segments produced by a segmentation process. The images were recursively broken into smaller blocks, forming a quad-tree type structure. Each block in the image tree was then classified as to the major distress prevalent. This technique should allow for easier measurement of extent and severity. Also it allows records to be kept on individual pieces of highway for comparison with future monitoring. The overall results of the classification were satisfactory, with longitudinal and transverse classification outperforming block and fatigue. This was to be expected as block and fatigue crack types are harder to differentiate. With additional research, this approach should prove to be very beneficial to the pavement management field