Pavement Defect Classification and Localization Using Hybrid Weakly Supervised and Supervised Deep Learning and GIS

Automated detection of road defects has historically been challenging for the pavement management industry. As a result, new methods have been developed over the past few years to handle this issue. Most of these methods relied on supervised machine learning techniques, such as object detection and segmentation methods, which need a large, annotated image dataset to train their models. However, annotating pavement defects is difficult and time-consuming due to their ununiformed and complex shapes. To address this challenge, a hybrid pavement defect classification and localization framework using weakly supervised and supervised deep learning methods is proposed in this thesis. This framework has two steps: (1) A robust hierarchical two-level classifier that classifies the defects in images, and (2) A method for defect localization combining weakly supervised and supervised techniques. In the localization method, first, defects are primarily localized using a weakly supervised method (i.e. Class Activation Mapping (CAM)). Next, based on the results of the first classifiers, the defects are segmented from the localized patches obtained in the previous step. The feature maps extracted from the CAM method are used to train a segmentation network once (i.e. U-Net or Mask R-CNN) to localize and segment the defects in the images. Thus, the proposed framework combines the advantages of weakly supervised and supervised methods. The supervised modules in the framework are trained once and can be used for any new data without the need to train. In other words, to use our framework on new dataset only the classifiers should be fine-tuned. Furthermore, the proposed framework introduced an innovative method designed to calculate the maximum crack width in pixels within linear segmented defect patches, derived from the localization module of the proposed framework. This method is particularly advantageous as it provides critical information that can be further employed in the calculation of the Pavement Condition Index (PCI). Additionally, the proposed method benefits from an asset management inspection system based on Geographic Information System (GIS) technology to prepare the dataset used in the training and testing. Thus, this advanced system serves a dual role within our framework. Firstly, it assists in the assembly and preparation of the dataset used in the model training process, providing a geographically organized collection of images and related data. Secondly, it plays a crucial role in the testing phase, offering a spatially accurate platform for evaluating the effectiveness of the model in real-world scenarios. A dataset from Georgia State in the USA was used in the case study. The proposed framework obtained high precision of 97%, 88%, 92% and 97% for localizing the alligator, block, longitudinal and transverse cracks, respectively. Considering all factors, such as annotation cost, and performance on the test dataset, the proposed localization method outperforms the supervised localization methods, such as instance segmentation and object detection for localizing road pavement defect

Revisión de métodos para la clasificación de fallas superficiales en pavimentos flexibles

The status of the road infrastructure affects the social, economic, and political environment of a nation. Evaluation of the pavement surface condition is essential to plan timely and effective interventions. Timely actions avoid operating cost overruns, prevent uncontrolled deterioration and reduce operational and safety inconveniences. The problem raises the concern of studying alternatives to evaluate the status of pavement, for which a large number of investigations on automatic detection of surface flaws in flexible pavements through image processing techniques have been developed. The objective of this article is to review and analyze these contributions. Based on the review, it was concluded that the performance of this type of systems is determined by two factors: data collection and processing. The analysis presented herein unfolds based on these factors. The development of systems that take advantage of the qualities of different sensors in data acquisition and that integrate the detection and classification of a variety of faults including severity data is considered opportune.El estado de la infraestructura vial impacta el entorno social, económico y político de una nación. La evaluación de la condición superficial del pavimento es esencial para planificar intervenciones oportunas y eficaces. Las acciones oportunas evitan sobrecostos de operación, impiden el deterioro no controlado y disminuyen los inconvenientes operacionales y de seguridad. El problema expuesto plantea la inquietud de estudiar alternativas para evaluar el estado del pavimento, por lo cual un gran número de investigaciones sobre detección automática de fallas superficiales en pavimentos flexibles a través de técnicas de procesamiento de imágenes han sido desarrolladas. El objetivo de este artículo es revisar y analizar estos aportes. Sobre la base de la revisión, se concluyó que el rendimiento de este tipo de sistemas está determinado por dos factores: la recopilación de los datos y su procesamiento. El análisis presentado se despliega en función de estos factores. Se considera oportuno el desarrollo de sistemas que aprovechen las cualidades de diferentes sensores en la adquisición de datos y que integren la detección y clasificación de variedad de fallas incluyendo datos de severidad

Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics

Three recent breakthroughs due to AI in arts and science serve as motivation: An award winning digital image, protein folding, fast matrix multiplication. Many recent developments in artificial neural networks, particularly deep learning (DL), applied and relevant to computational mechanics (solid, fluids, finite-element technology) are reviewed in detail. Both hybrid and pure machine learning (ML) methods are discussed. Hybrid methods combine traditional PDE discretizations with ML methods either (1) to help model complex nonlinear constitutive relations, (2) to nonlinearly reduce the model order for efficient simulation (turbulence), or (3) to accelerate the simulation by predicting certain components in the traditional integration methods. Here, methods (1) and (2) relied on Long-Short-Term Memory (LSTM) architecture, with method (3) relying on convolutional neural networks. Pure ML methods to solve (nonlinear) PDEs are represented by Physics-Informed Neural network (PINN) methods, which could be combined with attention mechanism to address discontinuous solutions. Both LSTM and attention architectures, together with modern and generalized classic optimizers to include stochasticity for DL networks, are extensively reviewed. Kernel machines, including Gaussian processes, are provided to sufficient depth for more advanced works such as shallow networks with infinite width. Not only addressing experts, readers are assumed familiar with computational mechanics, but not with DL, whose concepts and applications are built up from the basics, aiming at bringing first-time learners quickly to the forefront of research. History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics, even in well-known references. Positioning and pointing control of a large-deformable beam is given as an example.Comment: 275 pages, 158 figures. Appeared online on 2023.03.01 at CMES-Computer Modeling in Engineering & Science