Research and optimization of YOLO-based method for automatic pavement defect detection

According to the latest statistics at the end of 2022, the total length of highways in China has reached 5.3548 million kilometers, with a maintenance mileage of 5.3503 million kilometers, accounting for 99.9% of the total maintenance coverage. Relying on inefficient manual pavement detection methods is difficult to meet the needs of large-scale detection. To tackle this issue, experiments were conducted to explore deep learning-based intelligent identification models, leveraging pavement distress data as the fundamental basis. The dataset encompasses pavement micro-cracks, which hold particular significance for the purpose of pavement preventive maintenance. The two-stage model Faster R-CNN achieved a mean average precision (mAP) of 0.938, which surpassed the one-stage object detection algorithms YOLOv5 (mAP: 0.91) and YOLOv7 (mAP: 0.932). To balance model weight and detection performance, this study proposes a YOLO-based optimization method on the basis of YOLOv5. This method achieves comparable detection performance (mAP: 0.93) to that of two-stage detectors, while exhibiting only a minimal increase in the number of parameters. Overall, the two-stage model demonstrated excellent detection performance when using a residual network (ResNet) as the backbone, whereas the YOLO algorithm of the one-stage detection model proved to be more suitable for practical engineering applications

D5.1 SHM digital twin requirements for residential, industrial buildings and bridges

This deliverable presents a report of the needs for structural control on buildings (initial imperfections, deflections at service, stability, rheology) and on bridges (vibrations, modal shapes, deflections, stresses) based on state-of-the-art image-based and sensor-based techniques. To this end, the deliverable identifies and describes strategies that encompass state-of-the-art instrumentation and control for infrastructures (SHM technologies).Objectius de Desenvolupament Sostenible::8 - Treball Decent i Creixement EconòmicObjectius de Desenvolupament Sostenible::9 - Indústria, Innovació i InfraestructuraPreprin

Machine-Learning Framework for Efficient Multi-Asset Rehabilitation Planning

While smart cities are viewed as the way of the future, the infrastructure assets expected to support the different smart services are currently managed using frameworks that are outdated, subjective, and inefficient. Such inefficiencies have led to huge maintenance and rehabilitation backlogs that are far beyond the financial capabilities of cities, municipalities and large asset owners like school boards. For example, the cost to bring Ontario schools facilities to an acceptable level of service is estimated to be as high as $16 billion. Currently, most “smart asset initiatives” are geared towards building new assets and using sensors to get periodic info about their condition, with little thought given regarding the condition of existing assets. As such, there is a need to introduce a “smart rehabilitation” framework that answers the question “how to bring the current infrastructure assets up to speed to satisfy the needs of current and future generations?”. To contribute to the overall vision of smart cities (data-driven interconnected services), the introduced framework uses machine learning and smart analytics to tackle three main functions of smart asset rehabilitation frameworks: (1) it automates the inspection and condition assessment processes by using convolutional neural networks (CNNs) to develop a machine learning system where defects can be automatically detected, classified, and quantified from images; (2) it uses data mining and clustering techniques to classify the assets according to their condition and need for repairs, and then uses optimization to select which assets are most worthy of immediate repairs subject to the existing funding constraints, thus enhancing the fund allocation phase by reducing its subjectivity; and (3) it uses novel computations, visualizations, and algorithms to facilitate cost-effective and fast-tracked delivery of the required rehabilitation works by considering them as units of a large repetitive project. To verify the strengths and versatility of the model, the proposed framework is applied to built-up roofs of educational buildings such as schools and university campuses. First, images were collected from the University of Waterloo campus buildings to develop the image-based analysis module; a two-step CNN framework that can detect damages and classify them according to their type. Information from the image-based analysis were then combined with textual information related to building age and description and unsupervised learning was applied to develop the prioritization and fund allocation module. Results from this module are used as the inputs to an optimization procedure where the overall performance of the entire asset portfolio is maximized by selecting which buildings should undergo immediate repairs, given strict budgetary constraints. Finally, the selected rehabilitation works were scheduled as units in a large repetitive project for delivery planning. Accordingly, novel computations and algorithms were developed to create compact schedules with minimal gaps that comply with deadline constraints, and novel visualizations were introduced to showcase the crews movements and the timing of all tasks required in each unit. The proposed framework offers powerful decision support features for a proposed smart rehabilitation layer to be included into the overall smart city vision. This framework deals with existing assets and provides objective assessments, cost-effective prioritization, and time-effective delivery plans. While this study used the case of built-up roofs as an example application, the framework is scalable towards other asset components as well as other assets in general. For example, components such as parking lots and concrete elements would rely heavily on the image-based inspection module, while other components such as HVAC systems would place more emphasis on the data analytics component, including more parameters related to different performance metrics as part of the analysis. Overall, this framework has the potential to revolutionize the multi-billion-dollar business of infrastructure renewal and provide cost effective decisions that save taxpayers’ money on the long run

Surface analysis and visualization from multi-light image collections

Multi-Light Image Collections (MLICs) are stacks of photos of a scene acquired with a fixed viewpoint and a varying surface illumination that provides large amounts of visual and geometric information. Over the last decades, a wide variety of methods have been devised to extract information from MLICs and have shown its use in different application domains to support daily activities. In this thesis, we present methods that leverage a MLICs for surface analysis and visualization. First, we provide background information: acquisition setup, light calibration and application areas where MLICs have been successfully used for the research of daily analysis work. Following, we discuss the use of MLIC for surface visualization and analysis and available tools used to support the analysis. Here, we discuss methods that strive to support the direct exploration of the captured MLIC, methods that generate relightable models from MLIC, non-photorealistic visualization methods that rely on MLIC, methods that estimate normal map from MLIC and we point out visualization tools used to do MLIC analysis. In chapter 3 we propose novel benchmark datasets (RealRTI, SynthRTI and SynthPS) that can be used to evaluate algorithms that rely on MLIC and discusses available benchmark for validation of photometric algorithms that can be also used to validate other MLIC-based algorithms. In chapter 4, we evaluate the performance of different photometric stereo algorithms using SynthPS for cultural heritage applications. RealRTI and SynthRTI have been used to evaluate the performance of (Neural)RTI method. Then, in chapter 5, we present a neural network-based RTI method, aka NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. In this method using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, particularly in the case of challenging glossy materials. Finally, in chapter 6, we present a method for the detection of crack on the surface of paintings from multi-light image acquisitions and that can be used as well on single images and conclude our presentation

3D Segmentation and Damage Analysis from Robotic Scans of Disaster Sites

Disaster relief and response plays an important role in saving lives and reducing economic loss after earthquakes, windstorm events and man-made explosions. Mobile robots represent an effective solution to assist in post-disaster reconnaissance in areas that are dangerous to human agents. These robots need an accurate 3D semantic map of the site in order to carry out disaster relief work such as search and rescue and damage assessment. Thus, there exists a research need to automatically identify building elements and detect structural damage from laser-scanned points clouds acquired by mobile robots. Current methods for point cloud semantic segmentation mostly perform direct class prediction at the point level without considering object-level semantics and generalizability across datasets. Moreover, current segmentation methods are unsuitable for real-time operation because they are designed to work as a post-processing step and do not process points from new scans in an online manner. This research proposes a learnable region growing method to perform class-agnostic point cloud segmentation in a data-driven and generalizable manner. In addition, an anomaly-based crack segmentation method is proposed where a deep feature embedding is used as a basis for separation between inlier and outlier points. Finally, an incremental segmentation scheme is used to process point cloud data in an online fashion and combine semantic information across multiple scans.Ph.D

Deep learning in the construction industry: A review of present status and future innovations

The construction industry is known to be overwhelmed with resource planning, risk management and logistic challenges which often result in design defects, project delivery delays, cost overruns and contractual disputes. These challenges have instigated research in the application of advanced machine learning algorithms such as deep learning to help with diagnostic and prescriptive analysis of causes and preventive measures. However, the publicity created by tech firms like Google, Facebook and Amazon about Artificial Intelligence and applications to unstructured data is not the end of the field. There abound many applications of deep learning, particularly within the construction sector in areas such as site planning and management, health and safety and construction cost prediction, which are yet to be explored. The overall aim of this article was to review existing studies that have applied deep learning to prevalent construction challenges like structural health monitoring, construction site safety, building occupancy modelling and energy demand prediction. To the best of our knowledge, there is currently no extensive survey of the applications of deep learning techniques within the construction industry. This review would inspire future research into how best to apply image processing, computer vision, natural language processing techniques of deep learning to numerous challenges in the industry. Limitations of deep learning such as the black box challenge, ethics and GDPR, cybersecurity and cost, that can be expected by construction researchers and practitioners when adopting some of these techniques were also discussed

Symmetry in Structural Health Monitoring

In this Special Issue on symmetry, we mainly discuss the application of symmetry in various structural health monitoring. For example, considering the health monitoring of a known structure, by obtaining the static or dynamic response of the structure, using different signal processing methods, including some advanced filtering methods, to remove the influence of environmental noise, and extract structural feature parameters to determine the safety of the structure. These damage diagnosis methods can also be effectively applied to various types of infrastructure and mechanical equipment. For this reason, the vibration control of various structures and the knowledge of random structure dynamics should be considered, which will promote the rapid development of the structural health monitoring. Among them, signal extraction and evaluation methods are also worthy of study. The improvement of signal acquisition instruments and acquisition methods improves the accuracy of data. A good evaluation method will help to correctly understand the performance with different types of infrastructure and mechanical equipment

APPLICATIONS OF INNOVATIVE BUILDING MATERIAL AND COMPUTER VISION METHODS IN GEOTECHNICAL ENGINEERING

Ph.D