Classification, Localization, and Quantification of Structural Damage in Concrete Structures using Convolutional Neural Networks

Applications of Machine Learning (ML) algorithms in Structural Health Monitoring (SHM) have recently become of great interest owing to their superior ability to detect damage in engineering structures. ML algorithms used in this domain are classified into two major subfields: vibration-based and image-based SHM. Traditional condition survey techniques based on visual inspection have been the most widely used for monitoring concrete structures in service. Inspectors visually evaluate defects based on experience and engineering judgment. However, this process is subjective, time-consuming, and hampered by difficult access to numerous parts of complex structures. Accordingly, the present study proposes a nearly automated inspection model based on image processing, signal processing, and deep learning for detecting defects and identifying damage locations in typically inaccessible areas of concrete structures. The work conducted in this thesis achieved excellent damage localization and classification performance and could offer a nearly automated inspection platform for the colossal backlog of ageing civil engineering structures

Generating bridge geometric digital twins from point clouds

The automation of digital twinning for existing bridges from point clouds remains unresolved. Previous research yielded methods that can generate surface primitives combined with rule-based classification to create labelled cuboids and cylinders. While these methods work well in synthetic datasets or simplified cases, they encounter huge challenges when dealing with real-world point clouds. The proposed framework employs bridge engineering knowledge that mimics the intelligence of human modellers to detect and model reinforced concrete bridge objects in imperfect point clouds. Experiments on ten bridge point clouds indicate the framework can achieve high and reliable performance of geometric digital twin generation of existing bridges.This research is funded by EPSRC, EU Infravation SeeBridge project under Grant No. 31109806.0007 and Trimble Research Fun

Surface and Sub-Surface Analyses for Bridge Inspection

The development of bridge inspection solutions has been discussed in the recent past. In this dissertation, significant development and improvement on the state-of-the-art in the field of bridge inspection using multiple sensors (e.g. ground penetrating radar (GPR) and visual sensor) has been proposed. In the first part of this research (discussed in chapter 3), the focus is towards developing effective and novel methods for rebar detection and localization for sub-surface bridge inspection of steel rebars. The data has been collected using Ground Penetrating Radar (GPR) sensor on real bridge decks. In this regard, a number of different approaches have been successively developed that continue to improve the state-of-the-art in this particular research area. The second part (discussed in chapter 4) of this research deals with the development of an automated system for steel bridge defect detection system using a Multi-Directional Bicycle Robot. The training data has been acquired from actual bridges in Vietnam and validation is performed on data collected using Bicycle Robot from actual bridge located in Highway-80, Lovelock, Nevada, USA. A number of different proposed methods have been discussed in chapter 4. The final chapter of the dissertation will conclude the findings from the different parts and discuss ways of improving on the existing works in the near future

Online hybrid learning methods for real-time structural health monitoring using remote sensing and small displacement data

Structural health monitoring (SHM) by using remote sensing and synthetic aperture radar (SAR) images is a promising approach to assessing the safety and the integrity of civil structures. Apart from this issue, artificial intelligence and machine learning have brought great opportunities to SHM by learning an automated computational model for damage detection. Accordingly, this article proposes online hybrid learning methods to firstly deal with some major challenges in data-driven SHM and secondly detect damage via small displacement data from SAR images in a real-time manner. The proposed methods contain three main parts: (i) data augmentation by Hamiltonian Monte Carlo and slice sampling for addressing the problem of small displacement data, (ii) data normalization by an online deep transfer learning algorithm for removing the effects of environmental and/or operational variability from augmented data, and (iii) feature classification via a scalar novelty score. The major contributions of this research include proposing two online hybrid unsupervised learning methods and providing effective frameworks for online damage detection. A small set of displacement samples extracted from SAR images of TerraSar-X regarding a long-term monitoring scheme of the Tadcaster Bridge in United Kingdom is applied to validate the proposed methods

Anomaly detection and automatic labeling for solar cell quality inspection based on Generative Adversarial Network

Quality inspection applications in industry are required to move towards a zero-defect manufacturing scenario, withnon-destructive inspection and traceability of 100 % of produced parts. Developing robust fault detection and classification modelsfrom the start-up of the lines is challenging due to the difficulty in getting enough representative samples of the faulty patternsand the need to manually label them. This work presents a methodology to develop a robust inspection system, targeting thesepeculiarities, in the context of solar cell manufacturing. The methodology is divided into two phases: In the first phase, an anomalydetection model based on a Generative Adversarial Network (GAN) is employed. This model enables the detection and localizationof anomalous patterns within the solar cells from the beginning, using only non-defective samples for training and without anymanual labeling involved. In a second stage, as defective samples arise, the detected anomalies will be used as automaticallygenerated annotations for the supervised training of a Fully Convolutional Network that is capable of detecting multiple types offaults. The experimental results using 1873 EL images of monocrystalline cells show that (a) the anomaly detection scheme can beused to start detecting features with very little available data, (b) the anomaly detection may serve as automatic labeling in order totrain a supervised model, and (c) segmentation and classification results of supervised models trained with automatic labels arecomparable to the ones obtained from the models trained with manual labels.Comment: 20 pages, 10 figures, 6 tables. This article is part of the special issue "Condition Monitoring, Field Inspection and Fault Diagnostic Methods for Photovoltaic Systems" Published in MDPI - Sensors: see https://www.mdpi.com/journal/sensors/special_issues/Condition_Monitoring_Field_Inspection_and_Fault_Diagnostic_Methods_for_Photovoltaic_System

Machine-Learning Framework for Efficient Multi-Asset Rehabilitation Planning

While smart cities are viewed as the way of the future, the infrastructure assets expected to support the different smart services are currently managed using frameworks that are outdated, subjective, and inefficient. Such inefficiencies have led to huge maintenance and rehabilitation backlogs that are far beyond the financial capabilities of cities, municipalities and large asset owners like school boards. For example, the cost to bring Ontario schools facilities to an acceptable level of service is estimated to be as high as $16 billion. Currently, most “smart asset initiatives” are geared towards building new assets and using sensors to get periodic info about their condition, with little thought given regarding the condition of existing assets. As such, there is a need to introduce a “smart rehabilitation” framework that answers the question “how to bring the current infrastructure assets up to speed to satisfy the needs of current and future generations?”. To contribute to the overall vision of smart cities (data-driven interconnected services), the introduced framework uses machine learning and smart analytics to tackle three main functions of smart asset rehabilitation frameworks: (1) it automates the inspection and condition assessment processes by using convolutional neural networks (CNNs) to develop a machine learning system where defects can be automatically detected, classified, and quantified from images; (2) it uses data mining and clustering techniques to classify the assets according to their condition and need for repairs, and then uses optimization to select which assets are most worthy of immediate repairs subject to the existing funding constraints, thus enhancing the fund allocation phase by reducing its subjectivity; and (3) it uses novel computations, visualizations, and algorithms to facilitate cost-effective and fast-tracked delivery of the required rehabilitation works by considering them as units of a large repetitive project. To verify the strengths and versatility of the model, the proposed framework is applied to built-up roofs of educational buildings such as schools and university campuses. First, images were collected from the University of Waterloo campus buildings to develop the image-based analysis module; a two-step CNN framework that can detect damages and classify them according to their type. Information from the image-based analysis were then combined with textual information related to building age and description and unsupervised learning was applied to develop the prioritization and fund allocation module. Results from this module are used as the inputs to an optimization procedure where the overall performance of the entire asset portfolio is maximized by selecting which buildings should undergo immediate repairs, given strict budgetary constraints. Finally, the selected rehabilitation works were scheduled as units in a large repetitive project for delivery planning. Accordingly, novel computations and algorithms were developed to create compact schedules with minimal gaps that comply with deadline constraints, and novel visualizations were introduced to showcase the crews movements and the timing of all tasks required in each unit. The proposed framework offers powerful decision support features for a proposed smart rehabilitation layer to be included into the overall smart city vision. This framework deals with existing assets and provides objective assessments, cost-effective prioritization, and time-effective delivery plans. While this study used the case of built-up roofs as an example application, the framework is scalable towards other asset components as well as other assets in general. For example, components such as parking lots and concrete elements would rely heavily on the image-based inspection module, while other components such as HVAC systems would place more emphasis on the data analytics component, including more parameters related to different performance metrics as part of the analysis. Overall, this framework has the potential to revolutionize the multi-billion-dollar business of infrastructure renewal and provide cost effective decisions that save taxpayers’ money on the long run

Efficient Experimental and Data-Centered Workflow for Microstructure-Based Fatigue Data – Towards a Data Basis for Predictive AI Models

Background Early fatigue mechanisms for various materials are yet to be unveiled for the (very) high-cycle fatigue (VHCF) regime. This can be ascribed to a lack of available data capturing initial fatigue damage evolution, which continues to adversely affect data scientists and computational modeling experts attempting to derive microstructural dependencies from small sample size data and incomplete feature representations. Objective The aim of this work is to address this lack and to drive the digital transformation of materials such that future virtual component design can be rendered more reliable and more efficient. Achieving this relies on fatigue models that comprehensively capture all relevant dependencies. Methods To this end, this work proposes a combined experimental and data post-processing workflow to establish multimodal fatigue crack initiation and propagation data sets efficiently. It evolves around fatigue testing of mesoscale specimens to increase damage detection sensitivity, data fusion through multimodal registration to address data heterogeneity, and image-based data-driven damage localization. Results A workflow with a high degree of automation is established, that links large distortion-corrected microstructure data with damage localization and evolution kinetics. The workflow enables cycling up to the VHCF regime in comparatively short time spans, while maintaining unprecedented time resolution of damage evolution. Resulting data sets capture the interaction of damage with microstructural features and hold the potential to unravel a mechanistic understanding. Conclusions The proposed workflow lays the foundation for future data mining and data-driven modeling of microstructural fatigue by providing statistically meaningful data sets extendable to a wide range of materials

Artificial intelligence in construction asset management: a review of present status, challenges and future opportunities

The built environment is responsible for roughly 40% of global greenhouse emissions, making the sector a crucial factor for climate change and sustainability. Meanwhile, other sectors (like manufacturing) adopted Artificial Intelligence (AI) to solve complex, non-linear problems to reduce waste, inefficiency, and pollution. Therefore, many research efforts in the Architecture, Engineering, and Construction community have recently tried introducing AI into building asset management (AM) processes. Since AM encompasses a broad set of disciplines, an overview of several AI applications, current research gaps, and trends is needed. In this context, this study conducted the first state-of-the-art research on AI for building asset management. A total of 578 papers were analyzed with bibliometric tools to identify prominent institutions, topics, and journals. The quantitative analysis helped determine the most researched areas of AM and which AI techniques are applied. The areas were furtherly investigated by reading in-depth the 83 most relevant studies selected by screening the articles’ abstracts identified in the bibliometric analysis. The results reveal many applications for Energy Management, Condition assessment, Risk management, and Project management areas. Finally, the literature review identified three main trends that can be a reference point for future studies made by practitioners or researchers: Digital Twin, Generative Adversarial Networks (with synthetic images) for data augmentation, and Deep Reinforcement Learning

3D Segmentation and Damage Analysis from Robotic Scans of Disaster Sites

Disaster relief and response plays an important role in saving lives and reducing economic loss after earthquakes, windstorm events and man-made explosions. Mobile robots represent an effective solution to assist in post-disaster reconnaissance in areas that are dangerous to human agents. These robots need an accurate 3D semantic map of the site in order to carry out disaster relief work such as search and rescue and damage assessment. Thus, there exists a research need to automatically identify building elements and detect structural damage from laser-scanned points clouds acquired by mobile robots. Current methods for point cloud semantic segmentation mostly perform direct class prediction at the point level without considering object-level semantics and generalizability across datasets. Moreover, current segmentation methods are unsuitable for real-time operation because they are designed to work as a post-processing step and do not process points from new scans in an online manner. This research proposes a learnable region growing method to perform class-agnostic point cloud segmentation in a data-driven and generalizable manner. In addition, an anomaly-based crack segmentation method is proposed where a deep feature embedding is used as a basis for separation between inlier and outlier points. Finally, an incremental segmentation scheme is used to process point cloud data in an online fashion and combine semantic information across multiple scans.Ph.D