Computer vision-based structural assessment exploiting large volumes of images

Visual assessment is a process to understand the state of a structure based on evaluations originating from visual information. Recent advances in computer vision to explore new sensors, sensing platforms and high-performance computing have shed light on the potential for vision-based visual assessment in civil engineering structures. The use of low-cost, high-resolution visual sensors in conjunction with mobile and aerial platforms can overcome spatial and temporal limitations typically associated with other forms of sensing in civil structures. Also, GPU-accelerated and parallel computing offer unprecedented speed and performance, accelerating processing the collected visual data. However, despite the enormous endeavor in past research to implement such technologies, there are still many practical challenges to overcome to successfully apply these techniques in real world situations. A major challenge lies in dealing with a large volume of unordered and complex visual data, collected under uncontrolled circumstance (e.g. lighting, cluttered region, and variations in environmental conditions), while just a tiny fraction of them are useful for conducting actual assessment. Such difficulty induces an undesirable high rate of false-positive and false-negative errors, reducing the trustworthiness and efficiency of their implementation. To overcome the inherent challenges in using such images for visual assessment, high-level computer vision algorithms must be integrated with relevant prior knowledge and guidance, thus aiming to have similar performance with those of humans conducting visual assessment. Moreover, the techniques must be developed and validated in the realistic context of a large volume of real-world images, which is likely contain numerous practical challenges. In this dissertation, the novel use of computer vision algorithms is explored to address two promising applications of vision-based visual assessment in civil engineering: visual inspection, and visual data analysis for post-disaster evaluation. For both applications, powerful techniques are developed here to enable reliable and efficient visual assessment for civil structures and demonstrate them using a large volume of real-world images collected from actual structures. State-of-art computer vision techniques, such as structure-from-motion and convolutional neural network techniques, facilitate these tasks. The core techniques derived from this study are scalable and expandable to many other applications in vision-based visual assessment, and will serve to close the existing gaps between past research efforts and real-world implementations

Enhanced Concrete Bridge Assessment Using Artificial Intelligence and Mixed Reality

Conventional methods for visual assessment of civil infrastructures have certain limitations, such as subjectivity of the collected data, long inspection time, and high cost of labor. Although some new technologies (i.e. robotic techniques) that are currently in practice can collect objective, quantified data, the inspector\u27s own expertise is still critical in many instances since these technologies are not designed to work interactively with human inspector. This study aims to create a smart, human-centered method that offers significant contributions to infrastructure inspection, maintenance, management practice, and safety for the bridge owners. By developing a smart Mixed Reality (MR) framework, which can be integrated into a wearable holographic headset device, a bridge inspector, for example, can automatically analyze a certain defect such as a crack that he or she sees on an element, display its dimension information in real-time along with the condition state. Such systems can potentially decrease the time and cost of infrastructure inspections by accelerating essential tasks of the inspector such as defect measurement, condition assessment and data processing to management systems. The human centered artificial intelligence (AI) will help the inspector collect more quantified and objective data while incorporating inspector\u27s professional judgment. This study explains in detail the described system and related methodologies of implementing attention guided semi-supervised deep learning into mixed reality technology, which interacts with the human inspector during assessment. Thereby, the inspector and the AI will collaborate/communicate for improved visual inspection

A Systematic Review of Convolutional Neural Network-Based Structural Condition Assessment Techniques

With recent advances in non-contact sensing technology such as cameras, unmanned aerial and ground vehicles, the structural health monitoring (SHM) community has witnessed a prominent growth in deep learning-based condition assessment techniques of structural systems. These deep learning methods rely primarily on convolutional neural networks (CNNs). The CNN networks are trained using a large number of datasets for various types of damage and anomaly detection and post-disaster reconnaissance. The trained networks are then utilized to analyze newer data to detect the type and severity of the damage, enhancing the capabilities of non-contact sensors in developing autonomous SHM systems. In recent years, a broad range of CNN architectures has been developed by researchers to accommodate the extent of lighting and weather conditions, the quality of images, the amount of background and foreground noise, and multiclass damage in the structures. This paper presents a detailed literature review of existing CNN-based techniques in the context of infrastructure monitoring and maintenance. The review is categorized into multiple classes depending on the specific application and development of CNNs applied to data obtained from a wide range of structures. The challenges and limitations of the existing literature are discussed in detail at the end, followed by a brief conclusion on potential future research directions of CNN in structural condition assessment

Evaluation of new technologies to support asset management of metro systems

Since 1930, London Underground Limited (LUL) has performed visual inspections to understand the condition of the physical assets such as tunnels, bridges and structures. The major problem with this kind of inspection is the lack in quality of the data, as it depends on the ability of the inspector to assess and interpret the condition of the asset both accurately and with repeatability. In addition, data collection is time-consuming and, therefore, costly when the whole of the metro network needs to be regularly inspected and there are limited periods when access is available. The problems associated with access to the infrastructure have increased significantly with the implementation of the night tube and will increase further as the night tube is extended over the next 5 to 10 years. To determine the condition of metro assets and to predict the need for intervention, monitoring the changes in the assets’ condition is key to any further evaluation and maintenance planning. This thesis presents the outcomes of using new technologies such as Thermography, Kinematic and Static Laser Scanning, Close-Range Photogrammetry and Total Station to measure defects, such as water seepage, mortar loss in joints, lining face loss (in brick tunnels), cracks, corrosion, voids, cavities and spalls. Each technique is explored through three case studies that evaluate the performance and limitation in the determination of the asset condition. The first case study was performed to compare and contrast the use of Euroconsult’s high definition laser survey against a Principal Inspection Report to determine the level of consistency in predicting the asset condition. During this case study, reports from laser surveys and principal inspections of brick tunnels and covered ways were compared. This analysis showed that a direct comparison between the two inspections is not appropriate because the laser inspection does not capture all the defects mentioned in the Engineering Standard S1060. It also showed that to close the gap between the laser survey and visual inspection, laser surveys would have to be performed every year in brick tunnels and then compare any changes in asset condition with that from the previous scan. The second case study was performed using Infrared Thermography (IRT) to identify water seepage in the brick tunnels as well as test the system in a configuration that would allow the survey to be done from an engineering train. A set of calibration tests were performed in the lab and later the technique was trialled on an engineering train. The results showed that it is possible to measure the level of moisture on specific parts of the lining and that the comparison of surveys performed at different times can allow asset managers to react before a seepage is established, potentially reducing the risk of system disruption caused by water ingress in tunnels. The data also revealed that this technique could be used for other purposes, such as examining the condition of other assets such as brackets, cable supports and broken light bulbs. The third case study was performed using a Terrestrial Laser Scanner, Close-Range Photogrammetry and Total Station Survey to identify defects in structures. In order to test these technologies, a wing wall, located on the north-east wing of the HC3 underbridge at Ladbroke Grove Station, was chosen. This case study demonstrated that LUL can easily implement this type of technology to inspect rapidly their buildings and structures, being able to identify defects and monitor their assets for translation, rotation and changes in shape during changes in loading or the decay of the structure (insidious decline) and the construction of nearby assets. In this research, a large volume of data was captured, and further work is needed in order to manage the data using ‘big data’ concepts. Although it may not be possible to fully understand the insidious decline of an asset, the use of these techniques allows us to better understand how a civil asset behaves, potentially reducing the amount of reactive maintenance to a minimum, consequently reducing service costs and falls in revenue due to disruptions in the system. To successfully analyse the data from new technologies a combination of skills is required and different or retrained personal will be needed

Robotic navigation and inspection of bridge bearings

This thesis focuses on the development of a robotic platform for bridge bearing inspection. The existing literature on this topic highlights an aspiration for increased automation of bridge inspection, due to an increasing amount of ageing infrastructure and costly inspection. Furthermore, bridge bearings are highlighted as being one of the most costly components of the bridge to maintain. However, although autonomous robotic inspection is often stated as an aspiration, the existing literature for robotic bridge inspection often neglects to include the requirement of autonomous navigation. To achieve autonomous inspection, some methods for mapping and localising in the bridge structure are required. This thesis compares existing methods for simultaneous localisation and mapping (SLAM) with localisation-only methods. In addition, a method for using pre-existing data to create maps for localisation is proposed. A robotic platform was developed and these methods for localisation and mapping were then compared in a laboratory environment and then in a real bridge environment. The errors in the bridge environment are greater than in the laboratory environment, but remained within a defined error bound. A combined approach is suggested as an appropriate method for combining the lower errors of a SLAM approach with the advantages of a localisation approach for defining existing goals. Longer-term testing in a real bridge environment is still required. The use of existing inspection data is then extended to the creation of a simulation environment, with the goal of creating a methodology for testing different configurations of bridges or robots in a more realistic environment than laboratory testing, or other existing simulation environments. Finally, the inspection of the structure surrounding the bridge bearing is considered, with a particular focus on the detection and segmentation of cracks in concrete. A deep learning approach is used to segment cracks from an existing dataset and compared to an existing machine learning approach, with the deep-learning approach achieving a higher performance using a pixel-based evaluation. Other evaluation methods were also compared that take the structure of the crack, and other related datasets, into account. The generalisation of the approach for crack segmentation is evaluated by comparing the results of the trained on different datasets. Finally, recommendations for improving the datasets to allow better comparisons in future work is given

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

Adaptive Wavelet Neural Network for Terrestrial Laser Scanner-Based Crack Detection

Objective, accurate, and fast assessment of civil infrastructure conditions is critical to timely assess safety risks. Current practices rely on visual observations and manual interpretation of reports and sketches prepared by inspectors in the field, which are labor intensive, subject to personal judgment and experience, and prone to error. Terrestrial laser scanners (TLS) are promising for automatically identifying structural condition indicators, as they are capable of providing coverage for large areas with accuracy at long ranges. Major challenges in using this technology are in storing significant amount of data and extracting appropriate features enabling condition assessment. This paper proposes a novel adaptive wavelet neural network (WNN)-based approach to compress data into a combination of low- and high-resolution surfaces, and automatically detect concrete cracks and other forms of damage. The adaptive WNN is designed to sequentially self-organize and self-adapt in order to construct an optimized representation. The architecture of the WNN is based on a single-layer neural network consisting of Mexican hat wavelet functions. The strategy is to first construct a low-resolution representation of the point cloud, then detect and localize anomalies, and finally construct a high-resolution representation around these anomalies to enhance their characterization. The approach was verified on four cracked concrete specimens. The experimental results show that the proposed approach was capable of fitting the point cloud, and of detecting and fitting the crack. The results demonstrated data compression of 99.4%, 72.2%, 92.4% and 78.9% for the four specimens when using low resolution fit for crack detection. For specimens 1, 2 and 3, 97.1%, 42.5% and 63.9% compression of data were obtained for crack localization, which is a significant improvement over previous TLS based crack detection and measurement approaches. Using the proposed method for crack detection would enable automatic and remote assessment of structural conditions. This would, in turn, result in reducing costs associated with infrastructure management, and improving the overall quality of our infrastructure by enhancing maintenance operations