Investigation of Computer Vision Concepts and Methods for Structural Health Monitoring and Identification Applications

This study presents a comprehensive investigation of methods and technologies for developing a computer vision-based framework for Structural Health Monitoring (SHM) and Structural Identification (St-Id) for civil infrastructure systems, with particular emphasis on various types of bridges. SHM is implemented on various structures over the last two decades, yet, there are some issues such as considerable cost, field implementation time and excessive labor needs for the instrumentation of sensors, cable wiring work and possible interruptions during implementation. These issues make it only viable when major investments for SHM are warranted for decision making. For other cases, there needs to be a practical and effective solution, which computer-vision based framework can be a viable alternative. Computer vision based SHM has been explored over the last decade. Unlike most of the vision-based structural identification studies and practices, which focus either on structural input (vehicle location) estimation or on structural output (structural displacement and strain responses) estimation, the proposed framework combines the vision-based structural input and the structural output from non-contact sensors to overcome the limitations given above. First, this study develops a series of computer vision-based displacement measurement methods for structural response (structural output) monitoring which can be applied to different infrastructures such as grandstands, stadiums, towers, footbridges, small/medium span concrete bridges, railway bridges, and long span bridges, and under different loading cases such as human crowd, pedestrians, wind, vehicle, etc. Structural behavior, modal properties, load carrying capacities, structural serviceability and performance are investigated using vision-based methods and validated by comparing with conventional SHM approaches. In this study, some of the most famous landmark structures such as long span bridges are utilized as case studies. This study also investigated the serviceability status of structures by using computer vision-based methods. Subsequently, issues and considerations for computer vision-based measurement in field application are discussed and recommendations are provided for better results. This study also proposes a robust vision-based method for displacement measurement using spatio-temporal context learning and Taylor approximation to overcome the difficulties of vision-based monitoring under adverse environmental factors such as fog and illumination change. In addition, it is shown that the external load distribution on structures (structural input) can be estimated by using visual tracking, and afterward load rating of a bridge can be determined by using the load distribution factors extracted from computer vision-based methods. By combining the structural input and output results, the unit influence line (UIL) of structures are extracted during daily traffic just using cameras from which the external loads can be estimated by using just cameras and extracted UIL. Finally, the condition assessment at global structural level can be achieved using the structural input and output, both obtained from computer vision approaches, would give a normalized response irrespective of the type and/or load configurations of the vehicles or human loads

Image-based Automated Width Measurement of Surface Cracking

The detection of cracks is an important monitoring task in civil engineering infrastructure devoted to ensuring durability, structural safety, and integrity. It has been traditionally performed by visual inspection, and the measurement of crack width has been manually obtained with a crack-width comparator gauge (CWCG). Unfortunately, this technique is time-consuming, suffers from subjective judgement, and is error-prone due to the difficulty of ensuring a correct spatial measurement as the CWCG may not be correctly positioned in accordance with the crack orientation. Although algorithms for automatic crack detection have been developed, most of them have specifically focused on solving the segmentation problem through Deep Learning techniques failing to address the underlying problem: crack width evaluation, which is critical for the assessment of civil structures. This paper proposes a novel automated method for surface cracking width measurement based on digital image processing techniques. Our proposal consists of three stages: anisotropic smoothing, segmentation, and stabilized central points by k-means adjustment and allows the characterization of both crack width and curvature-related orientation. The method is validated by assessing the surface cracking of fiber-reinforced earthen construction materials. The preliminary results show that the proposal is robust, efficient, and highly accurate at estimating crack width in digital images. The method effectively discards false cracks and detects real ones as small as 0.15 mm width regardless of the lighting conditions

Coping with Data Scarcity in Deep Learning and Applications for Social Good

The recent years are experiencing an extremely fast evolution of the Computer Vision and Machine Learning fields: several application domains benefit from the newly developed technologies and industries are investing a growing amount of money in Artificial Intelligence. Convolutional Neural Networks and Deep Learning substantially contributed to the rise and the diffusion of AI-based solutions, creating the potential for many disruptive new businesses. The effectiveness of Deep Learning models is grounded by the availability of a huge amount of training data. Unfortunately, data collection and labeling is an extremely expensive task in terms of both time and costs; moreover, it frequently requires the collaboration of domain experts. In the first part of the thesis, I will investigate some methods for reducing the cost of data acquisition for Deep Learning applications in the relatively constrained industrial scenarios related to visual inspection. I will primarily assess the effectiveness of Deep Neural Networks in comparison with several classical Machine Learning algorithms requiring a smaller amount of data to be trained. Hereafter, I will introduce a hardware-based data augmentation approach, which leads to a considerable performance boost taking advantage of a novel illumination setup designed for this purpose. Finally, I will investigate the situation in which acquiring a sufficient number of training samples is not possible, in particular the most extreme situation: zero-shot learning (ZSL), which is the problem of multi-class classification when no training data is available for some of the classes. Visual features designed for image classification and trained offline have been shown to be useful for ZSL to generalize towards classes not seen during training. Nevertheless, I will show that recognition performances on unseen classes can be sharply improved by learning ad hoc semantic embedding (the pre-defined list of present and absent attributes that represent a class) and visual features, to increase the correlation between the two geometrical spaces and ease the metric learning process for ZSL. In the second part of the thesis, I will present some successful applications of state-of-the- art Computer Vision, Data Analysis and Artificial Intelligence methods. I will illustrate some solutions developed during the 2020 Coronavirus Pandemic for controlling the disease vii evolution and for reducing virus spreading. I will describe the first publicly available dataset for the analysis of face-touching behavior that we annotated and distributed, and I will illustrate an extensive evaluation of several computer vision methods applied to the produced dataset. Moreover, I will describe the privacy-preserving solution we developed for estimating the \u201cSocial Distance\u201d and its violations, given a single uncalibrated image in unconstrained scenarios. I will conclude the thesis with a Computer Vision solution developed in collaboration with the Egyptian Museum of Turin for digitally unwrapping mummies analyzing their CT scan, to support the archaeologists during mummy analysis and avoiding the devastating and irreversible process of physically unwrapping the bandages for removing amulets and jewels from the body

Applications of Computer Vision Technologies of Automated Crack Detection and Quantification for the Inspection of Civil Infrastructure Systems

Many components of existing civil infrastructure systems, such as road pavement, bridges, and buildings, are suffered from rapid aging, which require enormous nation\u27s resources from federal and state agencies to inspect and maintain them. Crack is one of important material and structural defects, which must be inspected not only for good maintenance of civil infrastructure with a high quality of safety and serviceability, but also for the opportunity to provide early warning against failure. Conventional human visual inspection is still considered as the primary inspection method. However, it is well established that human visual inspection is subjective and often inaccurate. In order to improve current manual visual inspection for crack detection and evaluation of civil infrastructure, this study explores the application of computer vision techniques as a non-destructive evaluation and testing (NDE&T) method for automated crack detection and quantification for different civil infrastructures. In this study, computer vision-based algorithms were developed and evaluated to deal with different situations of field inspection that inspectors could face with in crack detection and quantification. The depth, the distance between camera and object, is a necessary extrinsic parameter that has to be measured to quantify crack size since other parameters, such as focal length, resolution, and camera sensor size are intrinsic, which are usually known by camera manufacturers. Thus, computer vision techniques were evaluated with different crack inspection applications with constant and variable depths. For the fixed-depth applications, computer vision techniques were applied to two field studies, including 1) automated crack detection and quantification for road pavement using the Laser Road Imaging System (LRIS), and 2) automated crack detection on bridge cables surfaces, using a cable inspection robot. For the various-depth applications, two field studies were conducted, including 3) automated crack recognition and width measurement of concrete bridges\u27 cracks using a high-magnification telescopic lens, and 4) automated crack quantification and depth estimation using wearable glasses with stereovision cameras. From the realistic field applications of computer vision techniques, a novel self-adaptive image-processing algorithm was developed using a series of morphological transformations to connect fragmented crack pixels in digital images. The crack-defragmentation algorithm was evaluated with road pavement images. The results showed that the accuracy of automated crack detection, associated with artificial neural network classifier, was significantly improved by reducing both false positive and false negative. Using up to six crack features, including area, length, orientation, texture, intensity, and wheel-path location, crack detection accuracy was evaluated to find the optimal sets of crack features. Lab and field test results of different inspection applications show that proposed compute vision-based crack detection and quantification algorithms can detect and quantify cracks from different structures\u27 surface and depth. Some guidelines of applying computer vision techniques are also suggested for each crack inspection application

Non-Contact Evaluation Methods for Infrastructure Condition Assessment

The United States infrastructure, e.g. roads and bridges, are in a critical condition. Inspection, monitoring, and maintenance of these infrastructure in the traditional manner can be expensive, dangerous, time-consuming, and tied to human judgment (the inspector). Non-contact methods can help overcoming these challenges. In this dissertation two aspects of non-contact methods are explored: inspections using unmanned aerial systems (UASs), and conditions assessment using image processing and machine learning techniques. This presents a set of investigations to determine a guideline for remote autonomous bridge inspections

Unmanned aerial vehicle-based computer vision for structural vibration measurement and condition assessment: A concise survey

With the rapid advance in camera sensor technology, the acquisition of high-resolution images or videos has become extremely convenient and cost-effective. Computer vision that extracts semantic knowledge directly from digital images or videos, offers a promising solution for non-contact and full-field structural vibration measurement and condition assessment. Unmanned aerial vehicles (UAVs), also known as flying robots or drones, are being actively developed to suit a wide range of applications. Taking advantage of its excellent mobility and flexibility, camera-equipped UAV systems can facilitate the use of computer vision, thus enhancing the capacity of the structural condition assessment. The current article aims to provide a concise survey of the recent progress and applications of UAV-based computer vision in the field of structural dynamics. The different aspects to be discussed include the UAV system design and algorithmic development in computer vision. The main challenges, future trends, and opportunities to advance the technology and close the gap between research and practice will also be stated

Applied Measurement Systems

Measurement is a multidisciplinary experimental science. Measurement systems synergistically blend science, engineering and statistical methods to provide fundamental data for research, design and development, control of processes and operations, and facilitate safe and economic performance of systems. In recent years, measuring techniques have expanded rapidly and gained maturity, through extensive research activities and hardware advancements. With individual chapters authored by eminent professionals in their respective topics, Applied Measurement Systems attempts to provide a comprehensive presentation and in-depth guidance on some of the key applied and advanced topics in measurements for scientists, engineers and educators

Uncertainty Minimization in Robotic 3D Mapping Systems Operating in Dynamic Large-Scale Environments

This dissertation research is motivated by the potential and promise of 3D sensing technologies in safety and security applications. With specific focus on unmanned robotic mapping to aid clean-up of hazardous environments, under-vehicle inspection, automatic runway/pavement inspection and modeling of urban environments, we develop modular, multi-sensor, multi-modality robotic 3D imaging prototypes using localization/navigation hardware, laser range scanners and video cameras. While deploying our multi-modality complementary approach to pose and structure recovery in dynamic real-world operating conditions, we observe several data fusion issues that state-of-the-art methodologies are not able to handle. Different bounds on the noise model of heterogeneous sensors, the dynamism of the operating conditions and the interaction of the sensing mechanisms with the environment introduce situations where sensors can intermittently degenerate to accuracy levels lower than their design specification. This observation necessitates the derivation of methods to integrate multi-sensor data considering sensor conflict, performance degradation and potential failure during operation. Our work in this dissertation contributes the derivation of a fault-diagnosis framework inspired by information complexity theory to the data fusion literature. We implement the framework as opportunistic sensing intelligence that is able to evolve a belief policy on the sensors within the multi-agent 3D mapping systems to survive and counter concerns of failure in challenging operating conditions. The implementation of the information-theoretic framework, in addition to eliminating failed/non-functional sensors and avoiding catastrophic fusion, is able to minimize uncertainty during autonomous operation by adaptively deciding to fuse or choose believable sensors. We demonstrate our framework through experiments in multi-sensor robot state localization in large scale dynamic environments and vision-based 3D inference. Our modular hardware and software design of robotic imaging prototypes along with the opportunistic sensing intelligence provides significant improvements towards autonomous accurate photo-realistic 3D mapping and remote visualization of scenes for the motivating applications

Optical sensors for cultural heritage and biomedical applications

The current Ph.D. thesis is articulated in 4 different research paths. The main research topic is on the fiber Bragg grating (FBG) sensor and its applications, mainly related to the conservation of the original status of the artworks. The second topic is related to the development of a new methodology for measuring the cracking of the Structural Health Monitoring (SHM) of cultural heritage. In the third topic, it is addressed the subject on the complex diatribe related to the risk in delivering anesthesia; field in which I have been also working, being a Biomedical Engineer. Finally, in the last topic it is proposed a biomechanics study on the patellar taping with the purpose of finding a correlation between the taping and a neuromuscular response. A new era of pollution requires an important focus on the conservation of archaeological sites and monuments. In the last years, great efforts were made to develop various sensors for different tasks; the FBG was one of the most studied thanks to the multitude of applications and the surprising performances. An original fiber optic sensor that combines the fiber Bragg gratings with a pH responsive polymer coating for monitoring the pH of the rains on critical and prestigious monuments is proposed. In this study, the arrangement setup of the optical sensor is modeled with Comsol Multiphysics (Wave Optics Module), based on the FEM (Finite Element Method) solver. Monitoring the pH of the rain can be used by experts to predict and control the corrosion of specific materials, especially limestone and marble, thus scheduling timely restoration. This also depends on the materials under analysis and it can have an important impact in terms of cost reductions and higher maintenance efficiency. In conclusion, the swelling response of hydrogels to the change of surrounding pH allowed the development of a model of hydrogel coated FBG pH sensor. Modelling the FBG pH sensor for monitoring the rain in archaeology and in cultural heritage provided innovative results in terms of high sensitivity and small dimensions of the device, allowing better intervention planning. In the first chapter, a preliminary study regarding the optical ring resonator is conducted because, ultimately, the goal is to realize a sensor that combines the FBG and the ring resonator for future developments in order to improve the sensor performances. Along with the development of the FBG sensor, a new methodology for measuring the cracking for the Structural Health Monitoring (SHM) of cultural heritage is also studied. The methodology is characterized by being minimally invasive on the artefact that has to be preserved, which is one of the main qualities required in this field. The approach is to determine the relative distance between two optical tags, using advanced fitting algorithms for the objective function. Different kinds of objective-function were taken into account in order to identify the best configuration to determine the fitting parameters, useful to the SHM. The optical tags are introduced for this scope; they are nothing but adhesive labels with appropriate images, through which, by using advanced fitting methods and algorithms, it is possible to determine the absolute and relative position and three-dimensional rotations of the images. The third chapter of this thesis deals with the risk and perception of risk in delivering anesthesia. The study examines the different perceptions of risk associated with anesthesia systems from the viewpoints of the product manufacturer and the caregiver. Only little research has been done on the impact of the perception of risk for patient safety in anesthesia. The role of the manufacturer in mitigating the perception of risk is central in the work. The risk was examined as the probability of negative occurrences based on the Medical Device Reportable (MDR) events and these risks were compared to how the caregiver perceives and manages them when delivering anesthesia. Analysis of the manufacturer’s public Medical Device Reportable (MDR) events data was performed in the US market and it represents the actual risk achieved; the bibliographic review provided a perspective on how the risk is perceived and managed by the caregiver when delivering anesthesia. The goal of the research path is to highlight how the role of the manufacturers can have an impact on the reduction of perception of risk in anesthesia, increasing patient safety. Finally, a biomechanical proposal on the estimation of Centre of Mass (CoM) trajectory has been developed. Motion capture systems and force platforms are still considered the gold standard for the estimation of accurate CoM measurements. In the last decade, several methods based on inertial sensors systems have been proposed based on double integration of acceleration signals of pelvis-worn sensors (M. J. Floor-Westerdijk, 2012). Although the portability of those methodologies is higher, drift errors due to extremely lengthy time acquisitions affect measurements, limiting their use. For the purpose of avoiding drift error and providing an accurate tool for ambulatory and/or home CoM assessment, the accuracy of a novel method based on a Biomechanical Model (BM) will be investigated. Among the large number of potential applications, this novel approach could be used in the identification of the effects of the patellar taping on neuromuscular control. More specifically, the patellar taping technique proposed by McConnell (J. McCONNELL, 1986) allows patients to engage in pain-free physical therapy exercises, by medializing the patella. Although this technique has been demonstrated to reduce the perceived pain of patients with patellofemoral pain syndrome and improve neuromuscular activity (N. Aminaka and P. A. Gribble, 2008), a deeper investigation on how patellar tape influences postural stability thought CoM assessment could be of great interest in the long term management of Chondromalacia Patellae. It has been demonstrated, in fact, that patellar taping affects knee proprioception other than relieving pain in subjects with patellofemoral pain syndrome (M. J. Callaghan, 2008). The aim of this research program is to pursue through static and dynamic tasks performed twice both by healthy subjects and not-healthy ones, with and without patellar tape