Small unmanned airborne systems to support oil and gas pipeline monitoring and mapping

Acknowledgments We thank Johan Havelaar, Aeryon Labs Inc., AeronVironment Inc. and Aeronautics Inc. for kindly permitting the use of materials in Fig. 1.Peer reviewedPublisher PD

UAV-Based Bridge Inspection and Computational Simulations

The use of Unmanned Aerial Vehicles (UAV), commonly known as drones, has significantly increased in the field of civil engineering due to the poor condition of the United States’ infrastructure. The American Society of Civil Engineers (ASCE) recently reported that more than 9.1% of the United States’ bridges were structurally deficient and required attention and maintenance to ensure appropriate structural performance. Meanwhile, current practices are expensive and unsafe for bridge inspectors, requiring innovative and safer methods for the study of bridges. The goal of this paper was to identify better techniques to not only inspect, quantify, and determine the effect of damage on bridges to minimize the risk for inspectors, but also to determine their live-load performance using UAV-based computational simulation updating techniques. To accomplish the objective, an extensive literature review and survey to state departments of transportation (DOTs) was conducted to gain technical knowledge on current UAV-based inspection practices. To evaluate the efficiency of the UAV, the Keystone Interchange Bridges (i.e., Keystone Wye timber arch bridge and timber girder bridge) in the Black Hills National Forest near the city of Keystone, South Dakota (SD), were studied. To provide a more systematical and efficient UAV-enabled bride inspection method, a five-stage recommended bridge inspection protocol was developed. A UAV-image-based bridge damage quantification protocol involving image quality assessment and image-based damage measurement was recommended. Finally, using the damage information form the inspection and quantification of the bridges, a Finite Element (FE) model to determine the live-load performance of the Keystone Wye timber arch bridge in terms of Distribution Factors (DF) and Load Rating Factors (RF) was developed. It was concluded that the UAV served as an effective tool to supplement current inspection practices and provide damage information that can be used to update FE models to rationally estimate bridge performance

USE OF UNMANNED AERIAL VEHICLES (UAV) FOR URBAN TREE INVENTORIES

In contrast to standard aerial imagery, unmanned aerial systems (UAS) utilize recent technological advances to provide an affordable alternative for imagery acquisition. Increased value can be realized through clarity and detail providing higher resolution (2-5 cm) over traditional products. Many natural resource disciplines such as urban forestry will benefit from UAS. Tree inventories for risk assessment, biodiversity, planning, and design can be efficiently achieved with the UAS. Recent advances in photogrammetric processing have proved automated methods for three dimensional rendering of aerial imagery. Point clouds can be generated from images providing additional benefits. Association of spatial locational information within the point cloud can be used to produce elevation models i.e. digital elevation, digital terrain and digital surface. Taking advantage of this point cloud data, additional information such as tree heights can be obtained. Several software applications have been developed for LiDAR data which can be adapted to utilize UAS point clouds. This study examines solutions to provide tree inventory and heights from UAS imagery. Imagery taken with a micro-UAS was processed to produce a seamless orthorectified image. This image provided an accurate way to obtain a tree inventory within the study boundary. Utilizing several methods, tree height models were developed with variations in spatial accuracy. Model parameters were modified to offset spatial inconsistencies providing statistical equality of means. Statistical results (p = 0.756) with a level of significance (α = 0.01) between measured and modeled tree height means resulted with 82% of tree species obtaining accurate tree heights. Within this study, the UAS has proven to be an efficient tool for urban forestry providing a cost effective and reliable system to obtain remotely sensed data

Early Detection of Near-Surface Void Defects in Concrete Pavement Using Drone Based Thermography and GPR Methods

The goal of this research is to evaluate the feasibility and the performance of using UAV-mounted infrared thermography (IRT) and ground penetration radar (GPR) to detect sub-surface voids caused by consolidation issues in concrete pavement. The motivation of the study is to identify the consolidation defects as early as the initial set of concrete to avoid having this problem in large pavement sections, which is costly and time consuming to repair. Using the two technologies in combination to detect subsurface voids in the concrete initial set stage is new and aims to take advantage of the strengths and minimize the limitations of each method. UAV-based IRT can cover large areas of the pavements in a short amount of time, while GPR can provide higher accuracy in locating the defects horizontally and vertically. Therefore, the combination of the two technologies can allow detection of small voids in large areas with improved confidence. In this project, both laboratory and field tests were conducted with both methods, and coring samples were used for validation of results. The results from multiple specimens and multiple experiments suggested that both technologies performed well in detecting the subsurface voids in the concrete pavement’s initial set stage. Despite some limitations discussed in the report, the outcomes of the project provided evidence that these technologies can be used separately or together on the field as efficient and economical quality control tools in concrete pavement construction

Application of photogrammetry techniques for the visual assessment of vessels' cargo hold

Visual inspection is an integral part of Condition and Class surveys, with the results comprising of the surveyors’ opinion, documented by a sum of pictures indicating areas of interest. Although this way provides the most essential information, the communication of the results may be difficult, since isolated images cannot provide the context. Photogrammetry exploits pictorial data to provide 3D models, with a high level of accuracy and is not an uncommon method in the maritime environment. Use of such methods to support visual survey activities is examined in this work, providing the methodology for the data collection, which is structured in an algorithmic way, to enable realization by automated means (robots). The 3D model is provided, along with accuracy results

Application of photogrammetry techniques for the visual assessment of vessels’ cargo hold

Visual inspection is an integral part of Condition and Class surveys, with the results comprising of the surveyors’ opinion, documented by a sum of pictures indicating areas of interest. Although this way provides the most essential information, the communication of the results may be difficult, since isolated images cannot provide the context. Photogrammetry exploits pictorial data to provide 3D models, with a high level of accuracy and is not an uncommon method in the maritime environment. Use of such methods to support visual survey activities is examined in this work, providing the methodology for the data collection, which is structured in an algorithmic way, to enable realization by automated means (robots). The 3D model is provided, along with accuracy results

Computer Vision Applications for Autonomous Aerial Vehicles

Undoubtedly, unmanned aerial vehicles (UAVs) have experienced a great leap forward over the last decade. It is not surprising anymore to see a UAV being used to accomplish a certain task, which was previously carried out by humans or a former technology. The proliferation of special vision sensors, such as depth cameras, lidar sensors and thermal cameras, and major breakthroughs in computer vision and machine learning fields accelerated the advance of UAV research and technology. However, due to certain unique challenges imposed by UAVs, such as limited payload capacity, unreliable communication link with the ground stations and data safety, UAVs are compelled to perform many tasks on their onboard embedded processing units, which makes it difficult to readily implement the most advanced algorithms on UAVs. This thesis focuses on computer vision and machine learning applications for UAVs equipped with onboard embedded platforms, and presents algorithms that utilize data from multiple modalities. The presented work covers a broad spectrum of algorithms and applications for UAVs, such as indoor UAV perception, 3D understanding with deep learning, UAV localization, and structural inspection with UAVs. Visual guidance and scene understanding without relying on pre-installed tags or markers is the desired approach for fully autonomous navigation of UAVs in conjunction with the global positioning systems (GPS), or especially when GPS information is either unavailable or unreliable. Thus, semantic and geometric understanding of the surroundings become vital to utilize vision as guidance in the autonomous navigation pipelines. In this context, first, robust altitude measurement, safe landing zone detection and doorway detection methods are presented for autonomous UAVs operating indoors. These approaches are implemented on Google Project Tango platform, which is an embedded platform equipped with various sensors including a depth camera. Next, a modified capsule network for 3D object classification is presented with weight optimization so that the network can be fit and run on memory-constrained platforms. Then, a semantic segmentation method for 3D point clouds is developed for a more general visual perception on a UAV equipped with a 3D vision sensor. Next, this thesis presents algorithms for structural health monitoring applications involving UAVs. First, a 3D point cloud-based, drift-free and lightweight localization method is presented for depth camera-equipped UAVs that perform bridge inspection, where GPS signal is unreliable. Next, a thermal leakage detection algorithm is presented for detecting thermal anomalies on building envelopes using aerial thermography from UAVs. Then, building on our thermal anomaly identification expertise gained on the previous task, a novel performance anomaly identification metric (AIM) is presented for more reliable performance evaluation of thermal anomaly identification methods

Condition Assessment of Concrete Bridge Decks Using Ground and Airborne Infrared Thermography

Applications of nondestructive testing (NDT) technologies have shown promise in assessing the condition of existing concrete bridges. Infrared thermography (IRT) has gradually gained wider acceptance as a NDT and evaluation tool in the civil engineering field. The high capability of IRT in detecting subsurface delamination, commercial availability of infrared cameras, lower cost compared with other technologies, speed of data collection, and remote sensing are some of the expected benefits of applying this technique in bridge deck inspection practices. The research conducted in this thesis aims at developing a rational condition assessment system for concrete bridge decks based on IRT technology, and automating its analysis process in order to add this invaluable technique to the bridge inspector’s tool box. Ground penetrating radar (GPR) has also been vastly recognized as a NDT technique capable of evaluating the potential of active corrosion. Therefore, integrating IRT and GPR results in this research provides more precise assessments of bridge deck conditions. In addition, the research aims to establish a unique link between NDT technologies and inspector findings by developing a novel bridge deck condition rating index (BDCI). The proposed procedure captures the integrated results of IRT and GPR techniques, along with visual inspection judgements, thus overcoming the inherent scientific uncertainties of this process. Finally, the research aims to explore the potential application of unmanned aerial vehicle (UAV) infrared thermography for detecting hidden defects in concrete bridge decks. The NDT work in this thesis was conducted on full-scale deteriorated reinforced concrete bridge decks located in Montreal, Quebec and London, Ontario. The proposed models have been validated through various case studies. IRT, either from the ground or by utilizing a UAV with high-resolution thermal infrared imagery, was found to be an appropriate technology for inspecting and precisely detecting subsurface anomalies in concrete bridge decks. The proposed analysis produced thermal mosaic maps from the individual IR images. The k-means clustering classification technique was utilized to segment the mosaics and identify objective thresholds and, hence, to delineate different categories of delamination severity in the entire bridge decks. The proposed integration methodology of NDT technologies and visual inspection results provided more reliable BDCI. The information that was sought to identify the parameters affecting the integration process was gathered from bridge engineers with extensive experience and intuition. The analysis process utilized the fuzzy set theory to account for uncertainties and imprecision in the measurements of bridge deck defects detected by IRT and GPR testing along with bridge inspector observations. The developed system and models should stimulate wider acceptance of IRT as a rapid, systematic and cost-effective evaluation technique for detecting bridge deck delaminations. The proposed combination of IRT and GPR results should expand their correlative use in bridge deck inspection. Integrating the proposed BDCI procedure with existing bridge management systems can provide a detailed and timely picture of bridge health, thus helping transportation agencies in identifying critical deficiencies at various service life stages. Consequently, this can yield sizeable reductions in bridge inspection costs, effective allocation of limited maintenance and repair funds, and promote the safety, mobility, longevity, and reliability of our highway transportation assets