A model based design framework for safety verification of a semi-autonomous inspection drone

In this paper, we present a model based design approach to the development of a semi-autonomous control system for an inspection drone. The system is tasked with maintaining a set distance from the target being inspected and a constant relative pose, allowing the operator to manoeuvre the drone around the target with ease. It is essential that the robustness of the autonomous behaviour be thoroughly verified prior to actual implementation, as this will involve the flight of a large multi-rotor drone in close proximity to a solid structure. By utilising the Robotic Operating System to communicate between the autonomous controller and the drone, the same Simulink model can be used for numerical coverage testing, high fidelity simulation, offboard execution and final executable deploymen

Autonomous and scalable control for remote inspection with multiple aerial vehicles

© 2016 Elsevier B.V.A novel approach to the autonomous generation of trajectories for multiple aerial vehicles is presented, whereby an artificial kinematic field provides autonomous control in a distributed and highly scalable manner. The kinematic field is generated relative to a central target and is modified when a vehicle is in close proximity of another to avoid collisions. This control scheme is then applied to the mock visual inspection of a nuclear intermediate level waste storage drum. The inspection is completed using two commercially available quadcopters, in a laboratory environment, with the acquired visual inspection data processed and photogrammetrically meshed to generate a three-dimensional surface-meshed model of the drum. This paper contributes to the field of multi-agent coverage path planning for structural inspection and provides experimental validation of the control and inspection results

Human-machine interaction for unmanned surface systems

This research investigated the human-machine interaction (HMI) technologies for human-robot teams operating as unmanned surface systems (USS). An pilot role was found to be the most prevalent in the USS-related literature but additional human roles were determined to likely be necessary (e.g., Mission Specialist} though were not documented; interface needs have not yet been determined for any role. The human interfaces used by 67 Micro and Small X, Intermediate, Harbor, Fleet, and E,F,G-Class platforms were examined and it was determined that: i) the research literature does not well characterize the human roles present in unmanned surface systems, ii) domain complexity may necessitate increased automation of the robot platform for the human team, and iii) that unmanned surface vehicles likely lay on the human-machine interaction spectrum between unmanned ground vehicles and unmanned aerial vehicles. This work is expected to serve as a reference for future design and refinement of human interfaces for USSs and as a foundation for better understanding HMI in USSs

Context-Enabled Visualization Strategies for Automation Enabled Human-in-the-loop Inspection Systems to Enhance the Situation Awareness of Windstorm Risk Engineers

Insurance loss prevention survey, specifically windstorm risk inspection survey is the process of investigating potential damages associated with a building or structure in the event of an extreme weather condition such as a hurricane or tornado. Traditionally, the risk inspection process is highly subjective and depends on the skills of the engineer performing it. This dissertation investigates the sensemaking process of risk engineers while performing risk inspection with special focus on various factors influencing it. This research then investigates how context-based visualizations strategies enhance the situation awareness and performance of windstorm risk engineers. An initial study investigated the sensemaking process and situation awareness requirements of the windstorm risk engineers. The data frame theory of sensemaking was used as the framework to carry out this study. Ten windstorm risk engineers were interviewed, and the data collected were analyzed following an inductive thematic approach. The themes emerged from the data explained the sensemaking process of risk engineers, the process of making sense of contradicting information, importance of their experience level, internal and external biases influencing the inspection process, difficulty developing mental models, and potential technology interventions. More recently human in the loop systems such as drones have been used to improve the efficiency of windstorm risk inspection. This study provides recommendations to guide the design of such systems to support the sensemaking process and situation awareness of windstorm visual risk inspection. The second study investigated the effect of context-based visualization strategies to enhance the situation awareness of the windstorm risk engineers. More specifically, the study investigated how different types of information contribute towards the three levels of situation awareness. Following a between subjects study design 65 civil/construction engineering students completed this study. A checklist based and predictive display based decision aids were tested and found to be effective in supporting the situation awareness requirements as well as performance of windstorm risk engineers. However, the predictive display only helped with certain tasks like understanding the interaction among different components on the rooftop. For remaining tasks, checklist alone was sufficient. Moreover, the decision aids did not place any additional cognitive demand on the participants. This study helped us understand the advantages and disadvantages of the decision aids tested. The final study evaluated the transfer of training effect of the checklist and predictive display based decision aids. After one week of the previous study, participants completed a follow-up study without any decision aids. The performance and situation awareness of participants in the checklist and predictive display group did not change significantly from first trial to second trial. However, the performance and situation awareness of participants in the control condition improved significantly in the second trial. They attributed this to their exposure to SAGAT questionnaire in the first study. They knew what issues to look for and what tasks need to be completed in the simulation. The confounding effect of SAGAT questionnaires needs to be studied in future research efforts

Machine learning techniques for robotic and autonomous inspection of mechanical systems and civil infrastructure

Machine learning and in particular deep learning techniques have demonstrated the most efficacy in training, learning, analyzing, and modelling large complex structured and unstructured datasets. These techniques have recently been commonly deployed in different industries to support robotic and autonomous system (RAS) requirements and applications ranging from planning and navigation to machine vision and robot manipulation in complex environments. This paper reviews the state-of-the-art with regard to RAS technologies (including unmanned marine robot systems, unmanned ground robot systems, climbing and crawler robots, unmanned aerial vehicles, and space robot systems) and their application for the inspection and monitoring of mechanical systems and civil infrastructure. We explore various types of data provided by such systems and the analytical techniques being adopted to process and analyze these data. This paper provides a brief overview of machine learning and deep learning techniques, and more importantly, a classification of the literature which have reported the deployment of such techniques for RAS-based inspection and monitoring of utility pipelines, wind turbines, aircrafts, power lines, pressure vessels, bridges, etc. Our research provides documented information on the use of advanced data-driven technologies in the analysis of critical assets and examines the main challenges to the applications of such technologies in the industry

Collaborative Unmanned Vehicles for Inspection, Maintenance, and Repairs of Offshore Wind Turbines

Operations and maintenance of Offshore Wind Turbines (OWTs) are challenging, with manual operators constantly exposed to hazardous environments. Due to the high task complexity associated with the OWT, the transition to unmanned solutions remains stagnant. Efforts toward unmanned operations have been observed using Unmanned Aerial Vehicles (UAVs) and Unmanned Underwater Vehicles (UUVs) but are limited mostly to visual inspections only. Collaboration strategies between unmanned vehicles have introduced several opportunities that would enable unmanned operations for the OWT maintenance and repair activities. There have been many papers and reviews on collaborative UVs. However, most of the past papers reviewed collaborative UVs for surveillance purposes, search and rescue missions, and agricultural activities. This review aims to present the current capabilities of Unmanned Vehicles (UVs) used in OWT for Inspection, Maintenance, and Repair (IMR) operations. Strategies to implement collaborative UVs for complex tasks and their associated challenges are discussed together with the strategies to solve localization and navigation issues, prolong operation time, and establish effective communication within the OWT IMR operations. This paper also briefly discusses the potential failure modes for collaborative approaches and possible redundancy strategies to manage them. The collaborative strategies discussed herein will be of use to researchers and technology providers in identifying significant gaps that have hindered the implementation of full unmanned systems which have significant impacts towards the net zero strategy.</jats:p

Evaluation of the utility and performance of an autonomous surface vehicle for mobile monitoring of waterborne biochemical agents

Real-time water quality monitoring is crucial due to land utilization increases which can negatively impact aquatic ecosystems from surface water runoff. Conventional monitoring methodologies are laborious, expensive, and spatio-temporally limited. Autonomous surface vehicles (ASVs), equipped with sensors/instrumentation, serve as mobile sampling stations that reduce labor and enhance data resolution. However, ASV autopilot navigational accuracy is affected by environmental forces (wind, current, and waves) that can alter trajectories of planned paths and negatively affect spatio-temporal resolution of water quality data. This study demonstrated a commercially available solar powered ASV equipped with a multi-sensor payload ability to operate autonomously to accurately and repeatedly maintain established A-B line transects under varying environmental conditions, where lateral deviation from a planned linear route was measured and expressed as cross-track error (XTE). This work provides a framework for development of spatial/temporal resolution limitations of ASVs for real-time monitoring campaigns and future development of in-situ sampling technologies

Reference Model for Interoperability of Autonomous Systems

This thesis proposes a reference model to describe the components of an Un-manned Air, Ground, Surface, or Underwater System (UxS), and the use of a single Interoperability Building Block to command, control, and get feedback from such vehicles. The importance and advantages of such a reference model, with a standard nomenclature and taxonomy, is shown. We overview the concepts of interoperability and some efforts to achieve common refer-ence models in other areas. We then present an overview of existing un-manned systems, their history, characteristics, classification, and missions. The concept of Interoperability Building Blocks (IBB) is introduced to describe standards, protocols, data models, and frameworks, and a large set of these are analyzed. A new and powerful reference model for UxS, named RAMP, is proposed, that describes the various components that a UxS may have. It is a hierarchical model with four levels, that describes the vehicle components, the datalink, and the ground segment. The reference model is validated by showing how it can be applied in various projects the author worked on. An example is given on how a single standard was capable of controlling a set of heterogeneous UAVs, USVs, and UGVs

Standardization Roadmap for Unmanned Aircraft Systems, Version 1.0

This Standardization Roadmap for Unmanned Aircraft Systems, Version 1.0 (“roadmap”) represents the culmination of the UASSC’s work to identify existing standards and standards in development, assess gaps, and make recommendations for priority areas where there is a perceived need for additional standardization and/or pre-standardization R&D. The roadmap has examined 64 issue areas, identified a total of 60 gaps and corresponding recommendations across the topical areas of airworthiness; flight operations (both general concerns and application-specific ones including critical infrastructure inspections, commercial services, and public safety operations); and personnel training, qualifications, and certification. Of that total, 40 gaps/recommendations have been identified as high priority, 17 as medium priority, and 3 as low priority. A “gap” means no published standard or specification exists that covers the particular issue in question. In 36 cases, additional R&D is needed. The hope is that the roadmap will be broadly adopted by the standards community and that it will facilitate a more coherent and coordinated approach to the future development of standards for UAS. To that end, it is envisioned that the roadmap will be widely promoted and discussed over the course of the coming year, to assess progress on its implementation and to identify emerging issues that require further elaboration