Towards automated sample collection and return in extreme underwater environments

© The Author(s), 2022. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Billings, G., Walter, M., Pizarro, O., Johnson-Roberson, M., & Camilli, R. Towards automated sample collection and return in extreme underwater environments. Journal of Field Robotics, 2(1), (2022): 1351–1385, https://doi.org/10.55417/fr.2022045.In this report, we present the system design, operational strategy, and results of coordinated multivehicle field demonstrations of autonomous marine robotic technologies in search-for-life missions within the Pacific shelf margin of Costa Rica and the Santorini-Kolumbo caldera complex, which serve as analogs to environments that may exist in oceans beyond Earth. This report focuses on the automation of remotely operated vehicle (ROV) manipulator operations for targeted biological sample-collection-and-return from the seafloor. In the context of future extraterrestrial exploration missions to ocean worlds, an ROV is an analog to a planetary lander, which must be capable of high-level autonomy. Our field trials involve two underwater vehicles, the SuBastian ROV and the Nereid Under Ice (NUI) hybrid ROV for mixed initiative (i.e., teleoperated or autonomous) missions, both equipped seven-degrees-of-freedom hydraulic manipulators. We describe an adaptable, hardware-independent computer vision architecture that enables high-level automated manipulation. The vision system provides a three-dimensional understanding of the workspace to inform manipulator motion planning in complex unstructured environments. We demonstrate the effectiveness of the vision system and control framework through field trials in increasingly challenging environments, including the automated collection and return of biological samples from within the active undersea volcano Kolumbo. Based on our experiences in the field, we discuss the performance of our system and identify promising directions for future research.This work was funded under a NASA PSTAR grant, number NNX16AL08G, and by the National Science Foundation under grants IIS-1830660 and IIS-1830500. The authors would like to thank the Costa Rican Ministry of Environment and Energy and National System of Conservation Areas for permitting research operations at the Costa Rican shelf margin, and the Schmidt Ocean Institute (including the captain and crew of the R/V Falkor and ROV SuBastian) for their generous support and making the FK181210 expedition safe and highly successful. Additionally, the authors would like to thank the Greek Ministry of Foreign Affairs for permitting the 2019 Kolumbo Expedition to the Kolumbo and Santorini calderas, as well as Prof. Evi Nomikou and Dr. Aggelos Mallios for their expert guidance and tireless contributions to the expedition

Development of a Hybrid Simulator for Underwater Vehicles with Manipulators

This article describes a hybrid simulation approach meant to facilitate the realization of a simulator for underwater vehicles with one or more manipulators capable of simulating the interaction of the vehicle with objects and structures of the environment. The hybrid simulation approach is first described and motivated analytically, then an analysis of simulation accuracy is proposed, where, in particular, the implications of added mass simulation are discussed. Then, a possible implementation of the proposed architecture is shown, where a robotic simulator of articulated bodies, capable of stable and accurate simulation of contact forces, although unfit to simulate any serious hydrodynamic model, is tightly interfaced with a general purpose dynamic systems simulator that is used to simulate the hydrodynamic forces, the vehicle guidance, navigation, and control system, and also a man-machine interface. Software details and the technicalities needed to interface the two simulators are also briefly presented. Finally, the results of the simulation of three operational scenarios are proposed as qualitative assessment of the simulator capabilities

Autonomous subsea intervention (SEAVENTION)

This paper presents the main results and latest developments in a 4-year project called autonomous subsea intervention (SEAVENTION). In the project we have developed new methods for autonomous inspection, maintenance and repair (IMR) in subsea oil and gas operations with Unmanned Underwater Vehicles (UUVs). The results are also relevant for offshore wind, aquaculture and other industries. We discuss the trends and status for UUV-based IMR in the oil and gas industry and provide an overview of the state of the art in intervention with UUVs. We also present a 3-level taxonomy for UUV autonomy: mission-level, task-level and vehicle-level. To achieve robust 6D underwater pose estimation of objects for UUV intervention, we have developed marker-less approaches with input from 2D and 3D cameras, as well as marker-based approaches with associated uncertainty. We have carried out experiments with varying turbidity to evaluate full 6D pose estimates in challenging conditions. We have also devised a sensor autocalibration method for UUV localization. For intervention, we have developed methods for autonomous underwater grasping and a novel vision-based distance estimator. For high-level task planning, we have evaluated two frameworks for automated planning and acting (AI planning). We have implemented AI planning for subsea inspection scenarios which have been analyzed and formulated in collaboration with the industry partners. One of the frameworks, called T-REX demonstrates a reactive behavior to the dynamic and potentially uncertain nature of subsea operations. We have also presented an architecture for comparing and choosing between mission plans when new mission goals are introduced.publishedVersio

Autonomous Underwater Intervention: Experimental Results of the MARIS Project

open11noopenSimetti, E. ;Wanderlingh, F. ;Torelli, S. ;Bibuli, M. ;Odetti, A. ;Bruzzone, G. ; Lodi Rizzini, D. ;Aleotti, J. ;Palli, G. ;Moriello, L. ;Scarcia, U.Simetti, E.; Wanderlingh, F.; Torelli, S.; Bibuli, M.; Odetti, Angelo; Bruzzone, G.; Lodi Rizzini, D.; Aleotti, J.; Palli, G.; Moriello, L.; Scarcia, U

Optimisation of docking locations for remotely operated vehicles.

This thesis describes work aimed at developing practical methods for determining the best docking locations for an underwater remotely operated vehicle (ROV) when inspecting an offshore platform. ROVs are used extensively in the offshore oil and gas industry to conduct a large variety of intervention tasks such as visual inspection, operational monitoring, equipment installation and operation, debris recovery, and so on. However, they have found only limited use in the more difficult tasks such as the detailed inspection of complex weld geometries. These complex welds are, however, found extensively in the construction of the majority of offshore structures and platforms ('oil rigs'). Furthermore, there is a safety requirement to have them inspected regularly since failure of these welds can potentially lead to catastrophic failure of the structures, the majority of which are manned. A number of specialist ROV systems have been developed that are able to attach onto platform structures and use their manipulators to conduct inspection. However, due to the short reach of the manipulators and the complex geometry of the welds (often encumbered with protruding pipes and other fittings) the success of any inspection is crucially dependent on a good initial choice of ROV docking position. This thesis will describe the problems and current manual planning methods, and then detail the development of two new methods for automated optimisation of docking positions - firstly using neural networks, and secondly using more conventional numerical processing. This thesis will also review related work in the field, such as the development of neural networks and their applications in the general offshore environment and in the control of ROVs and robot manipulator arms, and other approaches to ROV docking. It will further describe the use of the system developed here for planning docking positions on example commercial ROV inspection work programmes

Underwater Robots Part I: Current Systems and Problem Pose

International audienceThis paper constitutes the first part of a general overview of underwater robotics. The second part is titled: Underwater Robots Part II: existing solutions and open issues

Autonomous Grasping Using Novel Distance Estimator

This paper introduces a novel distance estimator using monocular vision for autonomous underwater grasping. The presented method is also applicable to topside grasping operations. The estimator is developed for robot manipulators with a monocular camera placed near the gripper. The fact that the camera is attached near the gripper makes it possible to design a method for capturing images from different positions, as the relative position change can be measured. The presented system can estimate relative distance to an object of unknown size with good precision. The manipulator applied in the presented work is the SeaArm-2, a fully electric underwater small modular manipulator. The manipulator is unique in its integrated monocular camera in the end-effector module, and its design facilitates the use of different end-effector tools. The camera is used for supervision, object detection, and tracking. The distance estimator was validated in a laboratory setting through autonomous grasping experiments. The manipulator was able to search for and find, estimate the relative distance of, grasp, and retrieve the relevant object in 12 out of 12 trials.publishedVersio

An overview of robotics and autonomous systems for harsh environments

Across a wide range of industries and applications, robotics and autonomous systems can fulfil the crucial and challenging tasks such as inspection, exploration, monitoring, drilling, sampling and mapping in areas of scientific discovery, disaster prevention, human rescue and infrastructure management, etc. However, in many situations, the associated environment is either too dangerous or inaccessible to humans. Hence, a wide range of robots have been developed and deployed to replace or aid humans in these activities. A look at these harsh environment applications of robotics demonstrate the diversity of technologies developed. This paper reviews some key application areas of robotics that involve interactions with harsh environments (such as search and rescue, space exploration, and deep-sea operations), gives an overview of the developed technologies and provides a discussion of the key trends and future directions common to many of these areas