Multi-objective reinforcement learning for AUV thruster failure recovery

This paper investigates learning approaches for discovering fault-tolerant control policies to overcome thruster failures in Autonomous Underwater Vehicles (AUV). The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the vehicle. When a fault is detected and isolated the model of the AUV is reconfigured according to the new condition. To discover a set of optimal solutions a multi-objective reinforcement learning approach is employed which can deal with multiple conflicting objectives. Each optimal solution can be used to generate a trajectory that is able to navigate the AUV towards a specified target while satisfying multiple objectives. The discovered policies are executed on the robot in a closed-loop using AUVs state feedback. Unlike most existing methods which disregard the faulty thruster, our approach can also deal with partially broken thrusters to increase the persistent autonomy of the AUV. In addition, the proposed approach is applicable when the AUV either becomes under-actuated or remains redundant in the presence of a fault. We validate the proposed approach on the model of the Girona500 AUV

A conceptual design of a propulsion system for an autonomous underwater vehicle

The need for developing propulsion systems to support missions of increased endurance for autonomous underwater vehicles is investigated and a conceptual system is proposed, based on currently available technology and desired system characteristics. The investigation evaluates and ranks alternative energy sources and proposes the use of a closed Brayton cycle gas turbine power plant using a chemical energy heat source with a metallic fuel. A thruster system using electric propulsion motors and screw propellers is selected. Evaluation factors include reliability, depth independent operation, weight, endurance, quietness and efficiency. Reliability of the proposed system is analyzed and the design modified to meet proposed reliability requirements. A knowledge-based system is developed to manage the operation of the propulsion plant in an autonomous manner. A simulation system is developed using Common Lisp and the operation of the propulsion plant and its knowledge-based management system are evaluated using the simulator

An energy-aware architecture : a practical implementation for autonomous underwater vehicles

Energy awareness, fault tolerance and performance estimation are important aspects for extending the autonomy levels of today’s autonomous vehicles. Those are related to the concepts of survivability and reliability, two important factors that often limit the trust of end users in conducting large-scale deployments of such vehicles. With the aim of preparing the way for persistent autonomous operations this work focuses its efforts on investigating those effects on underwater vehicles capable of long-term missions. A novel energy-aware architecture for autonomous underwater vehicles (AUVs) is presented. This, by monitoring at runtime the vehicle’s energy usage, is capable of detecting and mitigating failures in the propulsion subsystem, one of the most common sources of mission-time problems. Furthermore it estimates the vehicle’s performance when operating in unknown environments and in the presence of external disturbances. These capabilities are a great contribution for reducing the operational uncertainty that most underwater platforms face during their deployment. Using knowledge collected while conducting real missions the proposed architecture allows the optimisation of on-board resource usage. This improves the vehicle’s effectiveness when operating in unknown stochastic scenarios or when facing the problem of resource scarcity. The architecture has been implemented on a real vehicle, Nessie AUV, used for real sea experiments as part of multiple research projects. These gave the opportunity of evaluating the improvements of the proposed system when considering more complex autonomous tasks. Together with Nessie AUV, the commercial platform IVER3 AUV has been involved in the evaluating the feasibility of this approach. Results and operational experience, gathered both in real sea scenarios and in controlled environment experiments, are discussed in detail showing the benefits and the operational constraints of the introduced architecture, alongside suggestions for future research directions

Nonlinear H ∞ optimal control scheme for an underwater vehicle with regional function formulation

A conventional region control technique cannot meet the demands for an accurate tracking performance in view of its inability to accommodate highly nonlinear system dynamics, imprecise hydrodynamic coefficients, and external disturbances. In this paper, a robust technique is presented for an Autonomous Underwater Vehicle (AUV) with region tracking function. Within this control scheme, nonlinear H∞ and region based control schemes are used. A Lyapunov-like function is presented for stability analysis of the proposed control law. Numerical simulations are presented to demonstrate the performance of the proposed tracking control of the AUV. It is shown that the proposed control law is robust against parameter uncertainties, external disturbances, and nonlinearities and it leads to uniform ultimate boundedness of the region tracking error

Force-Canceling Mixer Algorithm for Vehicles with Fully-Articulated Radially Symmetric Thruster Arrays

A new type of fully-holonomic aerial vehicle is identified and developed that can optionally utilize automatic cancellation of excessive thruster forces to maintain precise control despite little or no throttle authority. After defining the physical attributes of the new vehicle, a flight control mixer algorithm is defined and presented. This mixer is an input/output abstraction that grants a flight control system (or pilot) full authority of the vehicle\u27s position and orientation by means of an input translation vector and input torque vector. The mixer is shown to be general with respect to the number of thrusters in the system provided that they are distributed in a radially symmetric array. As the mixer is designed to operate independently of the chosen flight control system, it is completely agnostic to the type of control methodology implemented. Validation of both the vehicle\u27s holonomic capabilities and efficacy of the flight control mixing algorithm are provided by a custom MATLAB-based rigid body simulation environment

Design and MATLAB Simulation of Pitch Motion System Controller for Underwater Vehicle

Underwater vehicle is an important machine nowadays. It can perform multiple underwater complex tasks. For example, pipelines detection or mapping, underwater terrain exploration and underwater inspections. Due to rotation of the thruster at back of most underwater vehicle, it causes disturbances of fluid around the vehicle and affect the stability of the vehicle. Thus, a control system for the motion of the vehicle should be designed to compensate the instability. However, in this project the focus is directed to design a PID controller for one degree of vehicle’s motion which is pitch motion. The study is based on NPS AUV II which is an underwater vehicle. Recently, there are many researches related to underwater vehicle’s motion controller

A control framework for a remotely operated vehicle

A control framework enabling the automated maneuvering of a Remotely Operate Vehicle (ROV) is presented. The control architecture is structured according to the principle of composition of vehicle motions from a minimal set of elemental maneuvers that are designed and verified independently. The principled approach is based on distributed hybrid systems techniques, and spans integrated design, simulation and implementation as the same model is used throughout. Hybrid systems control techniques are used to synthesize the elemental maneuvers and to design protocols, which coordinate the execution of elemental maneuvers within a complex maneuver. This work is part of the Inspection of Underwater Structures (IES) project whose main objective is the implementation of a ROV-based system for the inspection of underwater structures

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties