Deep Reinforcement Learning Controller for 3D Path-following and Collision Avoidance by Autonomous Underwater Vehicles

Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, in complex systems, such as autonomous underwater vehicles performing the dual objective of path-following and collision avoidance, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning (DRL) techniques, to develop autonomous agents capable of achieving this hybrid objective without having \`a priori knowledge about the goal or the environment. Our results demonstrate the viability of DRL in path-following and avoiding collisions toward achieving human-level decision making in autonomous vehicle systems within extreme obstacle configurations

Development of Modeling and Simulation Platform for Path-Planning and Control of Autonomous Underwater Vehicles in Three-Dimensional Spaces

Autonomous underwater vehicles (AUVs) operating in deep sea and littoral environments have diverse applications including marine biology exploration, ocean environment monitoring, search for plane crash sites, inspection of ship-hulls and pipelines, underwater oil rig maintenance, border patrol, etc. Achieving autonomy in underwater vehicles relies on a tight integration between modules of sensing, navigation, decision-making, path-planning, trajectory tracking, and low-level control. This system integration task benefits from testing the related algorithms and techniques in a simulated environment before implementation in a physical test bed. This thesis reports on the development of a modeling and simulation platform that supports the design and testing of path planning and control algorithms in a synthetic AUV, representing a simulated version of a physical AUV. The approach allows integration between path-planners and closed-loop controllers that enable the synthetic AUV to track dynamically feasible trajectories in three-dimensional spaces. The dynamical behavior of the AUV is modeled using the equations of motion that incorporate the effects of external forces (e.g., buoyancy, gravity, hydrodynamic drag, centripetal force, Coriolis force, etc.), thrust forces, and inertial forces acting on the AUV. The equations of motion are translated into a state space formulation and the S-function feature of the Simulink and MATLAB scripts are used to evolve the state trajectories from initial conditions. A three-dimensional visualization of the resulting AUV motion is achieved by feeding the corresponding position and orientation states into an animation code. Experimental validation is carried out by performing integrated waypoint planner (e.g., using the popular A* algorithm) and PD controller implementations that allow the traversal of the synthetic AUV in two-dimensional (XY, XZ, YZ) and three-dimensional spaces. An underwater pipe-line inspection task carried out by the AUV is demonstrated in a simulated environment. The simulation testbed holds a potential to support planner and controller design for implementation in physical AUVs, thereby allowing exploration of various research topics in the field

Distributed Robust Learning-Based Backstepping Control Aided with Neurodynamics for Consensus Formation Tracking of Underwater Vessels

This paper addresses distributed robust learning-based control for consensus formation tracking of multiple underwater vessels, in which the system parameters of the marine vessels are assumed to be entirely unknown and subject to the modeling mismatch, oceanic disturbances, and noises. Towards this end, graph theory is used to allow us to synthesize the distributed controller with a stability guarantee. Due to the fact that the parameter uncertainties only arise in the vessels' dynamic model, the backstepping control technique is then employed. Subsequently, to overcome the difficulties in handling time-varying and unknown systems, an online learning procedure is developed in the proposed distributed formation control protocol. Moreover, modeling errors, environmental disturbances, and measurement noises are considered and tackled by introducing a neurodynamics model in the controller design to obtain a robust solution. Then, the stability analysis of the overall closed-loop system under the proposed scheme is provided to ensure the robust adaptive performance at the theoretical level. Finally, extensive simulation experiments are conducted to further verify the efficacy of the presented distributed control protocol