Deep Reinforcement Learning Controller for 3D Path-following and Collision Avoidance by Autonomous Underwater Vehicles

Control theory provides engineers with a multitude of tools to design controllers that manipulate the closed-loop behavior and stability of dynamical systems. These methods rely heavily on insights about the mathematical model governing the physical system. However, in complex systems, such as autonomous underwater vehicles performing the dual objective of path-following and collision avoidance, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning (DRL) techniques, to develop autonomous agents capable of achieving this hybrid objective without having \`a priori knowledge about the goal or the environment. Our results demonstrate the viability of DRL in path-following and avoiding collisions toward achieving human-level decision making in autonomous vehicle systems within extreme obstacle configurations

A Hierarchal Planning Framework for AUV Mission Management in a Spatio-Temporal Varying Ocean

The purpose of this paper is to provide a hierarchical dynamic mission planning framework for a single autonomous underwater vehicle (AUV) to accomplish task-assign process in a limited time interval while operating in an uncertain undersea environment, where spatio-temporal variability of the operating field is taken into account. To this end, a high level reactive mission planner and a low level motion planning system are constructed. The high level system is responsible for task priority assignment and guiding the vehicle toward a target of interest considering on-time termination of the mission. The lower layer is in charge of generating optimal trajectories based on sequence of tasks and dynamicity of operating terrain. The mission planner is able to reactively re-arrange the tasks based on mission/terrain updates while the low level planner is capable of coping unexpected changes of the terrain by correcting the old path and re-generating a new trajectory. As a result, the vehicle is able to undertake the maximum number of tasks with certain degree of maneuverability having situational awareness of the operating field. The computational engine of the mentioned framework is based on the biogeography based optimization (BBO) algorithm that is capable of providing efficient solutions. To evaluate the performance of the proposed framework, firstly, a realistic model of undersea environment is provided based on realistic map data, and then several scenarios, treated as real experiments, are designed through the simulation study. Additionally, to show the robustness and reliability of the framework, Monte-Carlo simulation is carried out and statistical analysis is performed. The results of simulations indicate the significant potential of the two-level hierarchical mission planning system in mission success and its applicability for real-time implementation

Medium access control, error control and routing in underwater acoustic networks: a discussion on protocol design and implementation

The journey of underwater communication which began from Leonardo’s era took four and a half centuries to find practical applications for military purposes during World War II. However, over the last three decades, underwater acoustic communications witnessed a massive development due to the advancements in the design of underwater communicating peripherals and their supporting protocols. Successively, doors are opened for a wide range of applications to employ in the underwater environment, such as oceanography, pollution monitoring, offshore exploration, disaster prevention, navigation assistance, monitoring, coastal patrol and surveillance. Different applications may have different characteristics and hence, may require different network architectures. For instance, routing protocols designed for unpartitioned multi-hop networks are not suitable for Delay-Tolerant Networks. Furthermore, single-hop networks do not need routing protocols at all. Therefore, before developing a protocol one must study the network architecture properly and design it accordingly. There are several other factors which should also be considered with the network architecture while designing an efficient protocol for underwater networks, such as long propagation delay, limited bandwidth, limited battery power, high bit error rate of the channel and several other adverse properties of the channel, such as, multi-path, fading and refractive behaviors. Moreover, the environment also has an impact on the performance of the protocols designed for underwater networks. Even temperature changes in a single day have an impact on the performance of the protocols. A good protocol designed for any network should consider some or all of these characteristics to achieve better performance. In this thesis, we first discuss the impact of the environment on the performance of MAC and routing protocols. From our investigation, we discover that even temperature changes within a day may affect the sound speed profile and hence, the channel changes and the protocol performance vary. After that we discuss several protocols which are specifically designed for underwater acoustic networks to serve different purposes and for different network architectures. Underwater Selective Repeat (USR) is an error control protocol designed to assure reliable data transmission in the MAC layer. One may suspect that employing an error control technique over a channel which already suffers from long propagation delays is a burden. However, USR utilizes long propagation by transmitting multiple packets in a single RTT using an interlacing technique. After USR, a routing protocol for surveillance networks is discussed where some sensors are laid down at the bottom of the sea and some sinks are placed outside the area. If a sensor detects an asset within its detection range, it announces the presence of intruders by transmitting packets to the sinks. It may happen that the discovered asset is an enemy ship or an enemy submarine which creates noise to jam the network. Therefore, in surveillance networks, it is necessary that the protocols have jamming resistance capabilities. Moreover, since the network supports multiple sinks with similar anycast address, we propose a Jamming Resistance multi-path Multi-Sink Routing Protocol (MSRP) using a source routing technique. However, the problem of source routing is that it suffers from large overhead (every packet includes the whole path information) with respect to other routing techniques, and also suffers from the unidirectional link problem. Therefore, another routing protocol based on a distance vector technique, called Multi-path Routing with Limited Cross-Path Interference (L-CROP) protocol is proposed, which employs a neighbor-aware multi-path discovery algorithm to support low interference multiple paths between each source-destination pair. Following that, another routing protocol is discussed for next generation coastal patrol and surveillance network, called Underwater Delay-Tolerant Network (UDTN) routing where some AUVs carry out the patrolling work of a given area and report to a shore based control-center. Since the area to be patrolled is large, AUVs experience intermittent connectivity. In our proposed protocol, two nodes that understand to be in contact with each other calculate and divide their contact duration equally so that every node gets a fair share of the contact duration to exchange data. Moreover, a probabilistic spray technique is employed to restrict the number of packet transmissions and for error correction a modified version of USR is employed. In the appendix, we discuss a framework which was designed by our research group to realize underwater communication through simulation which is used in most of the simulations in this thesis, called DESERT Underwater (short for DEsign, Simulate, Emulate and Realize Test-beds for Underwater network protocols). It is an underwater extension of the NS-Miracle simulator to support the design and implementation of underwater network protocols. Its creation assists the researchers in to utilizing the same codes designed for the simulator to employ in actual hardware devices and test in the real underwater scenario

Risk analysis and decision making for autonomous underwater vehicles

Risk analysis for autonomous underwater vehicles (AUVs) is essential to enable AUVs to explore extreme and dynamic environments. This research aims to augment existing risk analysis methods for AUVs, and it proposes a suite of methods to quantify mission risks and to support the implementation of safety-based decision making strategies for AUVs in harsh marine environments. This research firstly provides a systematic review of past progress of risk analysis research for AUV operations. The review answers key questions including fundamental concepts and evolving methods in the domain of risk analysis for AUVs, and it highlights future research trends to bridge existing gaps. Based on the state-of-the-art research, a copula-based approach is proposed for predicting the risk of AUV loss in underwater environments. The developed copula Bayesian network (CBN) aims to handle non-linear dependencies among environmental variables and inherent technical failures for AUVs, and therefore achieve accurate risk estimation for vehicle loss given various environmental observations. Furthermore, path planning for AUVs is an effective decision making strategy for mitigating risks and ensuring safer routing. A further study presents an offboard risk-based path planning approach for AUVs, considering a challenging environment with oil spill scenarios incorporated. The proposed global Risk-A* planner combines a Bayesian-based risk model for probabilistic risk reasoning and an A*-based algorithm for path searching. However, global path planning designed for static environments cannot handle the unpredictable situations that may emerge, and real-time replanned solutions are required to account for dynamic environmental observations. Therefore, a hybrid risk-aware decision making strategy is investigated for AUVs to combine static global planning with dynamic local re-planning. A dynamic risk analysis model based on the system theoretic process analysis (STPA) and BN is applied for generating a real-time risk map in target mission areas. The dynamic window algorithm (DWA) serves for local path planning to avoid moving obstacles. The proposed hybrid risk-aware decisionmaking architecture is essential for the real-life implementation of AUVs, leading eventually to a real-time adaptive path planning process onboard the AUV

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Control of Autonomous Underwater Vehicles

In this thesis an overview of Autonomous Underwater Vehicles (AUV) is presented which covers the advancements in AUV technology in last two decades, different components of AUV and the applications of AUVs. A glimpse on AUV research in India is presented. A nonlinear model of AUV is obtained through kinematics and dynamics equation which is linearized about an operating point to get linearized pitch & depth plane model. A two loop controller (PI control) is used to control the pitch and in turn the depth of the AUV. After having developed, simulated and analyzed the pitch and depth controller for a single AUV, we focus our attention towards developing formation control of three AUVs. The formation control for multiple Autonomous Underwater Vehicles (AUVs) is considered in spatial motions.The objective is to drive a leader AUV along a desired trajectory, and make the follower AUVs keep a desired formation with respect to the leader’s configuration in 3-dimensional spaces (leader-follower formation control). Also, an obstacle avoidance scheme, using pitch and depth control, is used to avoid static obstacles in the path of AUV. The results of the above three control objectives such as tracking control of AUV, controller for avoiding obstacles and formation control of multiple AUVs are presented and discussed in the thesis