Autonomous Drone Landings on an Unmanned Marine Vehicle using Deep Reinforcement Learning

This thesis describes with the integration of an Unmanned Surface Vehicle (USV) and an Unmanned Aerial Vehicle (UAV, also commonly known as drone) in a single Multi-Agent System (MAS). In marine robotics, the advantage offered by a MAS consists of exploiting the key features of a single robot to compensate for the shortcomings in the other. In this way, a USV can serve as the landing platform to alleviate the need for a UAV to be airborne for long periods time, whilst the latter can increase the overall environmental awareness thanks to the possibility to cover large portions of the prevailing environment with a camera (or more than one) mounted on it. There are numerous potential applications in which this system can be used, such as deployment in search and rescue missions, water and coastal monitoring, and reconnaissance and force protection, to name but a few. The theory developed is of a general nature. The landing manoeuvre has been accomplished mainly identifying, through artificial vision techniques, a fiducial marker placed on a flat surface serving as a landing platform. The raison d'etre for the thesis was to propose a new solution for autonomous landing that relies solely on onboard sensors and with minimum or no communications between the vehicles. To this end, initial work solved the problem while using only data from the cameras mounted on the in-flight drone. In the situation in which the tracking of the marker is interrupted, the current position of the USV is estimated and integrated into the control commands. The limitations of classic control theory used in this approached suggested the need for a new solution that empowered the flexibility of intelligent methods, such as fuzzy logic or artificial neural networks. The recent achievements obtained by deep reinforcement learning (DRL) techniques in end-to-end control in playing the Atari video-games suite represented a fascinating while challenging new way to see and address the landing problem. Therefore, novel architectures were designed for approximating the action-value function of a Q-learning algorithm and used to map raw input observation to high-level navigation actions. In this way, the UAV learnt how to land from high latitude without any human supervision, using only low-resolution grey-scale images and with a level of accuracy and robustness. Both the approaches have been implemented on a simulated test-bed based on Gazebo simulator and the model of the Parrot AR-Drone. The solution based on DRL was further verified experimentally using the Parrot Bebop 2 in a series of trials. The outcomes demonstrate that both these innovative methods are both feasible and practicable, not only in an outdoor marine scenario but also in indoor ones as well

A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off

A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off.Peer reviewe

Publications of the Jet Propulsion Laboratory, July 1961 through June 1962

Jpl bibliography on space science, 1961-196

An Open-Source Benchmark Simulator: Control of a BlueROV2 Underwater Robot

This paper presents a simulation model environment for the popular and low-cost remotely operated vehicle (ROV) BlueROV2 implemented in Simulink™ which has been designed and experimentally validated for benchmark control algorithms for underwater vehicles. The BlueROV2 model is based on Fossen’s equations and includes a kinematic model of the vehicle, the hydrodynamics of vehicle and water interaction, a dynamic model of the thrusters, and, lastly, the gravitational/buoyant forces. The hydrodynamic parameters and thruster model have been validated in a test facility. The benchmark model also includes the ocean current, modeled as constant velocity. The tether connecting the ROV to the top-site facility has been modeled using the lumped mass method and is implemented as a force input to the ROV model. At last, to show the usefulness of the benchmark model, a case study is presented where a BlueROV2 is deployed to inspect an offshore monopile structure. The case study uses a sliding mode controller designed for the BlueROV2. The controller fulfills the design criteria defined for the case study by following the provided trajectory with a low error. It is concluded that the simulator establishes a benchmark for future control schemes for position control and trajectory tracking under the influence of environmental disturbances

Pose Detection and Control of Unmanned Underwater Vehicles (UUVs) Utilizing an Optical Detector Array

As part of the research for development of a leader-follower formation between unmanned underwater vehicles (UUVs), this study presents an optical feedback system for UUV navigation via an optical detector array. Capabilities of pose detection and control in a static-dynamic system (e.g. UUV navigation into a docking station) and a dynamic-dynamic system (e.g. UUV to UUV leader-follower system) are investigated. In both systems, a single light source is utilized as a guiding beacon for a tracker/follower UUV. The UUV uses an optical array consisting of photodiodes to receive the light field emitted from the light source. For UUV navigation applications, accurate pose estimation is essential. In order to evaluate the feasibility of underwater distance detection, the effective communication range between two platforms, i.e. light source and optical detector, and the optimum spectral range that allowed maximum light transmission are calculated. Based on the light attenuation in underwater, the geometry and dimensions of an optical detector array are determined, and the boundary conditions for the developed pose detection algorithms along with the error sources in the experiments are identified. As a test bed to determine optical array dimensions and size, a simulator, i.e. numerical software, is developed, where planar and curved array geometries of varying number of elements are analytically compared and evaluated. Results show that the curved optical detector array is able to distinguish 5 degree- of-freedom (DOF) motion (translation in x, y, z-axes and pitch and yaw rotations) with respect to a single light source. Analytical pose detection and control algorithms are developed for both static-dynamic and dynamic-dynamic systems. Results show that a 5 x 5 curved detector array with the implementation of SMC is reasonably sufficient for practical UUV positioning applications. The capabilities of an optical detector array to determine the pose of a UUV in 3-DOF (x, y and z-axes) are experimentally tested. An experimental platform consisting of a 5 x 5 photodiode array mounted on a hemispherical surface is used to sample the light field emitted from a single light source. Pose detection algorithms are developed to detect pose for steady-state and dynamic cases. Monte Carlo analysis is conducted to assess the pose estimation uncertainty under varying environmental and hardware conditions such as water turbidity, temperature variations in water and electrically-based noise. Monte Carlo analysis results show that the pose uncertainties (within 95% confidence interval) associated with x, y and z-axes are 0.78 m, 0.67 m and 0.56 m, respectively. Experimental results demonstrate that x, y and z-axes pose estimates are accurate to within 0.5 m, 0.2 m and 0.2 m, respectively

AutoTuning Environment for Static Obstacle Avoidance Methods Applied to USVs

This work is focused on reactive Static Obstacle Avoidance (SOA) methods used to increase the autonomy of Unmanned Surface Vehicles (USVs). Currently, there are multiple approaches to avoid obstacles, which can be applied to different types of USV. In order to assist in the choice of the SOA method for a particular vessel and to accelerate the pretuning process necessary for its implementation, this paper proposes a new AutoTuning Environment for Static Obstacle Avoidance (ATESOA) methods applied to USVs. In this environment, a new simplified modelling of a LIDAR (Laser Imaging Detection and Ranging) sensor is proposed based on numerical simulations. This sensor model provides a realistic environment for the tuning of SOA methods that, due to its low load computation, is used by evolutionary algorithms for the autotuning. In order to analyze the proposed ATESOA, three SOA methods were adapted and implemented to consider the measurements given by the LIDAR model. Furthermore, a mathematical model is proposed and evaluated for using as USV in the simulation enviroment. The results obtained in numerical simulations show how the new ATESOA is able to adjust the SOA methods in scenarios with different obstacle distributions

Modelling and control of a UAV-USV collaboration scheme for fluvial operations

This thesis focuses on a Model Based Design approach to the dynamic modelling and control design of a multi-robot solution based on a collab- oration scheme between a UAV and USV. The purpose of the system is to provide a suitable platform to autonomously perform limnology related surveys. The dynamic models of both platforms are derived from a Newton- Euler formalism and implemented through block oriented modelling us- ing the Simscape Multibody toolset within Simulink. The implementation of both the simulation architecture and the control architecture are de- scribed and explained. This control architecture is based on PID feedback loops that are used for achieving control of the UAV and USV dynamics. Finally, the built simulator is used to asses the performance and relia- bility of the designed controllers and the dynamic modelling approaches selectedIngeniería en Tecnologías Industriale