A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off.Peer reviewe

A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off

Controlling a cargo ship without human experience based on deep Q-network

Human experience is regarded as an indispensable part of artificial intelligence in the process of controlling or decision making for autonomous cargo ships. In this paper, a novel Deep Q-Network-based (DQN) approach is proposed, which performs satisfactorily in controlling a cargo ship automatically without any human experience. At the very beginning, we use the model of KRISO Very Large Crude Carrier (KVLCC2) to describe a cargo ship. To manipulate this ship has to conquer great inertia and relatively insufficient driving force. Subsequently, customary waterways, regulations, conventions are described with Artificial Potential Field and value-functions in DQN. Based on this, the artificial intelligence of planning and controlling a cargo ship can be obtained by undertaking sufficient training, which can control the ship directly, while avoiding collisions, keeping its position in the middle of the route as much as possible. In simulation experiments, it is demonstrated that such an approach performs better than manual works and other traditional methods in most conditions, which makes the proposed method a promising solution in improving the autonomy level of cargo ships

Path planning and collision avoidance for autonomous surface vehicles I: a review

Autonomous surface vehicles are gaining increasing attention worldwide due to the potential benefits of improving safety and efficiency. This has raised the interest in developing methods for path planning that can reduce the risk of collisions, groundings, and stranding accidents at sea, as well as costs and time expenditure. In this paper, we review guidance, and more specifically, path planning algorithms of autonomous surface vehicles and their classification. In particular, we highlight vessel autonomy, regulatory framework, guidance, navigation and control components, advances in the industry, and previous reviews in the field. In addition, we analyse the terminology used in the literature and attempt to clarify ambiguities in commonly used terms related to path planning. Finally, we summarise and discuss our findings and highlight the potential need for new regulations for autonomous surface vehicles

Path planning and collision avoidance for autonomous surface vehicles II: a comparative study of algorithms

Artificial intelligence is an enabling technology for autonomous surface vehicles, with methods such as evolutionary algorithms, artificial potential fields, fast marching methods, and many others becoming increasingly popular for solving problems such as path planning and collision avoidance. However, there currently is no unified way to evaluate the performance of different algorithms, for example with regard to safety or risk. This paper is a step in that direction and offers a comparative study of current state-of-the art path planning and collision avoidance algorithms for autonomous surface vehicles. Across 45 selected papers, we compare important performance properties of the proposed algorithms related to the vessel and the environment it is operating in. We also analyse how safety is incorporated, and what components constitute the objective function in these algorithms. Finally, we focus on comparing advantages and limitations of the 45 analysed papers. A key finding is the need for a unified platform for evaluating and comparing the performance of algorithms under a large set of possible real-world scenarios

Learn to navigate: cooperative path planning for unmanned surface vehicles using deep reinforcement learning

Unmanned surface vehicle (USV) has witnessed a rapid growth in the recent decade and has been applied in various practical applications in both military and civilian domains. USVs can either be deployed as a single unit or multiple vehicles in a fleet to conduct ocean missions. Central to the control of USV and USV formations, path planning is the key technology that ensures the navigation safety by generating collision free trajectories. Compared with conventional path planning algorithms, the deep reinforcement learning (RL) based planning algorithms provides a new resolution by integrating a high-level artificial intelligence. This work investigates the application of deep reinforcement learning algorithms for USV and USV formation path planning with specific focus on a reliable obstacle avoidance in constrained maritime environments. For single USV planning, with the primary aim being to calculate a shortest collision avoiding path, the designed RL path planning algorithm is able to solve other complex issues such as the compliance with vehicle motion constraints. The USV formation maintenance algorithm is capable of calculating suitable paths for the formation and retain the formation shape robustly or vary shapes where necessary, which is promising to assist with the navigation in environments with cluttered obstacles. The developed three sets of algorithms are validated and tested in computer-based simulations and practical maritime environments extracted from real harbour areas in the UK

Collision probability reduction method for tracking control in automatic docking/berthing using reinforcement learning

Automation of berthing maneuvers in shipping is a pressing issue as the berthing maneuver is one of the most stressful tasks seafarers undertake. Berthing control problems are often tackled by tracking a predefined trajectory or path. Maintaining a tracking error of zero under an uncertain environment is impossible; the tracking controller is nonetheless required to bring vessels close to desired berths. The tracking controller must prioritize the avoidance of tracking errors that may cause collisions with obstacles. This paper proposes a training method based on reinforcement learning for a trajectory tracking controller that reduces the probability of collisions with static obstacles. Via numerical simulations, we show that the proposed method reduces the probability of collisions during berthing maneuvers. Furthermore, this paper shows the tracking performance in a model experiment.The version of record of this article, first published in Journal of Marine Science and Technology (Japan), is available online at Publisher’s website: https://doi.org/10.1007/s00773-023-00962-

Spatial-temporal recurrent reinforcement learning for autonomous ships

The paper proposes a spatial-temporal recurrent neural network architecture for Deep $Q$-Networks to steer an autonomous ship. The network design allows handling an arbitrary number of surrounding target ships while offering robustness to partial observability. Further, a state-of-the-art collision risk metric is proposed to enable an easier assessment of different situations by the agent. The COLREG rules of maritime traffic are explicitly considered in the design of the reward function. The final policy is validated on a custom set of newly created single-ship encounters called "Around the Clock" problems and the commonly chosen Imazu (1987) problems, which include 18 multi-ship scenarios. Additionally, the framework shows robustness when deployed simultaneously in multi-agent scenarios. The proposed network architecture is compatible with other deep reinforcement learning algorithms, including actor-critic frameworks

Safe Maneuvering Near Offshore Installations: A New Algorithmic Tool

Maneuvers of human-operated and autonomous marine vessels in the safety zone of drilling rigs, wind farms and other installations present a risk of collision. This article proposes an algorithmic toolkit that ensures maneuver safety, taking into account the restrictions imposed by ship dynamics. The algorithms can be used for anomaly detection, decision making by a human operator or an unmanned vehicle guidance system. We also consider a response to failures in the vessel's control systems and emergency escape maneuvers. Data used by the algorithms come from the vessel's dynamic positioning control system and positional survey charts of the marine installations

COLREGs-compliant dynamic collision avoidance algorithm based on deep deterministic policy gradient

978-987In order to reduce collision avoidance accidents and improve the safety of ship navigation, a dynamic collision avoidance algorithm based on deep reinforcement learning is proposed in this paper. In order to avoid the fuzziness and uncertainty in the encounter process, the degree of risk is formulated to quantify the collision risk. International regulations for preventing collisions at sea (COLREGs) are quantified reasonably. Considering the factors of collision, position, speed, course and compliance with the COLREGs, the reward function of the algorithm is designed to ensure that the collision avoidance decision is safe and effective and meet the requirements of the COLREGs. Based on DDPG algorithm, the sample data processing mechanism is improved, the utilization rate of experience is improved, and the problems of long learning time and unstable training are solved. The navigation and collision avoidance for multiple ships are simulated respectively. The results show that this method can effectively avoid obstacle ships under the requirements of COLREGs, and it has good real-time performance and safety