Reliable Navigational Scene Perception for Autonomous Ships in Maritime Environment

Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships

A Machine Learning Approach for Predicting Docking-Based Structures

A indústria marítima encontra-se ainda numa fase inicial relativamente à automatização de processos e integração da Inteligência Artificial (AI). O desenvolvimento de Veículos Autónomos de Superfície (ASVs) insere-se neste âmbito, trazendo a possibilidade de reduzir drasticamente a intervenção humana nesta indústria e as consequências inerentes ao erro humano. A investigação no contexto da operação deste tipo de veículos tem crescido recentemente, apesar de a sua utilização em larga escala ainda encontrar enúmeras limitações, tais como a necessidade de assegurar a segurança do veículo, intervenientes e infra-estruturas, e de desenvolver a tecnologia apropriada. O processo de atracagem encontra-se entre as operações mais complexas e exigentes para um veículo marítimo. De facto, a maioria dos acidentes marítimos ocorre em portos. Tendo em conta esta tendência, assim como a diminuta investigação científica acerca do tema que se verifica atualmente, o processo de atracagem de um veículo autónomo mostra-se como uma problemática interessante e promissora. Neste sentido, esta dissertação pretende contribuir para o processo de tornar um veículo marítimo capaz de atracar autonomamente, através do desenvolvimento de uma ferramenta de perceção necessária para esta manobra. O trabalho realizado consiste numa rede de Deep Learning (DL) capaz de detetar a presença de uma doca no meio circundante do veículo bem como de classificar a estrutura identificada. Considerando que não existe muita informação referente a este contexto, tornou-se necessário proceder a uma recolha de dados no simulador 3D Gazebo, durante a qual cinco modelos de diferentes docas foram colocados num ambiente marítimo simulado e foi capturada informação de vários sensores, como um LiDAR e um par de câmaras stereo. Os dados recolhidos foram usados para o treino do classificador, que é composto por duas redes em cascata: uma para detetar a existência de uma doca na vizinhança do veículo e outra para classificar o tipo da estrutura. Esta dissertação também propõe um algorithmo de análise da ocupação da doca, a partir de uma abordagem baseada em template matching. O trabalho desenvolvido foi testado nos dados recolhidos, assim como numa configuração mais dinâmica no simulador mencionado acima, tendo em consideração diferentes condições de ruído para melhor replicar um ambiente marítimo autêntico. O modelo de deteção obteve uma precisão de 96.44% em condições ótimas e uma precisão média de 90.43% na análise do efeito de ruído, com um desvio máximo de 2.8%. O modelo de classificação do tipo de doca obteve uma precisão máxima de 86.70% e média de 80.90% nos testes de ruído. O algoritmo de análise da ocupação permite determinar o número de vagas da estrutura assim como as respetivas coordenadas. Este trabalho foi desenvolvido com uma abordagem que permite facilmente a adaptação a outros tipos de docas, diferenciando-se, assim, de outros projetos da área.The maritime industry has only recently taken its first steps towards automation and Artificial Intelligence (AI). The development of Autonomous Surface Vehicles (ASVs) falls under this scope, uncovering the possibility of greatly reducing the role of human intervention in this industry and the inherent consequences of human error. Research concerning the operation of this type of vehicles has seen an upward trend in recent years, although its full-scale application still encounters various limitations, such as the need to ensure safety and develop the required technology. The docking and undocking processes are among the most challenging tasks for a maritime vehicle. In fact, most maritime accidents occur in sea ports and harbours. Such evidence, along with the scarceness of scientific endeavours in this topic, make the docking approach of an ASV an alluring matter to delve into. With this intention in mind, this dissertation aims to take one step further towards enabling a vessel to dock autonomously, by providing a perception tool necessary for this maneuver. The developed work comprises a Deep Learning (DL) network able to detect the existence of a docking platform in the vehicle's surrounding environment and classify the type of the structure. As data concerning this context is not widely available, it was necessary to conduct a data acquisition process in the 3D simulator Gazebo, in which models of five different types of docks were placed in a simulated maritime environment and data from several sensors, such as a LiDAR and a stereo camera, was captured. The gathered data was used for the training of the cascaded classifier, which was composed of two networks: a first one to ascertain whether there is a dock in the vicinity of the vehicle and a second one to categorize the structure. This dissertation also proposes a mechanism to perform an occupation analysis of the dock, based on a template matching approach. The developed work was tested on the gathered data and in a dynamic setup in the aforementioned simulator, taking into account different noise conditions to better replicate an authentic maritime environment. The detection model achieved an accuracy of 96.44% in optimal conditions and an average of 90.43% considering light to very severe noise conditions, with a deviation of 2.8% for the worst case. The categorizing model obtained a maximum accuracy of 86.70% and an average accuracy of 80.90% for noise tests. The occupation analysis algorithm is able to return the number and coordinates of the vacant spots of the dock. Both the cascade classifier and the occupation tool were developed through an approach that allows an easy adaptation to other types of docks, thus differentiating it from other works on the topic

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off

A Survey of Recent Machine Learning Solutions for Ship Collision Avoidance and Mission Planning

Machine Learning (ML) techniques have gained significant traction as a means of improving the autonomy of marine vehicles over the last few years. This article surveys the recent ML approaches utilised for ship collision avoidance (COLAV) and mission planning. Following an overview of the ever-expanding ML exploitation for maritime vehicles, key topics in the mission planning of ships are outlined. Notable papers with direct and indirect applications to the COLAV subject are technically reviewed and compared. Critiques, challenges, and future directions are also identified. The outcome clearly demonstrates the thriving research in this field, even though commercial marine ships incorporating machine intelligence able to perform autonomously under all operating conditions are still a long way off.Peer reviewe

Conception of control paradigms for teleoperated driving tasks in urban environments

Development of concepts and computationally efficient motion planning methods for teleoperated drivingEntwicklung von Konzepten und recheneffizienten Bewegungsplanungsmethoden für teleoperiertes Fahre

Advances in Artificial Intelligence: Models, Optimization, and Machine Learning

The present book contains all the articles accepted and published in the Special Issue “Advances in Artificial Intelligence: Models, Optimization, and Machine Learning” of the MDPI Mathematics journal, which covers a wide range of topics connected to the theory and applications of artificial intelligence and its subfields. These topics include, among others, deep learning and classic machine learning algorithms, neural modelling, architectures and learning algorithms, biologically inspired optimization algorithms, algorithms for autonomous driving, probabilistic models and Bayesian reasoning, intelligent agents and multiagent systems. We hope that the scientific results presented in this book will serve as valuable sources of documentation and inspiration for anyone willing to pursue research in artificial intelligence, machine learning and their widespread applications