Machine Learning for Enhanced Maritime Situation Awareness: Leveraging Historical AIS Data for Ship Trajectory Prediction

In this thesis, methods to support high level situation awareness in ship navigators through appropriate automation are investigated. Situation awareness relates to the perception of the environment (level 1), comprehension of the situation (level 2), and projection of future dynamics (level 3). Ship navigators likely conduct mental simulations of future ship traffic (level 3 projections), that facilitate proactive collision avoidance actions. Such actions may include minor speed and/or heading alterations that can prevent future close-encounter situations from arising, enhancing the overall safety of maritime operations. Currently, there is limited automation support for level 3 projections, where the most common approaches utilize linear predictions based on constant speed and course values. Such approaches, however, are not capable of predicting more complex ship behavior. Ship navigators likely facilitate such predictions by developing models for level 3 situation awareness through experience. It is, therefore, suggested in this thesis to develop methods that emulate the development of high level human situation awareness. This is facilitated by leveraging machine learning, where navigational experience is artificially represented by historical AIS data. First, methods are developed to emulate human situation awareness by developing categorization functions. In this manner, historical ship behavior is categorized to reflect distinct patterns. To facilitate this, machine learning is leveraged to generate meaningful representations of historical AIS trajectories, and discover clusters of specific behavior. Second, methods are developed to facilitate pattern matching of an observed trajectory segment to clusters of historical ship behavior. Finally, the research in this thesis presents methods to predict future ship behavior with respect to a given cluster. Such predictions are, furthermore, on a scale intended to support proactive collision avoidance actions. Two main approaches are used to facilitate these functions. The first utilizes eigendecomposition-based approaches via locally extracted AIS trajectory segments. Anomaly detection is also facilitated via this approach in support of the outlined functions. The second utilizes deep learning-based approaches applied to regionally extracted trajectories. Both approaches are found to be successful in discovering clusters of specific ship behavior in relevant data sets, classifying a trajectory segment to a given cluster or clusters, as well as predicting the future behavior. Furthermore, the local ship behavior techniques can be trained to facilitate live predictions. The deep learning-based techniques, however, require significantly more training time. These models will, therefore, need to be pre-trained. Once trained, however, the deep learning models will facilitate almost instantaneous predictions

Spatio-Temporal Deep Learning Approaches for Addressing Track Association Problem using Automatic Identification System (AIS) Data

In the realm of marine surveillance, track association constitutes a pivotal yet challenging task, involving the identification and tracking of unlabelled vessel trajectories. The need for accurate data association algorithms stems from the urge to spot unusual vessel movements or threat detection. These algorithms link sequential observations containing location and motion information to specific moving objects, helping to build their real-time trajectories. These threat detection algorithms will be useful when a vessel attempts to conceal its identity. The algorithm can then identify and track the specific vessel from its incoming signal. The data for this study is sourced from the Automatic Identification System, which serves as a communication medium between neighboring ships and the control center. While traditional methods have relied on sequential tracking and physics-based models, the emergence of deep learning has significantly transformed techniques typically used in trajectory prediction, clustering, and anomaly detection. This transformation is largely attributed to the deep learning algorithm’s capability to model complex nonlinear relationships while capturing both the spatial and temporal dynamics of ship movement. Capitalizing on this computational advantage, our study focuses on evaluating different deep learning architectures such as Multi Model Long Short-Term Memory (LSTM), 1D Convolutional-LSTM, and Temporal-Graph Convolutional Neural Networks— in addressing the problem of track association. The performance of these proposed models are compared against different deep learning algorithms specialized in track association tasks using several real-life AIS datasets

Maritime modular anomaly detection framework

Detecting maritime anomalies is an extremely important task for maritime agencies around the globe. With the number of vessels at seas growing exponentially, the need for novel automated methods to support them with their routines and upgrade existing technologies is undeniable. MARISA, the Maritime Integrated Surveillance Awareness project, aims at fostering collaboration between 22 governmental organisations and enhance the reaction and decision-making capabilities of the maritime authorities. This work describes our contributions to the development of MARISA’s common toolkit for the detection of maritime anomalies. These efforts, as part of a Masters’ dissertation, lead to the development of the Modular Anomaly Detection Framework, MAD-F, a full data pipe-line which applies efficient and reliable routines to raw vessel navigational data in order to output potential maritime vessel anomalies. The anomalies considered for this work were defined by the experts from various maritime institutions, through MARISA, and allowed us to implement solutions given the real needs in the industry. The MADF functionalities will be validated through actual real maritime exercises. In its current state, we believe that the MAD-F is able to support maritime agencies and be integrated into their legacy systems.Detetar anomalias marítimas é uma tarefa extremamente importante para agências marítimas á escala mundial. Com o número de embarcações em mar crescendo exponencial, a necessidade de desenvolver novas rotinas de suporte ás suas atividades e de atualizar as tecnologias existentes é inegável. MARISA, o projeto de Conscientização da Vigilância Integrada Marítima, visa fomentar a colaboração entre 22 organizações governamentais e melhorar as capacidades de reação e tomada de decisões das autoridades marítimas. Este trabalho descreve as nossas contribuições para o desenvolvimento do toolkit global MARISA, que tem como âmbito a deteção de anomalias marítimas. Estas contribuições servem como parte do desenvolvimento da Modular Anomaly Detection Framework (MAD-F), que serve como um data-pipeline completo que transforma dados de embarcações não estruturados em potenciais anomalias, através do uso de métodos eficientes para tal. As anomalias consideradas para este trabalho foram definidas através do projeto MARISA por especialistas marítimos, e permitiram-nos trabalhar em necessidades reais e atuais do sector. As funcionalidades desenvolvidas serão validadas através de exercícios marítimos reias. No estado atual do MAD-F acreditamos que este será capaz de apoiar agências marítimas, e de posteriormente ser integrado nos sistemas dos mesmos

Deep Learning Methods for Vessel Trajectory Prediction based on Recurrent Neural Networks

Data-driven methods open up unprecedented possibilities for maritime surveillance using Automatic Identification System (AIS) data. In this work, we explore deep learning strategies using historical AIS observations to address the problem of predicting future vessel trajectories with a prediction horizon of several hours. We propose novel sequence-to-sequence vessel trajectory prediction models based on encoder-decoder recurrent neural networks (RNNs) that are trained on historical trajectory data to predict future trajectory samples given previous observations. The proposed architecture combines Long Short-Term Memory (LSTM) RNNs for sequence modeling to encode the observed data and generate future predictions with different intermediate aggregation layers to capture space-time dependencies in sequential data. Experimental results on vessel trajectories from an AIS dataset made freely available by the Danish Maritime Authority show the effectiveness of deep-learning methods for trajectory prediction based on sequence-to-sequence neural networks, which achieve better performance than baseline approaches based on linear regression or on the Multi-Layer Perceptron (MLP) architecture. The comparative evaluation of results shows: i) the superiority of attention pooling over static pooling for the specific application, and ii) the remarkable performance improvement that can be obtained with labeled trajectories, i.e., when predictions are conditioned on a low-level context representation encoded from the sequence of past observations, as well as on additional inputs (e.g., port of departure or arrival) about the vessel's high-level intention, which may be available from AIS.Comment: Accepted for publications in IEEE Transactions on Aerospace and Electronic Systems, 17 pages, 9 figure

Incorporation of AIS data-based machine learning into unsupervised route planning for maritime autonomous surface ships

Maritime Autonomous Surface Ships (MASS) are deemed as the future of maritime transport. Although showing attractiveness in terms of the solutions to emerging challenges such as carbon emission and insufficient labor caused by black swan events such as COVID-19, the applications of MASS have revealed problems in practice, among which MASS navigation safety presents a prioritized concern. To ensure safety, rational route planning for MASS is evident as the most critical step to avoiding any relevant collision accidents. This paper aims to develop a holistic framework for the unsupervised route planning of MASS using machine learning methods based on Automatic Identification System (AIS) data, including the coherent steps of new feature measurement, pattern extraction, and route planning algorithms. Historical AIS data from manned ships are trained to extract and generate movement patterns. The route planning for MASS is derived from the movement patterns according to a dynamic optimization method and a feature extraction algorithm. Numerical experiments are constructed on real AIS data to demonstrate the effectiveness of the proposed method in solving the route planning for different types of MASS

Behavioral Model Anomaly Detection in Automatic Identification Systems (AIS)

Over 90% of all goods in the world, at some point in their life, are on a vessel at sea. Currently, the maritime industry relies on the Automatic Identification System (AIS) for collision avoidance and vessel tracking. AIS is an unencrypted, unauthenticated protocol that is vulnerable to various types of cyber attacks allowing malicious actors to alter the location of vessels. With the advent of the Ocean of Things (OoT), vessels are sharing more information than vessel location alone at sea. Increasingly, more information is becoming critical for safe and efficient operation at sea. This method is a novel approach of applying machine learning to build vessel behavior models that exploits such information. These models will allow vessels to detect anomalous communication from vessels nearby. This will enable vessels to determine the quality of the message shared between each other and, more critically, identify malicious actors

Multiple-Aspect Analysis of Semantic Trajectories

This open access book constitutes the refereed post-conference proceedings of the First International Workshop on Multiple-Aspect Analysis of Semantic Trajectories, MASTER 2019, held in conjunction with the 19th European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2019, in Würzburg, Germany, in September 2019. The 8 full papers presented were carefully reviewed and selected from 12 submissions. They represent an interesting mix of techniques to solve recurrent as well as new problems in the semantic trajectory domain, such as data representation models, data management systems, machine learning approaches for anomaly detection, and common pathways identification

Context Exploitation in Data Fusion

Complex and dynamic environments constitute a challenge for existing tracking algorithms. For this reason, modern solutions are trying to utilize any available information which could help to constrain, improve or explain the measurements. So called Context Information (CI) is understood as information that surrounds an element of interest, whose knowledge may help understanding the (estimated) situation and also in reacting to that situation. However, context discovery and exploitation are still largely unexplored research topics. Until now, the context has been extensively exploited as a parameter in system and measurement models which led to the development of numerous approaches for the linear or non-linear constrained estimation and target tracking. More specifically, the spatial or static context is the most common source of the ambient information, i.e. features, utilized for recursive enhancement of the state variables either in the prediction or the measurement update of the filters. In the case of multiple model estimators, context can not only be related to the state but also to a certain mode of the filter. Common practice for multiple model scenarios is to represent states and context as a joint distribution of Gaussian mixtures. These approaches are commonly referred as the join tracking and classification. Alternatively, the usefulness of context was also demonstrated in aiding the measurement data association. Process of formulating a hypothesis, which assigns a particular measurement to the track, is traditionally governed by the empirical knowledge of the noise characteristics of sensors and operating environment, i.e. probability of detection, false alarm, clutter noise, which can be further enhanced by conditioning on context. We believe that interactions between the environment and the object could be classified into actions, activities and intents, and formed into structured graphs with contextual links translated into arcs. By learning the environment model we will be able to make prediction on the target\u2019s future actions based on its past observation. Probability of target future action could be utilized in the fusion process to adjust tracker confidence on measurements. By incorporating contextual knowledge of the environment, in the form of a likelihood function, in the filter measurement update step, we have been able to reduce uncertainties of the tracking solution and improve the consistency of the track. The promising results demonstrate that the fusion of CI brings a significant performance improvement in comparison to the regular tracking approaches