Are object detection assessment criteria ready for maritime computer vision?

Maritime vessels equipped with visible and infrared cameras can complement other conventional sensors for object detection. However, application of computer vision techniques in maritime domain received attention only recently. The maritime environment offers its own unique requirements and challenges. Assessment of the quality of detections is a fundamental need in computer vision. However, the conventional assessment metrics suitable for usual object detection are deficient in the maritime setting. Thus, a large body of related work in computer vision appears inapplicable to the maritime setting at the first sight. We discuss the problem of defining assessment metrics suitable for maritime computer vision. We consider new bottom edge proximity metrics as assessment metrics for maritime computer vision. These metrics indicate that existing computer vision approaches are indeed promising for maritime computer vision and can play a foundational role in the emerging field of maritime computer vision

Spatial-temporal recurrent reinforcement learning for autonomous ships

The paper proposes a spatial-temporal recurrent neural network architecture for Deep $Q$-Networks to steer an autonomous ship. The network design allows handling an arbitrary number of surrounding target ships while offering robustness to partial observability. Further, a state-of-the-art collision risk metric is proposed to enable an easier assessment of different situations by the agent. The COLREG rules of maritime traffic are explicitly considered in the design of the reward function. The final policy is validated on a custom set of newly created single-ship encounters called "Around the Clock" problems and the commonly chosen Imazu (1987) problems, which include 18 multi-ship scenarios. Additionally, the framework shows robustness when deployed simultaneously in multi-agent scenarios. The proposed network architecture is compatible with other deep reinforcement learning algorithms, including actor-critic frameworks

Quadruplet Selection Methods for Deep Embedding Learning

Recognition of objects with subtle differences has been used in many practical applications, such as car model recognition and maritime vessel identification. For discrimination of the objects in fine-grained detail, we focus on deep embedding learning by using a multi-task learning framework, in which the hierarchical labels (coarse and fine labels) of the samples are utilized both for classification and a quadruplet-based loss function. In order to improve the recognition strength of the learned features, we present a novel feature selection method specifically designed for four training samples of a quadruplet. By experiments, it is observed that the selection of very hard negative samples with relatively easy positive ones from the same coarse and fine classes significantly increases some performance metrics in a fine-grained dataset when compared to selecting the quadruplet samples randomly. The feature embedding learned by the proposed method achieves favorable performance against its state-of-the-art counterparts.Comment: 6 pages, 2 figures, accepted by IEEE ICIP 201

Spatio-Temporal Deep Learning Approaches for Addressing Track Association Problem using Automatic Identification System (AIS) Data

In the realm of marine surveillance, track association constitutes a pivotal yet challenging task, involving the identification and tracking of unlabelled vessel trajectories. The need for accurate data association algorithms stems from the urge to spot unusual vessel movements or threat detection. These algorithms link sequential observations containing location and motion information to specific moving objects, helping to build their real-time trajectories. These threat detection algorithms will be useful when a vessel attempts to conceal its identity. The algorithm can then identify and track the specific vessel from its incoming signal. The data for this study is sourced from the Automatic Identification System, which serves as a communication medium between neighboring ships and the control center. While traditional methods have relied on sequential tracking and physics-based models, the emergence of deep learning has significantly transformed techniques typically used in trajectory prediction, clustering, and anomaly detection. This transformation is largely attributed to the deep learning algorithm’s capability to model complex nonlinear relationships while capturing both the spatial and temporal dynamics of ship movement. Capitalizing on this computational advantage, our study focuses on evaluating different deep learning architectures such as Multi Model Long Short-Term Memory (LSTM), 1D Convolutional-LSTM, and Temporal-Graph Convolutional Neural Networks— in addressing the problem of track association. The performance of these proposed models are compared against different deep learning algorithms specialized in track association tasks using several real-life AIS datasets

Automatic Maritime Traffic Anomalous Behaviors Detection

Maritime traffic plays a very important role in the world economy, with over 90% of global trading done through naval transportation. The high amount of vessel traffic, mainly due to cargo transportation, leads to several new risks, threats, and concerns, such as increased criminal activity in the sea. The OVERSEE project is proprietary software developed by Crit ical Software and used by Marinha Portuguesa, Irish Coast Guard, and Papua New Guinea’s Coast Guard. The OVERSEE project displays vessel information in real-time through AIS messages, which are mandatory for most cargo vessels to report consistently. Anomaly de tection and behavior monitoring tools are computer-based systems that analyse real-time data to detect anomalous behaviors. This project aims to develop a solution capable of detecting anomalous behaviors committed by vessels using AIS messages, which will be re ported in real-time automatically via e-mail and the extant OVERSEE graphical interface. The solution is developed with the use of Long Short-Term Memory Recurrent Neural Net works, and a deeper analysis is provided to compare the obtained results with the ideal results. The network training and testing are done with real data, with cross-classification techniques to improve the trustworthiness of the algorithm, hence providing more accurate results.O tráfego marítimo desempenha um papel muito importante na economia mundial, com mais de 90% do comércio global feito por meio do transporte naval. O grande volume de tráfego de embarcações, principalmente devido ao transporte de cargas, leva a vários novos riscos, ameaças e preocupações, como o aumento da criminalidade no mar. O projeto OVERSEE é um software proprietário desenvolvido pela Critical Software e usado pela Marinha Portuguesa, Guarda Costeira Irlandesa e Guarda Costeira da Papua Nova Guiné. O projeto OVERSEE exibe informações da embarcação em tempo real por meio de mensagens AIS, cuja maioria das embarcações de carga são obrigadas a relatar num período de tempo regular. As ferramentas de detecção de anomalias e monitoramento de comportamento são sistemas baseados em computador que analisam dados em tempo real para detetar comportamentos anómalos. Este projeto visa desenvolver uma solução capaz de detetar comportamentos anómalos cometidos por embarcações por meio de mensagens AIS, que serão reportados em tempo real automaticamente via e-mail e interface gráfica existente do OVERSEE. A solução está desenvolvida com o uso de Redes Neurais Recorrentes1 de Memória-Curta de Longo Prazo2 . Uma análise mais profunda é fornecida para comparar os resultados obtidos com os resultados ideais. O treinamento e teste da rede são feitos com dados reais, com técnicas de classificação cruzada para melhorar a confiabilidade do algoritmo, fornecendo resultados mais precisos