Deep Learning Methods for Vessel Trajectory Prediction based on Recurrent Neural Networks

Data-driven methods open up unprecedented possibilities for maritime surveillance using Automatic Identification System (AIS) data. In this work, we explore deep learning strategies using historical AIS observations to address the problem of predicting future vessel trajectories with a prediction horizon of several hours. We propose novel sequence-to-sequence vessel trajectory prediction models based on encoder-decoder recurrent neural networks (RNNs) that are trained on historical trajectory data to predict future trajectory samples given previous observations. The proposed architecture combines Long Short-Term Memory (LSTM) RNNs for sequence modeling to encode the observed data and generate future predictions with different intermediate aggregation layers to capture space-time dependencies in sequential data. Experimental results on vessel trajectories from an AIS dataset made freely available by the Danish Maritime Authority show the effectiveness of deep-learning methods for trajectory prediction based on sequence-to-sequence neural networks, which achieve better performance than baseline approaches based on linear regression or on the Multi-Layer Perceptron (MLP) architecture. The comparative evaluation of results shows: i) the superiority of attention pooling over static pooling for the specific application, and ii) the remarkable performance improvement that can be obtained with labeled trajectories, i.e., when predictions are conditioned on a low-level context representation encoded from the sequence of past observations, as well as on additional inputs (e.g., port of departure or arrival) about the vessel's high-level intention, which may be available from AIS.Comment: Accepted for publications in IEEE Transactions on Aerospace and Electronic Systems, 17 pages, 9 figure

Jump Particle Filtering Framework for Joint Target Tracking and Intent Recognition

This paper presents a Bayesian framework for inferring the posterior of the extended state of a target, incorporating its underlying goal or intent, such as any intermediate waypoints and/or final destination. The methodology is thus for joint tracking and intent recognition. Several novel latent intent models are proposed here within a virtual leader formulation. They capture the influence of the target's hidden goal on its instantaneous behaviour. In this context, various motion models, including for highly maneuvering objects, are also considered. The a priori unknown target intent (e.g. destination) can dynamically change over time and take any value within the state space (e.g. a location or spatial region). A sequential Monte Carlo (particle filtering) approach is introduced for the simultaneous estimation of the target's (kinematic) state and its intent. Rao-Blackwellisation is employed to enhance the statistical performance of the inference routine. Simulated data and real radar measurements are used to demonstrate the efficacy of the proposed techniques.Comment: Submitted to IEEE Transactions on Aerospace and Electronic Systems (T-AES

Modeling intent and destination prediction within a Bayesian framework: Predictive touch as a usecase

Abstract In various scenarios, the motion of a tracked object, for example, a pointing apparatus, pedestrian, animal, vehicle, and others, is driven by achieving a premeditated goal such as reaching a destination. This is albeit the various possible trajectories to this endpoint. This paper presents a generic Bayesian framework that utilizes stochastic models that can capture the influence of intent (viz., destination) on the object behavior. It leads to simple algorithms to infer, as early as possible, the intended endpoint from noisy sensory observations, with relatively low computational and training data requirements. This framework is introduced in the context of the novel predictive touch technology for intelligent user interfaces and touchless interactions. It can determine, early in the interaction task or pointing gesture, the interface item the user intends to select on the display (e.g., touchscreen) and accordingly simplify as well as expedite the selection task. This is shown to significantly improve the usability of displays in vehicles, especially under the influence of perturbations due to road and driving conditions, and enable intuitive contact-free interactions. Data collected in instrumented vehicles are shown to demonstrate the effectiveness of the proposed intent prediction approach.</jats:p

Detection of malicious intent in non-cooperative drone surveillance

In this paper, a Bayesian approach is proposed for the early detection of a drone threatening or anomalous behaviour in a surveyed region. This is in relation to revealing, as early as possible, the drone intent to either leave a geographical area where it is authorised to fly (e.g. to conduct inspection work) or reach a prohibited zone (e.g. runway protection zones at airports or a critical infrastructure site). The inference here is based on the noisy sensory observations of the target state from a non-cooperative surveillance system such as a radar. Data from Aveillant's Gamekeeper radar from a live drone trial is used to illustrate the efficacy of the introduced approach

Bayesian Approaches to Tracking, Sensor Fusion and Intent Prediction

This thesis presents work on the development of model-based Bayesian approaches to object tracking and intent prediction. Successful navigation/positioning applications rely fundamentally on the choice of appropriate dynamic model and the design of effective tracking algorithms capable of maximising the use of the structure of the dynamic system and the information from sensors. While the tracking problem with frequent and accurate position data has been well studied, we push back the frontiers of current technology where an object can undergo fast manoeuvres and position fixes are limited. On the other hand, intent prediction techniques which extract higher level information such as the intended destination of a moving object can be designed, given the ability to perform successful tracking. Such techniques can play important roles in various application areas, including traffic monitoring, intelligent human computer interaction systems and autonomous route planning. In the first part of this thesis Bayesian tracking methods are designed based on a standard fix-rate setting in which the dynamic system is formulated into a Markovian state space form. We show that the combination of an intrinsic coordinate dynamic model and sensors in the object's body frame leads to novel state space models according to which efficient proposal kernels can be designed and implemented by the sequential Monte Carlo (SMC) methods. Also, sequential Markov chain Monte Carlo schemes are considered for the first time to tackle the sequential batch inference problems due to the presence of infrequent position data. Performance evaluation on both synthetic and real-world data shows that the proposed algorithms are superior to simpler particle filters, implying that they can be favourable alternatives to tracking problems with inertial sensors. The modelling assumption that leads to Markovian state space models can be restrictive for real-world systems as it stipulates that the state sequence has to be synchronised with the observations. In the second major part of this thesis we relax this assumption and work with a more natural class of models, termed variable rate models. We generalise the existing variable rate intrinsic model to incorporate acceleration, speed, distance and position data and introduce new variable rate particle filtering methods tailored to the derived model to accommodate multi-sensor multi-rate tracking scenarios. The proposed algorithms can achieve substantial improvements in terms of tracking accuracy and robustness over a bootstrap variable rate particle filter. Moreover, full Bayesian inference schemes for the learning of both the hidden state and system parameters are presented, with numerical results illustrating their effectiveness. The last part of the thesis is about designing efficient intent prediction algorithms within a Bayesian framework. A pseudo-observation based approach to the incorporation of destination knowledge is introduced, making the mathematics of the dynamical model and the observation process consistent with the Markov state process. Based on the new interpretation, two algorithms are proposed to sequentially estimate the probability of all possible endpoints. Whilst the synthetic maritime surveillance data demonstrate that the proposed methods can achieve comparable prediction performance with reduced computational cost in comparison to the existing bridging distribution based methods, the results on an extensive freehand pointing database, which contains 95 three-dimensional pointing trajectories, show that the new algorithms can outperform other state-of-the-art techniques. Some sensitivity tests are also performed, confirming the good robustness of the introduced methods against model mismatches

Intent-informed state estimation for tracking guided targets

This paper proposes a state estimation and prediction for tracking guided targets using intent information. A conditionally Markov process is used to describe the destination-oriented target motion, and the collision intent is incorporated through the zero-effort-miss guidance information. The expected arrival time necessary for the conditionally Markov model is determined through the collision geometry and destination motion. Finally, the Kalman filter technique is used to estimate and predict the target state. Numerical simulations demonstrate that the proposed approach can improve state estimation accuracy in both static and dynamic destination cases