Video trajectory analysis using unsupervised clustering and multi-criteria ranking

Surveillance camera usage has increased significantly for visual surveillance. Manual analysis of large video data recorded by cameras may not be feasible on a larger scale. In various applications, deep learning-guided supervised systems are used to track and identify unusual patterns. However, such systems depend on learning which may not be possible. Unsupervised methods relay on suitable features and demand cluster analysis by experts. In this paper, we propose an unsupervised trajectory clustering method referred to as t-Cluster. Our proposed method prepares indexes of object trajectories by fusing high-level interpretable features such as origin, destination, path, and deviation. Next, the clusters are fused using multi-criteria decision making and trajectories are ranked accordingly. The method is able to place abnormal patterns on the top of the list. We have evaluated our algorithm and compared it against competent baseline trajectory clustering methods applied to videos taken from publicly available benchmark datasets. We have obtained higher clustering accuracies on public datasets with significantly lesser computation overhead

Physics Infused LSTM Network for Track Association Based on Marine Vessel Automatic Identification System Data

In marine surveillance, a crucial task is distinguishing between normal and abnormal vessel movements to timely identify potential threats. Subsequently, the vessels need to be monitored and tracked until necessary action can be taken. To achieve this, a track association problem is formulated where multiple vessels\u27 unlabeled geographic and motion parameters are associated with their true labels. These parameters are typically obtained from the Automatic Identification System (AIS) database, which enables real-time tracking of marine vessels equipped with AIS. The parameters are time-stamped and collected over a long period, and therefore, modeling the inherent temporal patterns in the data is crucial for successful track association. The problem is further complicated by infrequent data collection (time gap) and track overlaps. Traditionally, physics-based models and Kalman-filtering algorithms are used for tracking problems. However, the performance of Kalman filtering is limited in the presence of time-gap and overlapping tracks, while physics-based models are unable to model temporal patterns. To address these limitations, this work employs LSTM, a special neural network architecture, for marine vessel track association. LSTM is capable of modeling long-term temporal patterns and associating a data point with its true track. The performance of LSTM is investigated, and its strengths and limitations are identified. To further improve the performance of LSTM, an integration of the physics-based model and LSTM is proposed. The performance of the joint model is evaluated on multiple AIS datasets with varying characteristics. According to the findings, the physics-based model performs better when there is very little or no time gap in the dataset. However, when there are time gaps and multiple overlapping tracks, LSTM outperforms the physics-based model. Additionally, LSTM is more effective with larger datasets as it can learn the historical patterns of the features. Nevertheless, the joint model consistently outperforms the individual models by leveraging the strengths of both approaches. Given that the AIS dataset commonly provides a long stretch of historical information with frequent time gaps, the combined model should improve the accuracy of vessel tracking

A multi-task analysis and modelling paradigm using LSTM for multi-source monitoring data of inland vessels

The vessel monitoring data provide important information for people to understand the vessel dynamic status in real time and make appropriate decisions in vessel management and operations. However, some of the essential data may be incomplete or unavailable. In order to recover or predict the missing information and best exploit the vessels monitoring data, this paper combines statistical analysis, data mining and neural network methods to propose a multi-task analysis and modelling framework for multi-source monitoring data of inland vessels. Specifically, an advanced neural network, Long Short-Term Memory (LSTM) was tailored and employed to tackle three important tasks, including vessel trajectory repair, engine speed modelling and fuel consumption prediction. The developed models have been validated using the real-life vessel monitoring data and shown to outperform some other widely used modelling methods. In addition, statistics and data technologies were employed for data extraction, classification and cleaning, and an algorithm was designed for identification of the vessel navigational state

Adaptively constrained dynamic time warping for time series classification and clustering

Time series classification and clustering are important for data mining research, which is conducive to recognizing movement patterns, finding customary routes, and detecting abnormal trajectories in transport (e.g. road and maritime) traffic. The dynamic time warping (DTW) algorithm is a classical distance measurement method for time series analysis. However, the over-stretching and over-compression problems are typical drawbacks of using DTW to measure distances. To address these drawbacks, an adaptive constrained DTW (ACDTW) algorithm is developed to calculate the distances between trajectories more accurately by introducing new adaptive penalty functions. Two different penalties are proposed to effectively and automatically adapt to the situations in which multiple points in one time series correspond to a single point in another time series. The novel ACDTW algorithm can adaptively adjust the correspondence between two trajectories and obtain greater accuracy between different trajectories. Numerous experiments on classification and clustering are undertaken using the UCR time series archive and real vessel trajectories. The classification results demonstrate that the ACDTW algorithm performs better than four state-of-the-art algorithms on the UCR time series archive. Furthermore, the clustering results reveal that the ACDTW algorithm has the best performance among three existing algorithms in modeling maritime traffic vessel trajectory. © 2020 Elsevier Inc

Towards safe navigation environment: The imminent role of spatio-temporal pattern mining in maritime piracy incidents analysis

Since the new century, we have witnessed the fast evolution of pirate attack modes in terms of locations, time, used weapons, and targeted ships. It reveals that the current understanding of pirate attack spatio-temporal patterns is fading, requiring new technologies of big data analysis to master the hidden rules of piracy-related risk spatio-temporal patterns and rationalize the development of relevant anti-piracy measures and policies. This paper aims to develop a new framework of spatio-temporal pattern mining to realize the visualization and analysis of maritime piracy incidents from different standpoints using a new piracy incident database generated from three datasets. Time-based, space-based, and spatial-temporal pattern mining of piracy incidents are systematically investigated to dissect the influence of different risk factors and mine the characteristics of the incidents. Moreover, a novel Fast Adaptive Dynamic Time Warping (FADTW) method is proposed to uncover the hidden temporal and spatial-temporal patterns of piracy incidents. Furthermore, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is applied to extract the spatial distribution patterns and discover the high-risk areas. Finally, risk factors-based classification exploration has uncovered different spatial patterns. The findings, showing the global and local features of piracy incidents, have made significant contributions to rationalizing anti-pirate measures for safe navigation

Deep bi-directional information-empowered ship trajectory prediction for maritime autonomous surface ships

It is critical to have accurate ship trajectory prediction for collision avoidance and intelligent traffic management of manned ships and emerging Maritime Autonomous Surface Ships (MASS). Deep learning methods for accurate prediction based on AIS data have emerged as a contemporary maritime transportation research focus. However, concerns about its accuracy and computational efficiency widely exist across both academic and industrial sectors, necessitating the discovery of new solutions. This paper aims to develop a new prediction approach called Deep Bi-Directional Information-Empowered (DBDIE) by utilising integrated multiple networks and an attention mechanism to address the above issues. The new DBDIE model extracts valuable features by fusing the Bi-directional Long Short-Term Memory (Bi-LSTM) and the Bi-directional Gated Recurrent Unit (Bi-GRU) neural networks. Additionally, the weights of the two bi-directional units are optimised using an attention mechanism, and the final prediction results are obtained through a weight self-adjustment mechanism. The effectiveness of the proposed model is verified through comprehensive comparisons with state-of-the-art deep learning methods, including Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Bi-LSTM, Bi-GRU, Sequence to Sequence (Seq2Seq), and Transformer neural networks. The experimental results demonstrate that the new DBDIE model achieves the most satisfactory prediction outcomes than all other classical methods, providing a new solution to improving the accuracy and effectiveness of predicting ship trajectories, which becomes increasingly important in the era of the safe navigation of mixed manned ships and MASS. As a result, the findings can aid the development and implementation of proactive preventive measures to avoid collisions, enhance maritime traffic management efficiency, and ensure maritime safety

AIS data-driven ship trajectory prediction modelling and analysis based on machine learning and deep learning methods

Maritime transport faces new safety challenges in an increasingly complex traffic environment caused by large-scale and high-speed ships, particularly with the introduction of intelligent and autonomous ships. It is evident that Automatic Identification System (AIS) data-driven ship trajectory prediction can effectively aid in identifying abnormal ship behaviours and reducing maritime risks such as collision, stranding, and contact. Furthermore, trajectory prediction is widely recognised as one of the critical technologies for realising safe autonomous navigation. The prediction methods and their performance are the key factors for future safe and automatic shipping. Currently, ship trajectory prediction lacks the real performance measurement and analysis of different algorithms, including classical machine learning and emerging deep learning methods. This paper aims to systematically analyse the performance of ship trajectory prediction methods and pioneer experimental tests to reveal their advantages and disadvantages as well as fitness in different scenarios involving complicated systems. To do so, five machine learning methods (i.e., Kalman Filter (KF), Support Vector Progression (SVR), Back Propagation network (BP), Gaussian Process Regression (GPR), and Random Forest (RF)) and seven deep learning methods (i.e., Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gate Recurrent Unit (GRU), Bi-directional Long Short-Term Memory (Bi-LSTM), Sequence to Sequence (Seq2seq), Bi-directional Gate Recurrent Unit (Bi-GRU), and Transformer) are first extracted from the state-of-the-art literature review and then employed to implement the trajectory prediction and compare their prediction performance in the real world. Three AIS datasets are collected from the waters of representative traffic features, including a normal channel (i.e., the Chengshan Jiao Promontory), complex traffic (i.e., the Zhoushan Archipelago), and a port area (i.e., Caofeidian port). They are selected to test and analyse the performance of all twelve methods based on six evaluation indexes and explore the characteristics and effectiveness of the twelve trajectory prediction methods in detail. The experimental results provide a novel perspective, comparison, and benchmark for ship trajectory prediction research, which not only demonstrates the fitness of each method in different maritime traffic scenarios, but also makes significant contributions to maritime safety and autonomous shipping development

Incorporation of AIS data-based machine learning into unsupervised route planning for maritime autonomous surface ships

Maritime Autonomous Surface Ships (MASS) are deemed as the future of maritime transport. Although showing attractiveness in terms of the solutions to emerging challenges such as carbon emission and insufficient labor caused by black swan events such as COVID-19, the applications of MASS have revealed problems in practice, among which MASS navigation safety presents a prioritized concern. To ensure safety, rational route planning for MASS is evident as the most critical step to avoiding any relevant collision accidents. This paper aims to develop a holistic framework for the unsupervised route planning of MASS using machine learning methods based on Automatic Identification System (AIS) data, including the coherent steps of new feature measurement, pattern extraction, and route planning algorithms. Historical AIS data from manned ships are trained to extract and generate movement patterns. The route planning for MASS is derived from the movement patterns according to a dynamic optimization method and a feature extraction algorithm. Numerical experiments are constructed on real AIS data to demonstrate the effectiveness of the proposed method in solving the route planning for different types of MASS

Spatio-Temporal Vessel Trajectory Clustering Based on Data Mapping and Density

Automatic identification systems (AISs) serve as a complement to radar systems, and they have been installed and widely used onboard ships to identify targets and improve navigational safety based on a very high-frequency data communication scheme. AIS networks have also been constructed to enhance traffic safety and improve management in main harbors. AISs record vessel trajectories, which include rich traffic flow information, and they represent the foundation for identifying locations and analyzing motion features. However, the inclusion of redundant information will reduce the accuracy of trajectory clustering; therefore, trajectory data mining has become an important research direction. To extract useful information with high accuracy and low computational costs, trajectory mapping and clustering methods are combined in this paper to explore big data acquired from AISs. In particular, the merge distance (MD) is used to measure the similarities between different trajectories, and multidimensional scaling (MDS) is adopted to construct a suitable low-dimensional spatial expression of the similarities between trajectories. An improved density-based spatial clustering of applications with noise (DBSCAN) algorithm is then proposed to cluster spatial points to acquire the optimal cluster. A fusion of the MD, MDS, and improved DBSCAN algorithms can identify the course of trajectories and attain a better clustering performance. Experiments are conducted using a real AIS trajectory database for a bridge area waterway and the Mississippi River to verify the effectiveness of the proposed method. The experiments also show that the newly proposed method presents a higher accuracy than classical ones, such as spectral clustering and affinity propagation clustering