Data mining for anomaly detection in maritime traffic data

For the past few years, oceans have become once again, an important means of communication and transport. In fact, traffic density throughout the globe has suffered a substantial growth, which has risen some concerns. With this expansion, the need to achieve a high Maritime Situational Awareness (MSA) is imperative. At the present time, this need may be more easily fulfilled thanks to the vast amount of data available regarding maritime traffic. However, this brings in another issue: data overload. Currently, there are so many data sources, so many data to obtain information from, that the operators cannot handle it. There is a pressing need for systems that help to sift through all the data, analysing and correlating, helping in this way the decision making process. In this dissertation, the main goal is to use different sources of data in order to detect anomalies and contribute to a clear Recognised Maritime Picture (RMP). In order to do so, it is necessary to know what types of data exist and which ones are available for further analysis. The data chosen for this dissertation was Automatic Identification System (AIS) and Monitorização Contínua das Atividades da Pesca (MONICAP) data, also known as Vessel Monitoring System (VMS) data. In order to store 1 year worth of AIS and MONICAP data, a PostgreSQL database was created. To analyse and draw conclusions from the data, a data mining tool was used, namely, Orange. Tests were conducted in order to assess the correlation between data sources and find anomalies. The importance of data correlation has never been so important and with this dissertation the aim is to show that there is a simple and effective way to get answers from great amounts of data.Nos últimos anos, os oceanos tornaram-se, mais uma vez, um importante meio de comunicação e transporte. De facto, a densidade de tráfego global sofreu um crescimento substancial, o que levantou algumas preocupações. Com esta expansão, a necessidade de atingir um elevado Conhecimento Situacional Marítimo (CSM) é imperativa. Hoje em dia, esta necessidade pode ser satisfeita mais facilmente graças à vasta quantidade de dados disponíveis de tráfego marítimo. No entanto, isso leva a outra questão: sobrecarga de dados. Atualmente existem tantas fontes de dados, tantos dados dos quais extrair informação, que os operadores não conseguem acompanhar. Existe uma necessidade premente para sistemas que ajudem a escrutinar todos os dados, analisando e correlacionando, contribuindo desta maneira ao processo de tomada de decisão. Nesta dissertação, o principal objetivo é usar diferentes fontes de dados para detetar anomalias e contribuir para uma clara Recognised Maritime Picture (RMP). Para tal, é necessário saber que tipos de dados existem e quais é que se encontram disponíveis para análise posterior. Os dados escolhidos para esta dissertação foram dados Automatic Identification System (AIS) e dados de Monitorização Contínua das Atividades da Pesca (MONICAP), também conhecidos como dados de Vessel Monitoring System (VMS). De forma a armazenar dados correspondentes a um ano de AIS e MONICAP, foi criada uma base de dados em PostgreSQL. Para analisar e retirar conclusões, foi utilizada uma ferramenta de data mining, nomeadamente, o Orange. De modo a que pudesse ser avaliada a correlação entre fontes de dados e serem detetadas anomalias foram realizados vários testes. A correlação de dados nunca foi tão importante e pretende-se com esta dissertação mostrar que existe uma forma simples e eficaz de obter respostas de grandes quantidades de dado

Dynamic decision making for situational awareness using drones: Requirements, identification and comparison of decision support methods

Decision makers increasingly operate in real-time information-rich environments, where limited time is available for interpreting data to inform decisions. These environments are driven by static or mobile sensing devices that can provide numerous dynamic data points. A prominent approach in this space is to utilise drones, which can be deployed to gather targeted information. However, deciding how best to deploy available drones is nontrivial, and stands to benefit from decision support aids that plan routes. Such a system must operate under time constraints created by the changing attributes of routes as the situation unfolds. This study describes a dynamic decision support system (DSS) for situational awareness with drones. The system applies Multi-Criteria Decision Making (MCDM) methods within a dynamic genetic algorithm to provide a continuously revised ranking of routes. Five desiderata for dynamic decision support are presented. It is shown how a dynamic DSS can be equipped with declarative specification of preferences (Desiderata 1), dynamic revision of recommendations (Desiderata 2), and high diversity of options (Desiderata 3). The study then compares four MCDM methods, namely the Weighted Product Model (WPM), the Analytic Hierarchy Process (AHP), the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), and the Preference Ranking Organization METHod for Enrichment Evaluation (PROMETHEE), with regards to how consistently they trade-off between criteria (Desiderata 4) and the stability of results under small changes to criteria values (Desiderata 5). To evaluate the trade-offs between criteria we analyse the smoothness of change in criteria outcomes as criteria weightings increase for each algorithm. The outcomes are calculated by automating the selection of routes in a case study that applies drones to the task of harbour management. The stability of results for the different MCDM methods are compared. Perturbations were applied to sets of routes ranked by each algorithm then each algorithm was reapplied and the magnitude of the changes in ranking was assessed. Overall, TOPSIS was found to be the algorithm which made the most consistent trade-offs between criteria, only under-performing another algorithm with respect to a single criterion. AHP and WPM were the next most consistent algorithms and PROMETHEE was the least consistent algorithm. TOPSIS was also found to be the most stable method under small changes to criteria values. AHP was the second most stable, followed by PROMETHEE and WPM respectively. The results show that TOPSIS achieves the best result for both Desiderata 4 and 5 and consequently the study finds TOPSIS to be an appropriate MCDM method for dynamic decision support.<br/

End-to-end anomaly detection in stream data

Nowadays, huge volumes of data are generated with increasing velocity through various systems, applications, and activities. This increases the demand for stream and time series analysis to react to changing conditions in real-time for enhanced efficiency and quality of service delivery as well as upgraded safety and security in private and public sectors. Despite its very rich history, time series anomaly detection is still one of the vital topics in machine learning research and is receiving increasing attention. Identifying hidden patterns and selecting an appropriate model that fits the observed data well and also carries over to unobserved data is not a trivial task. Due to the increasing diversity of data sources and associated stochastic processes, this pivotal data analysis topic is loaded with various challenges like complex latent patterns, concept drift, and overfitting that may mislead the model and cause a high false alarm rate. Handling these challenges leads the advanced anomaly detection methods to develop sophisticated decision logic, which turns them into mysterious and inexplicable black-boxes. Contrary to this trend, end-users expect transparency and verifiability to trust a model and the outcomes it produces. Also, pointing the users to the most anomalous/malicious areas of time series and causal features could save them time, energy, and money. For the mentioned reasons, this thesis is addressing the crucial challenges in an end-to-end pipeline of stream-based anomaly detection through the three essential phases of behavior prediction, inference, and interpretation. The first step is focused on devising a time series model that leads to high average accuracy as well as small error deviation. On this basis, we propose higher-quality anomaly detection and scoring techniques that utilize the related contexts to reclassify the observations and post-pruning the unjustified events. Last but not least, we make the predictive process transparent and verifiable by providing meaningful reasoning behind its generated results based on the understandable concepts by a human. The provided insight can pinpoint the anomalous regions of time series and explain why the current status of a system has been flagged as anomalous. Stream-based anomaly detection research is a principal area of innovation to support our economy, security, and even the safety and health of societies worldwide. We believe our proposed analysis techniques can contribute to building a situational awareness platform and open new perspectives in a variety of domains like cybersecurity, and health

Adversarial AI Testcases for Maritime Autonomous Systems

Contemporary maritime operations such as shipping are a vital component constituting global trade and defence. The evolution towards maritime autonomous systems, often providing significant benefits (e.g., cost, physical safety), requires the utilisation of artificial intelligence (AI) to automate the functions of a conventional crew. However, unsecured AI systems can be plagued with vulnerabilities naturally inherent within complex AI models. The adversarial AI threat, primarily only evaluated in a laboratory environment, increases the likelihood of strategic adversarial exploitation and attacks on mission-critical AI, including maritime autonomous systems. This work evaluates AI threats to maritime autonomous systems in situ. The results show that multiple attacks can be used against real-world maritime autonomous systems with a range of lethality. However, the effects of AI attacks vary in a dynamic and complex environment from that proposed in lower entropy laboratory environments. We propose a set of adversarial test examples and demonstrate their use, specifically in the marine environment. The results of this paper highlight security risks and deliver a set of principles to mitigate threats to AI, throughout the AI lifecycle, in an evolving threat landscape.</jats:p

Incorporation of AIS data-based machine learning into unsupervised route planning for maritime autonomous surface ships

Maritime Autonomous Surface Ships (MASS) are deemed as the future of maritime transport. Although showing attractiveness in terms of the solutions to emerging challenges such as carbon emission and insufficient labor caused by black swan events such as COVID-19, the applications of MASS have revealed problems in practice, among which MASS navigation safety presents a prioritized concern. To ensure safety, rational route planning for MASS is evident as the most critical step to avoiding any relevant collision accidents. This paper aims to develop a holistic framework for the unsupervised route planning of MASS using machine learning methods based on Automatic Identification System (AIS) data, including the coherent steps of new feature measurement, pattern extraction, and route planning algorithms. Historical AIS data from manned ships are trained to extract and generate movement patterns. The route planning for MASS is derived from the movement patterns according to a dynamic optimization method and a feature extraction algorithm. Numerical experiments are constructed on real AIS data to demonstrate the effectiveness of the proposed method in solving the route planning for different types of MASS

Reliable Navigational Scene Perception for Autonomous Ships in Maritime Environment

Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships

Multi-aspect rule-based AI: Methods, taxonomy, challenges and directions towards automation, intelligence and transparent cybersecurity modeling for critical infrastructures

Critical infrastructure (CI) typically refers to the essential physical and virtual systems, assets, and services that are vital for the functioning and well-being of a society, economy, or nation. However, the rapid proliferation and dynamism of today\u27s cyber threats in digital environments may disrupt CI functionalities, which would have a debilitating impact on public safety, economic stability, and national security. This has led to much interest in effective cybersecurity solutions regarding automation and intelligent decision-making, where AI-based modeling is potentially significant. In this paper, we take into account “Rule-based AI” rather than other black-box solutions since model transparency, i.e., human interpretation, explainability, and trustworthiness in decision-making, is an essential factor, particularly in cybersecurity application areas. This article provides an in-depth study on multi-aspect rule based AI modeling considering human interpretable decisions as well as security automation and intelligence for CI. We also provide a taxonomy of rule generation methods by taking into account not only knowledge-driven approaches based on human expertise but also data-driven approaches, i.e., extracting insights or useful knowledge from data, and their hybridization. This understanding can help security analysts and professionals comprehend how systems work, identify potential threats and anomalies, and make better decisions in various real-world application areas. We also cover how these techniques can address diverse cybersecurity concerns such as threat detection, mitigation, prediction, diagnosis for root cause findings, and so on in different CI sectors, such as energy, defence, transport, health, water, agriculture, etc. We conclude this paper with a list of identified issues and opportunities for future research, as well as their potential solution directions for how researchers and professionals might tackle future generation cybersecurity modeling in this emerging area of study