Fusing uncertain knowledge and evidence for maritime situational awareness via Markov Logic Networks

The concepts of event and anomaly are important building blocks for developing a situational picture of the observed environment. We here relate these concepts to the JDL fusion model and demonstrate the power of Markov Logic Networks (MLNs) for encoding uncertain knowledge and compute inferences according to observed evidence. MLNs combine the expressive power of first-order logic and the probabilistic uncertainty management of Markov networks. Within this framework, different types of knowledge (e.g. a priori, contextual) with associated uncertainty can be fused together for situation assessment by expressing unobservable complex events as a logical combination of simpler evidences. We also develop a mechanism to evaluate the level of completion of complex events and show how, along with event probability, it could provide additional useful information to the operator. Examples are demonstrated on two maritime scenarios of rules for event and anomaly detection

Overview of contextual tracking approaches in information fusion

Proceedings of: Geospatial InfoFusion III. 2-3 May 2013 Baltimore, Maryland, United States.Many information fusion solutions work well in the intended scenarios; but the applications, supporting data, and capabilities change over varying contexts. One example is weather data for electro-optical target trackers of which standards have evolved over decades. The operating conditions of: technology changes, sensor/target variations, and the contextual environment can inhibit performance if not included in the initial systems design. In this paper, we seek to define and categorize different types of contextual information. We describe five contextual information categories that support target tracking: (1) domain knowledge from a user to aid the information fusion process through selection, cueing, and analysis, (2) environment-to-hardware processing for sensor management, (3) known distribution of entities for situation/threat assessment, (4) historical traffic behavior for situation awareness patterns of life (POL), and (5) road information for target tracking and identification. Appropriate characterization and representation of contextual information is needed for future high-level information fusion systems design to take advantage of the large data content available for a priori knowledge target tracking algorithm construction, implementation, and application.Publicad

Context Exploitation in Data Fusion

Complex and dynamic environments constitute a challenge for existing tracking algorithms. For this reason, modern solutions are trying to utilize any available information which could help to constrain, improve or explain the measurements. So called Context Information (CI) is understood as information that surrounds an element of interest, whose knowledge may help understanding the (estimated) situation and also in reacting to that situation. However, context discovery and exploitation are still largely unexplored research topics. Until now, the context has been extensively exploited as a parameter in system and measurement models which led to the development of numerous approaches for the linear or non-linear constrained estimation and target tracking. More specifically, the spatial or static context is the most common source of the ambient information, i.e. features, utilized for recursive enhancement of the state variables either in the prediction or the measurement update of the filters. In the case of multiple model estimators, context can not only be related to the state but also to a certain mode of the filter. Common practice for multiple model scenarios is to represent states and context as a joint distribution of Gaussian mixtures. These approaches are commonly referred as the join tracking and classification. Alternatively, the usefulness of context was also demonstrated in aiding the measurement data association. Process of formulating a hypothesis, which assigns a particular measurement to the track, is traditionally governed by the empirical knowledge of the noise characteristics of sensors and operating environment, i.e. probability of detection, false alarm, clutter noise, which can be further enhanced by conditioning on context. We believe that interactions between the environment and the object could be classified into actions, activities and intents, and formed into structured graphs with contextual links translated into arcs. By learning the environment model we will be able to make prediction on the target\u2019s future actions based on its past observation. Probability of target future action could be utilized in the fusion process to adjust tracker confidence on measurements. By incorporating contextual knowledge of the environment, in the form of a likelihood function, in the filter measurement update step, we have been able to reduce uncertainties of the tracking solution and improve the consistency of the track. The promising results demonstrate that the fusion of CI brings a significant performance improvement in comparison to the regular tracking approaches

Context-based Information Fusion: A survey and discussion

This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of \u201ccontext\u201d. It shows how its fortune in the distributed computing world eventually permeated in the world of IF, discussing the current strategies and techniques, and hinting possible future trends. IF processes can represent context at different levels (structural and physical constraints of the scenario, a priori known operational rules between entities and environment, dynamic relationships modelled to interpret the system output, etc.). In addition to the survey, several novel context exploitation dynamics and architectural aspects peculiar to the fusion domain are presented and discussed

Software agents & human behavior

People make important decisions in emergencies. Often these decisions involve high stakes in terms of lives and property. Bhopal disaster (1984), Piper Alpha disaster (1988), Montara blowout (2009), and explosion on Deepwater Horizon (2010) are a few examples among many industrial incidents. In these incidents, those who were in-charge took critical decisions under various ental stressors such as time, fatigue, and panic. This thesis presents an application of naturalistic decision-making (NDM), which is a recent decision-making theory inspired by experts making decisions in real emergencies. This study develops an intelligent agent model that can be programed to make human-like decisions in emergencies. The agent model has three major components: (1) A spatial learning module, which the agent uses to learn escape routes that are designated routes in a facility for emergency evacuation, (2) a situation recognition module, which is used to recognize or distinguish among evolving emergency situations, and (3) a decision-support module, which exploits modules in (1) and (2), and implements an NDM based decision-logic for producing human-like decisions in emergencies. The spatial learning module comprises a generalized stochastic Petri net-based model of spatial learning. The model classifies routes into five classes based on landmarks, which are objects with salient spatial features. These classes deal with the question of how difficult a landmark turns out to be when an agent observes it the first time during a route traversal. An extension to the spatial learning model is also proposed where the question of how successive route traversals may impact retention of a route in the agent’s memory is investigated. The situation awareness module uses Markov logic network (MLN) to define different offshore emergency situations using First-order Logic (FOL) rules. The purpose of this module is to give the agent the necessary experience of dealing with emergencies. The potential of this module lies in the fact that different training samples can be used to produce agents having different experience or capability to deal with an emergency situation. To demonstrate this fact, two agents were developed and trained using two different sets of empirical observations. The two are found to be different in recognizing the prepare-to-abandon-platform alarm (PAPA ), and similar to each other in recognition of an emergency using other cues. Finally, the decision-support module is proposed as a union of spatial-learning module, situation awareness module, and NDM based decision-logic. The NDM-based decision-logic is inspired by Klein’s (1998) recognition primed decision-making (RPDM) model. The agent’s attitudes related to decision-making as per the RPDM are represented in the form of belief, desire, and intention (BDI). The decision-logic involves recognition of situations based on experience (as proposed in situation-recognition module), and recognition of situations based on classification, where ontological classification is used to guide the agent in cases where the agent’s experience about confronting a situation is inadequate. At the planning stage, the decision-logic exploits the agent’s spatial knowledge (as proposed in spatial-learning module) about the layout of the environment to make adjustments in the course of actions relevant to a decision that has already been made as a by-product of situation recognition. The proposed agent model has potential to be used to improve virtual training environment’s fidelity by adding agents that exhibit human-like intelligence in performing tasks related to emergency evacuation. Notwithstanding, the potential to exploit the basis provided here, in the form of an agent representing human fallibility, should not be ignored for fields like human reliability analysis

Movement Analytics: Current Status, Application to Manufacturing, and Future Prospects from an AI Perspective

Data-driven decision making is becoming an integral part of manufacturing companies. Data is collected and commonly used to improve efficiency and produce high quality items for the customers. IoT-based and other forms of object tracking are an emerging tool for collecting movement data of objects/entities (e.g. human workers, moving vehicles, trolleys etc.) over space and time. Movement data can provide valuable insights like process bottlenecks, resource utilization, effective working time etc. that can be used for decision making and improving efficiency. Turning movement data into valuable information for industrial management and decision making requires analysis methods. We refer to this process as movement analytics. The purpose of this document is to review the current state of work for movement analytics both in manufacturing and more broadly. We survey relevant work from both a theoretical perspective and an application perspective. From the theoretical perspective, we put an emphasis on useful methods from two research areas: machine learning, and logic-based knowledge representation. We also review their combinations in view of movement analytics, and we discuss promising areas for future development and application. Furthermore, we touch on constraint optimization. From an application perspective, we review applications of these methods to movement analytics in a general sense and across various industries. We also describe currently available commercial off-the-shelf products for tracking in manufacturing, and we overview main concepts of digital twins and their applications

Why We Cannot (Yet) Ensure the Cybersecurity of Safety-Critical Systems

There is a growing threat to the cyber-security of safety-critical systems. The introduction of Commercial Off The Shelf (COTS) software, including Linux, specialist VOIP applications and Satellite Based Augmentation Systems across the aviation, maritime, rail and power-generation infrastructures has created common, vulnerabilities. In consequence, more people now possess the technical skills required to identify and exploit vulnerabilities in safety-critical systems. Arguably for the first time there is the potential for cross-modal attacks leading to future ‘cyber storms’. This situation is compounded by the failure of public-private partnerships to establish the cyber-security of safety critical applications. The fiscal crisis has prevented governments from attracting and retaining competent regulators at the intersection of safety and cyber-security. In particular, we argue that superficial similarities between safety and security have led to security policies that cannot be implemented in safety-critical systems. Existing office-based security standards, such as the ISO27k series, cannot easily be integrated with standards such as IEC61508 or ISO26262. Hybrid standards such as IEC 62443 lack credible validation. There is an urgent need to move beyond high-level policies and address the more detailed engineering challenges that threaten the cyber-security of safety-critical systems. In particular, we consider the ways in which cyber-security concerns undermine traditional forms of safety engineering, for example by invalidating conventional forms of risk assessment. We also summarise the ways in which safety concerns frustrate the deployment of conventional mechanisms for cyber-security, including intrusion detection systems

Enhancing maritime defence and security through persistently autonomous operations and situation awareness systems

This thesis is concerned with autonomous operations with Autonomous Underwater Vehicles(AUVs) and maritime situation awareness in the context of enhancing maritime defence and security. The problem of autonomous operations with AUVs is one of persistence. That is, AUVs get stuck due to a lack of cognitive ability to deal with a situation and require intervention from a human operator. This thesis focuses on addressing vehicle subsystem failures and changes in high level mission priorities in a manner that preserves autonomy during Mine Counter measures (MCM) operations in unknown environments. This is not a trivial task. The approach followed utilizes ontologies for representing knowledge about the operational environment, the vehicle as well as mission planning and execution. Reasoning about the vehicle capabilities and consequently the actions it can execute is continuous and occurs in real time. Vehicle component faults are incorporated into the reasoning process as a means of driving adaptive planning and execution. Adaptive planning is based on a Planning Domain Definition Language (PDDL) planner. Adaptive execution is prioritized over adaptive planning as mission planning can be very demanding in terms of computational resources. Changes in high level mission priorities are also addressed as part of the adaptive planning behaviour of the system. The main contribution of this thesis regarding persistently autonomous operations is an ontological framework that drives an adaptive behaviour for increasing persistent autonomy of AUVs in unexpected situations. That is, when vehicle component faults threaten to put the mission at risk and changes in high level mission priorities should be incorporated as part of decision making. Building maritime situation awareness for maritime security is a very difficult task. High volumes of information gathered from various sources as well as their efficient fusion taking into consideration any contradictions and the requirement for reliable decision making and (re)action under potentially multiple interpretations of a situation are the most prominent challenges. To address those challenges and help alleviate the burden from humans which usually undertake such tasks, this thesis is concerned with maritime situation awareness built with Markov Logic Networks(MLNs) that support humans in their decision making. However, commonly maritime situation awareness systems rely on human experts to transfer their knowledge into the system before it can be deployed. In that respect, a promising alternative for training MLNs with data is presented. In addition, an in depth evaluation of their performance is provided during which the significance of interpreting an unfolding situation in context is demonstrated. To the best of the author’s knowledge, it is the first time that MLNs are trained with data and evaluated using cross validation in the context of building maritime situation awareness for maritime security

A Bayesian network to manage risks of maritime piracy against offshore oil fields

International audienceIn recent years, pirate attacks against shipping and oil field installations have become more frequent and more serious. This article proposes an innovative solution to the problem of offshore piracy from the perspective of the entire processing chain: from the detection of a potential threat to the implementation of a response. The response to an attack must take into account multiple variables: the characteristics of the threat and the potential target, existing protection tools, environmental constraints, etc. The potential of Bayesian networks is used to manage this large number of parameters and identify appropriate counter-measures