A Multi-Agent Systems Approach for Analysis of Stepping Stone Attacks

Stepping stone attacks are one of the most sophisticated cyber-attacks, in which attackers make a chain of compromised hosts to reach a victim target. In this Dissertation, an analytic model with Multi-Agent systems approach has been proposed to analyze the propagation of stepping stones attacks in dynamic vulnerability graphs. Because the vulnerability configuration in a network is inherently dynamic, in this Dissertation a biased min-consensus technique for dynamic graphs with fixed and switching topology is proposed as a distributed technique to calculate the most vulnerable path for stepping stones attacks in dynamic vulnerability graphs. We use min-plus algebra to analyze and provide necessary and sufficient convergence conditions to the shortest path in the fixed topology case. A necessary condition for the switching topology case is provided. Most cyber-attacks involve an attacker launching a multi-stage attack by exploiting a sequence of hosts. This multi-stage attack generates a chain of ``stepping stones” from the origin to target. The choice of stepping stones is a function of the degree of exploitability, the impact, attacker’s capability, masking origin location, and intent. In this Dissertation, we model and analyze scenarios wherein an attacker employs multiple strategies to choose stepping stones. The problem is modeled as an Adjacency Quadratic Shortest Path using dynamic vulnerability graphs with multi-agent dynamic system approach. With this approach, the shortest stepping stone path with maximum node degree and the shortest stepping stone path with maximum impact are modeled and analyzed. Because embedded controllers are omnipresent in networks, in this Dissertation as a Risk Mitigation Strategy, a cyber-attack tolerant control strategy for embedded controllers is proposed. A dual redundant control architecture that combines two identical controllers that are switched periodically between active and restart modes is proposed. The strategy is addressed to mitigate the impact due to the corruption of the controller software by an adversary. We analyze the impact of the resetting and restarting the controller software and performance of the switching process. The minimum requirements in the control design, for effective mitigation of cyber-attacks to the control software that implies a “fast” switching period is provided. The simulation results demonstrate the effectiveness of the proposed strategy when the time to fully reset and restart the controller is faster than the time taken by an adversary to compromise the controller. The results also provide insights into the stability and safety regions and the factors that determine the effectiveness of the proposed strategy

A tutorial on recursive models for analyzing and predicting path choice behavior

The problem at the heart of this tutorial consists in modeling the path choice behavior of network users. This problem has been extensively studied in transportation science, where it is known as the route choice problem. In this literature, individuals' choice of paths are typically predicted using discrete choice models. This article is a tutorial on a specific category of discrete choice models called recursive, and it makes three main contributions: First, for the purpose of assisting future research on route choice, we provide a comprehensive background on the problem, linking it to different fields including inverse optimization and inverse reinforcement learning. Second, we formally introduce the problem and the recursive modeling idea along with an overview of existing models, their properties and applications. Third, we extensively analyze illustrative examples from different angles so that a novice reader can gain intuition on the problem and the advantages provided by recursive models in comparison to path-based ones

Intelligent Cooperative Control Architecture: A Framework for Performance Improvement Using Safe Learning

Planning for multi-agent systems such as task assignment for teams of limited-fuel unmanned aerial vehicles (UAVs) is challenging due to uncertainties in the assumed models and the very large size of the planning space. Researchers have developed fast cooperative planners based on simple models (e.g., linear and deterministic dynamics), yet inaccuracies in assumed models will impact the resulting performance. Learning techniques are capable of adapting the model and providing better policies asymptotically compared to cooperative planners, yet they often violate the safety conditions of the system due to their exploratory nature. Moreover they frequently require an impractically large number of interactions to perform well. This paper introduces the intelligent Cooperative Control Architecture (iCCA) as a framework for combining cooperative planners and reinforcement learning techniques. iCCA improves the policy of the cooperative planner, while reduces the risk and sample complexity of the learner. Empirical results in gridworld and task assignment for fuel-limited UAV domains with problem sizes up to 9 billion state-action pairs verify the advantage of iCCA over pure learning and planning strategies

Interactive Planning and Sensing for Aircraft in Uncertain Environments with Spatiotemporally Evolving Threats

Autonomous aerial, terrestrial, and marine vehicles provide a platform for several applications including cargo transport, information gathering, surveillance, reconnaissance, and search-and-rescue. To enable such applications, two main technical problems are commonly addressed.On the one hand, the motion-planning problem addresses optimal motion to a destination: an application example is the delivery of a package in the shortest time with least fuel. Solutions to this problem often assume that all relevant information about the environment is available, possibly with some uncertainty. On the other hand, the information gathering problem addresses the maximization of some metric of information about the environment: application examples include such as surveillance and environmental monitoring. Solutions to the motion-planning problem in vehicular autonomy assume that information about the environment is available from three sources: (1) the vehicle’s own onboard sensors, (2) stationary sensor installations (e.g. ground radar stations), and (3) other information gathering vehicles, i.e., mobile sensors, especially with the recent emphasis on collaborative teams of autonomous vehicles with heterogeneous capabilities. Each source typically processes the raw sensor data via estimation algorithms. These estimates are then available to a decision making system such as a motion- planning algorithm. The motion-planner may use some or all of the estimates provided. There is an underlying assumption of “separation� between the motion-planning algorithm and the information about environment. This separation is common in linear feedback control systems, where estimation algorithms are designed independent of control laws, and control laws are designed with the assumption that the estimated state is the true state. In the case of motion-planning, there is no reason to believe that such a separation between the motion-planning algorithm and the sources of estimated environment information will lead to optimal motion plans, even if the motion planner and the estimators are themselves optimal. The goal of this dissertation is to investigate whether the removal of this separation, via interactive motion-planning and sensing, can significantly improve the optimality of motion- planning. The major contribution of this work is interactive planning and sensing. We consider the problem of planning the path of a vehicle, which we refer to as the actor, to traverse a threat field with minimum threat exposure. The threat field is an unknown, time- variant, and strictly positive scalar field defined on a compact 2D spatial domain – the actor’s workspace. The threat field is estimated by a network of mobile sensors that can measure the threat field pointwise. All measurements are noisy. The objective is to determine a path for the actor to reach a desired goal with minimum risk, which is a measure sensitive not only to the threat exposure itself, but also to the uncertainty therein. A novelty of this problem setup is that the actor can communicate with the sensor network and request that the sensors position themselves in a procedure we call sensor reconfiguration such that the actor’s risk is minimized. This work continues with a foundation in motion planning in time-varying fields where waiting is a control input. Waiting is examined in the context of finding an optimal path with considerations for the cost of exposure to a threat field, the cost of movement, and the cost of waiting. For example, an application where waiting may be beneficial in motion-planning is the delivery of a package where adverse weather may pose a risk to the safety of a UAV and its cargo. In such scenarios, an optimal plan may include “waiting until the storm passes.� Results on computational efficiency and optimality of considering waiting in path- planning algorithms are presented. In addition, the relationship of waiting in a time- varying field represented with varying levels of resolution, or multiresolution is studied. Interactive planning and sensing is further developed for the case of time-varying environments. This proposed extension allows for the evaluation of different mission windows, finite sensor network reconfiguration durations, finite planning durations, and varying number of available sensors. Finally, the proposed method considers the effect of waiting in the path planner under the interactive planning and sensing for time-varying fields framework. Future work considers various extensions of the proposed interactive planning and sensing framework including: generalizing the environment using Gaussian processes, sensor reconfiguration costs, multiresolution implementations, nonlinear parameters, decentralized sensor networks and an application to aerial payload delivery by parafoil