Resilient Autonomous Control of Distributed Multi-agent Systems in Contested Environments

An autonomous and resilient controller is proposed for leader-follower multi-agent systems under uncertainties and cyber-physical attacks. The leader is assumed non-autonomous with a nonzero control input, which allows changing the team behavior or mission in response to environmental changes. A resilient learning-based control protocol is presented to find optimal solutions to the synchronization problem in the presence of attacks and system dynamic uncertainties. An observer-based distributed H_infinity controller is first designed to prevent propagating the effects of attacks on sensors and actuators throughout the network, as well as to attenuate the effect of these attacks on the compromised agent itself. Non-homogeneous game algebraic Riccati equations are derived to solve the H_infinity optimal synchronization problem and off-policy reinforcement learning is utilized to learn their solution without requiring any knowledge of the agent's dynamics. A trust-confidence based distributed control protocol is then proposed to mitigate attacks that hijack the entire node and attacks on communication links. A confidence value is defined for each agent based solely on its local evidence. The proposed resilient reinforcement learning algorithm employs the confidence value of each agent to indicate the trustworthiness of its own information and broadcast it to its neighbors to put weights on the data they receive from it during and after learning. If the confidence value of an agent is low, it employs a trust mechanism to identify compromised agents and remove the data it receives from them from the learning process. Simulation results are provided to show the effectiveness of the proposed approach

Event-triggered robust control for multi-player nonzero-sum games with input constraints and mismatched uncertainties

In this article, an event-triggered robust control (ETRC) method is investigated for multi-player nonzero-sum games of continuous-time input constrained nonlinear systems with mismatched uncertainties. By constructing an auxiliary system and designing an appropriate value function, the robust control problem of input constrained nonlinear systems is transformed into an optimal regulation problem. Then, a critic neural network (NN) is adopted to approximate the value function of each player for solving the event-triggered coupled Hamilton-Jacobi equation and obtaining control laws. Based on a designed event-triggering condition, control laws are updated when events occur only. Thus, both computational burden and communication bandwidth are reduced. We prove that the weight approximation errors of critic NNs and the closed-loop uncertain multi-player system states are all uniformly ultimately bounded thanks to the Lyapunov's direct method. Finally, two examples are provided to demonstrate the effectiveness of the developed ETRC method

Game-Theoretic Frameworks and Strategies for Defense Against Network Jamming and Collocation Attacks

Modern networks are becoming increasingly more complex, heterogeneous, and densely connected. While more diverse services are enabled to an ever-increasing number of users through ubiquitous networking and pervasive computing, several important challenges have emerged. For example, densely connected networks are prone to higher levels of interference, which makes them more vulnerable to jamming attacks. Also, the utilization of software-based protocols to perform routing, load balancing and power management functions in Software-Defined Networks gives rise to more vulnerabilities that could be exploited by malicious users and adversaries. Moreover, the increased reliance on cloud computing services due to a growing demand for communication and computation resources poses formidable security challenges due to the shared nature and virtualization of cloud computing. In this thesis, we study two types of attacks: jamming attacks on wireless networks and side-channel attacks on cloud computing servers. The former attacks disrupt the natural network operation by exploiting the static topology and dynamic channel assignment in wireless networks, while the latter attacks seek to gain access to unauthorized data by co-residing with target virtual machines (VMs) on the same physical node in a cloud server. In both attacks, the adversary faces a static attack surface and achieves her illegitimate goal by exploiting a stationary aspect of the network functionality. Hence, this dissertation proposes and develops counter approaches to both attacks using moving target defense strategies. We study the strategic interactions between the adversary and the network administrator within a game-theoretic framework. First, in the context of jamming attacks, we present and analyze a game-theoretic formulation between the adversary and the network defender. In this problem, the attack surface is the network connectivity (the static topology) as the adversary jams a subset of nodes to increase the level of interference in the network. On the other side, the defender makes judicious adjustments of the transmission footprint of the various nodes, thereby continuously adapting the underlying network topology to reduce the impact of the attack. The defender\u27s strategy is based on playing Nash equilibrium strategies securing a worst-case network utility. Moreover, scalable decomposition-based approaches are developed yielding a scalable defense strategy whose performance closely approaches that of the non-decomposed game for large-scale and dense networks. We study a class of games considering discrete as well as continuous power levels. In the second problem, we consider multi-tenant clouds, where a number of VMs are typically collocated on the same physical machine to optimize performance and power consumption and maximize profit. This increases the risk of a malicious virtual machine performing side-channel attacks and leaking sensitive information from neighboring VMs. The attack surface, in this case, is the static residency of VMs on a set of physical nodes, hence we develop a timed migration defense approach. Specifically, we analyze a timing game in which the cloud provider decides when to migrate a VM to a different physical machine to mitigate the risk of being compromised by a collocated malicious VM. The adversary decides the rate at which she launches new VMs to collocate with the victim VMs. Our formulation captures a data leakage model in which the cost incurred by the cloud provider depends on the duration of collocation with malicious VMs. It also captures costs incurred by the adversary in launching new VMs and by the defender in migrating VMs. We establish sufficient conditions for the existence of Nash equilibria for general cost functions, as well as for specific instantiations, and characterize the best response for both players. Furthermore, we extend our model to characterize its impact on the attacker\u27s payoff when the cloud utilizes intrusion detection systems that detect side-channel attacks. Our theoretical findings are corroborated with extensive numerical results in various settings as well as a proof-of-concept implementation in a realistic cloud setting

Game theoretic control of multi-agent systems: from centralised to distributed control

Differential game theory provides a framework to study the dynamic strategic interactions between multiple decisors, or players, each with an individual criterion to optimise. Noting the analogy between the concepts of "players'' and "agents'', it seems apparent that this framework is well-suited for control of multi-agent systems (MAS). Most of the existing results in the field of differential games assume that players have access to the full state of the system. This assumption, while holding reasonable in certain scenarios, does not apply in contexts where decisions are to be made by each individual agent based only on available local information. This poses a significant challenge in terms of the control design: distributed control laws, which take into account what information is available, are required. In the present work concepts borrowed from differential game theory and graph theory are exploited to formulate systematic frameworks for control of MAS, in a quest to shift the paradigm from centralised to distributed control. We introduce some preliminaries on differential game theory and graph theory, the latter for modeling communication constraints between the agents. Motivated by the difficulties associated with obtaining exact Nash equilibrium solutions for nonzero-sum differential games, we consider three approximate Nash equilibrium concepts and provide different characterisations of these in terms a class of static optimisation problems often encountered in control theory. Considering the multi-agent collision avoidance problem, we present a game theoretic approach, based on a (centralised) hybrid controller implementation of the control strategies, capable of ensuring collision-free trajectories and global convergence of the error system. We make a first step towards distributed control by introducing differential games with partial information, a framework for distributed control of MAS subject to local communication constraints, in which we assume that the agents share their control strategies with their neighbours. This assumption which, in the case of non-acyclic communication graphs, translates into the requirement of shared reasoning between groups of agents, is then relaxed through the introduction of a framework based on the concept of distributed differential games, i.e. a collection of multiple (fictitious) local differential games played by each individual agent in the MAS. Finally, we revisit the multi-agent collision avoidance problem in a distributed setting: considering time-varying communication graph topologies, which enable to model proximity-based communication constraints, we design differential games characterised by a Nash equilibrium solution which yields collision-free trajectories guaranteeing that all the agents reach their goal, provided no deadlocks occur. The efficacy of the game theoretic frameworks introduced in this thesis is demonstrated on several case studies of practical importance, related to robotic coordination and control of microgrids.Open Acces

Compositional Set Invariance in Network Systems with Assume-Guarantee Contracts

This paper presents an assume-guarantee reasoning approach to the computation of robust invariant sets for network systems. Parameterized signal temporal logic (pSTL) is used to formally describe the behaviors of the subsystems, which we use as the template for the contract. We show that set invariance can be proved with a valid assume-guarantee contract by reasoning about individual subsystems. If a valid assume-guarantee contract with monotonic pSTL template is known, it can be further refined by value iteration. When such a contract is not known, an epigraph method is proposed to solve for a contract that is valid, ---an approach that has linear complexity for a sparse network. A microgrid example is used to demonstrate the proposed method. The simulation result shows that together with control barrier functions, the states of all the subsystems can be bounded inside the individual robust invariant sets.Comment: Submitted to 2019 American Control Conferenc