Resilient Autonomous Control of Distributed Multi-agent Systems in Contested Environments

An autonomous and resilient controller is proposed for leader-follower multi-agent systems under uncertainties and cyber-physical attacks. The leader is assumed non-autonomous with a nonzero control input, which allows changing the team behavior or mission in response to environmental changes. A resilient learning-based control protocol is presented to find optimal solutions to the synchronization problem in the presence of attacks and system dynamic uncertainties. An observer-based distributed H_infinity controller is first designed to prevent propagating the effects of attacks on sensors and actuators throughout the network, as well as to attenuate the effect of these attacks on the compromised agent itself. Non-homogeneous game algebraic Riccati equations are derived to solve the H_infinity optimal synchronization problem and off-policy reinforcement learning is utilized to learn their solution without requiring any knowledge of the agent's dynamics. A trust-confidence based distributed control protocol is then proposed to mitigate attacks that hijack the entire node and attacks on communication links. A confidence value is defined for each agent based solely on its local evidence. The proposed resilient reinforcement learning algorithm employs the confidence value of each agent to indicate the trustworthiness of its own information and broadcast it to its neighbors to put weights on the data they receive from it during and after learning. If the confidence value of an agent is low, it employs a trust mechanism to identify compromised agents and remove the data it receives from them from the learning process. Simulation results are provided to show the effectiveness of the proposed approach

Data-driven Safe Control of Linear Systems Under Epistemic and Aleatory Uncertainties

Safe control of constrained linear systems under both epistemic and aleatory uncertainties is considered. The aleatory uncertainty characterizes random noises and is modeled by a probability distribution function (PDF) and the epistemic uncertainty characterizes the lack of knowledge on the system dynamics. Data-based probabilistic safe controllers are designed for the cases where the noise PDF is 1) zero-mean Gaussian with a known covariance, 2) zero-mean Gaussian with an uncertain covariance, and 3) zero-mean non-Gaussian with an unknown distribution. Easy-to-check model-based conditions for guaranteeing probabilistic safety are provided for the first case by introducing probabilistic contractive sets. These results are then extended to the second and third cases by leveraging distributionally-robust probabilistic safe control and conditional value-at-risk (CVaR) based probabilistic safe control, respectively. Data-based implementations of these probabilistic safe controllers are then considered. It is shown that data-richness requirements for directly learning a safe controller is considerably weaker than data-richness requirements for model-based safe control approaches that undertake a model identification. Moreover, an upper bound on the minimal risk level, under which the existence of a safe controller is guaranteed, is learned using collected data. A simulation example is provided to show the effectiveness of the proposed approach

A Risk-Averse Preview-based QQ-Learning Algorithm: Application to Highway Driving of Autonomous Vehicles

A risk-averse preview-based $Q$-learning planner is presented for navigation of autonomous vehicles. To this end, the multi-lane road ahead of a vehicle is represented by a finite-state non-stationary Markov decision process (MDP). A risk assessment unit module is then presented that leverages the preview information provided by sensors along with a stochastic reachability module to assign reward values to the MDP states and update them as scenarios develop. A sampling-based risk-averse preview-based $Q$-learning algorithm is finally developed that generates samples using the preview information and reward function to learn risk-averse optimal planning strategies without actual interaction with the environment. The risk factor is imposed on the objective function to avoid fluctuation of the $Q$ values, which can jeopardize the vehicle's safety and/or performance. The overall hybrid automaton model of the system is leveraged to develop a feasibility check unit module that detects unfeasible plans and enables the planner system to proactively react to the changes of the environment. Theoretical results are provided to bound the number of samples required to guarantee $\epsilon$-optimal planning with a high probability. Finally, to verify the efficiency of the presented algorithm, its implementation on highway driving of an autonomous vehicle in a varying traffic density is considered