2,565 research outputs found
CopyCAT: Taking Control of Neural Policies with Constant Attacks
We propose a new perspective on adversarial attacks against deep
reinforcement learning agents. Our main contribution is CopyCAT, a targeted
attack able to consistently lure an agent into following an outsider's policy.
It is pre-computed, therefore fast inferred, and could thus be usable in a
real-time scenario. We show its effectiveness on Atari 2600 games in the novel
read-only setting. In this setting, the adversary cannot directly modify the
agent's state -- its representation of the environment -- but can only attack
the agent's observation -- its perception of the environment. Directly
modifying the agent's state would require a write-access to the agent's inner
workings and we argue that this assumption is too strong in realistic settings.Comment: AAMAS 202
MIR2: Towards Provably Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization
Robust multi-agent reinforcement learning (MARL) necessitates resilience to
uncertain or worst-case actions by unknown allies. Existing max-min
optimization techniques in robust MARL seek to enhance resilience by training
agents against worst-case adversaries, but this becomes intractable as the
number of agents grows, leading to exponentially increasing worst-case
scenarios. Attempts to simplify this complexity often yield overly pessimistic
policies, inadequate robustness across scenarios and high computational
demands. Unlike these approaches, humans naturally learn adaptive and resilient
behaviors without the necessity of preparing for every conceivable worst-case
scenario. Motivated by this, we propose MIR2, which trains policy in routine
scenarios and minimize Mutual Information as Robust Regularization.
Theoretically, we frame robustness as an inference problem and prove that
minimizing mutual information between histories and actions implicitly
maximizes a lower bound on robustness under certain assumptions. Further
analysis reveals that our proposed approach prevents agents from overreacting
to others through an information bottleneck and aligns the policy with a robust
action prior. Empirically, our MIR2 displays even greater resilience against
worst-case adversaries than max-min optimization in StarCraft II, Multi-agent
Mujoco and rendezvous. Our superiority is consistent when deployed in
challenging real-world robot swarm control scenario. See code and demo videos
in Supplementary Materials
- …