1,275 research outputs found
A First Order Meta Stackelberg Method for Robust Federated Learning (Technical Report)
Recent research efforts indicate that federated learning (FL) systems are
vulnerable to a variety of security breaches. While numerous defense strategies
have been suggested, they are mainly designed to counter specific attack
patterns and lack adaptability, rendering them less effective when facing
uncertain or adaptive threats. This work models adversarial FL as a Bayesian
Stackelberg Markov game (BSMG) between the defender and the attacker to address
the lack of adaptability to uncertain adaptive attacks. We further devise an
effective meta-learning technique to solve for the Stackelberg equilibrium,
leading to a resilient and adaptable defense. The experiment results suggest
that our meta-Stackelberg learning approach excels in combating intense model
poisoning and backdoor attacks of indeterminate types
Adaptive Discounting of Training Time Attacks
Among the most insidious attacks on Reinforcement Learning (RL) solutions are
training-time attacks (TTAs) that create loopholes and backdoors in the learned
behaviour. Not limited to a simple disruption, constructive TTAs (C-TTAs) are
now available, where the attacker forces a specific, target behaviour upon a
training RL agent (victim). However, even state-of-the-art C-TTAs focus on
target behaviours that could be naturally adopted by the victim if not for a
particular feature of the environment dynamics, which C-TTAs exploit. In this
work, we show that a C-TTA is possible even when the target behaviour is
un-adoptable due to both environment dynamics as well as non-optimality with
respect to the victim objective(s). To find efficient attacks in this context,
we develop a specialised flavour of the DDPG algorithm, which we term
gammaDDPG, that learns this stronger version of C-TTA. gammaDDPG dynamically
alters the attack policy planning horizon based on the victim's current
behaviour. This improves effort distribution throughout the attack timeline and
reduces the effect of uncertainty the attacker has about the victim. To
demonstrate the features of our method and better relate the results to prior
research, we borrow a 3D grid domain from a state-of-the-art C-TTA for our
experiments. Code is available at "bit.ly/github-rb-gDDPG".Comment: 19 pages, 7 figure
Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learning
Lately, differential privacy (DP) has been introduced in cooperative
multiagent reinforcement learning (CMARL) to safeguard the agents' privacy
against adversarial inference during knowledge sharing. Nevertheless, we argue
that the noise introduced by DP mechanisms may inadvertently give rise to a
novel poisoning threat, specifically in the context of private knowledge
sharing during CMARL, which remains unexplored in the literature. To address
this shortcoming, we present an adaptive, privacy-exploiting, and
evasion-resilient localized poisoning attack (PeLPA) that capitalizes on the
inherent DP-noise to circumvent anomaly detection systems and hinder the
optimal convergence of the CMARL model. We rigorously evaluate our proposed
PeLPA attack in diverse environments, encompassing both non-adversarial and
multiple-adversarial contexts. Our findings reveal that, in a medium-scale
environment, the PeLPA attack with attacker ratios of 20% and 40% can lead to
an increase in average steps to goal by 50.69% and 64.41%, respectively.
Furthermore, under similar conditions, PeLPA can result in a 1.4x and 1.6x
computational time increase in optimal reward attainment and a 1.18x and 1.38x
slower convergence for attacker ratios of 20% and 40%, respectively.Comment: 6 pages, 4 figures, Published in the proceeding of the ICMLC 2023,
9-11 July 2023, The University of Adelaide, Adelaide, Australi
Data Poisoning Attacks in Contextual Bandits
We study offline data poisoning attacks in contextual bandits, a class of
reinforcement learning problems with important applications in online
recommendation and adaptive medical treatment, among others. We provide a
general attack framework based on convex optimization and show that by slightly
manipulating rewards in the data, an attacker can force the bandit algorithm to
pull a target arm for a target contextual vector. The target arm and target
contextual vector are both chosen by the attacker. That is, the attacker can
hijack the behavior of a contextual bandit. We also investigate the feasibility
and the side effects of such attacks, and identify future directions for
defense. Experiments on both synthetic and real-world data demonstrate the
efficiency of the attack algorithm.Comment: GameSec 201
Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments
We study black-box reward poisoning attacks against reinforcement learning (RL), in which an adversary aims to manipulate the rewards to mislead a sequence of RL agents with unknown algorithms to learn a nefarious policy in an environment unknown to the adversary a priori. That is, our attack makes minimum assumptions on the prior knowledge of the adversary: it has no initial knowledge of the environment or the learner, and neither does it observe the learner's internal mechanism except for its performed actions. We design a novel black-box attack, U2, that can provably achieve a near-matching performance to the state-of-the-art white-box attack, demonstrating the feasibility of reward poisoning even in the most challenging black-box setting
- …