236 research outputs found
Camouflage Adversarial Attacks on Multiple Agent Systems
The multi-agent reinforcement learning systems (MARL) based on the Markov
decision process (MDP) have emerged in many critical applications. To improve
the robustness/defense of MARL systems against adversarial attacks, the study
of various adversarial attacks on reinforcement learning systems is very
important. Previous works on adversarial attacks considered some possible
features to attack in MDP, such as the action poisoning attacks, the reward
poisoning attacks, and the state perception attacks. In this paper, we propose
a brand-new form of attack called the camouflage attack in the MARL systems. In
the camouflage attack, the attackers change the appearances of some objects
without changing the actual objects themselves; and the camouflaged appearances
may look the same to all the targeted recipient (victim) agents. The
camouflaged appearances can mislead the recipient agents to misguided actions.
We design algorithms that give the optimal camouflage attacks minimizing the
rewards of recipient agents. Our numerical and theoretical results show that
camouflage attacks can rival the more conventional, but likely more difficult
state perception attacks. We also investigate cost-constrained camouflage
attacks and showed numerically how cost budgets affect the attack performance.Comment: arXiv admin note: text overlap with arXiv:2311.0085
Digital Deception: Generative Artificial Intelligence in Social Engineering and Phishing
The advancement of Artificial Intelligence (AI) and Machine Learning (ML) has
profound implications for both the utility and security of our digital
interactions. This paper investigates the transformative role of Generative AI
in Social Engineering (SE) attacks. We conduct a systematic review of social
engineering and AI capabilities and use a theory of social engineering to
identify three pillars where Generative AI amplifies the impact of SE attacks:
Realistic Content Creation, Advanced Targeting and Personalization, and
Automated Attack Infrastructure. We integrate these elements into a conceptual
model designed to investigate the complex nature of AI-driven SE attacks - the
Generative AI Social Engineering Framework. We further explore human
implications and potential countermeasures to mitigate these risks. Our study
aims to foster a deeper understanding of the risks, human implications, and
countermeasures associated with this emerging paradigm, thereby contributing to
a more secure and trustworthy human-computer interaction.Comment: Submitted to CHI 202
Reward Delay Attacks on Deep Reinforcement Learning
Most reinforcement learning algorithms implicitly assume strong synchrony. We
present novel attacks targeting Q-learning that exploit a vulnerability
entailed by this assumption by delaying the reward signal for a limited time
period. We consider two types of attack goals: targeted attacks, which aim to
cause a target policy to be learned, and untargeted attacks, which simply aim
to induce a policy with a low reward. We evaluate the efficacy of the proposed
attacks through a series of experiments. Our first observation is that
reward-delay attacks are extremely effective when the goal is simply to
minimize reward. Indeed, we find that even naive baseline reward-delay attacks
are also highly successful in minimizing the reward. Targeted attacks, on the
other hand, are more challenging, although we nevertheless demonstrate that the
proposed approaches remain highly effective at achieving the attacker's
targets. In addition, we introduce a second threat model that captures a
minimal mitigation that ensures that rewards cannot be used out of sequence. We
find that this mitigation remains insufficient to ensure robustness to attacks
that delay, but preserve the order, of rewards.Comment: 20 pages, 9 figures, Conference on Decision and Game Theory for
Securit
Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments
We study black-box reward poisoning attacks against reinforcement learning (RL), in which an adversary aims to manipulate the rewards to mislead a sequence of RL agents with unknown algorithms to learn a nefarious policy in an environment unknown to the adversary a priori. That is, our attack makes minimum assumptions on the prior knowledge of the adversary: it has no initial knowledge of the environment or the learner, and neither does it observe the learner's internal mechanism except for its performed actions. We design a novel black-box attack, U2, that can provably achieve a near-matching performance to the state-of-the-art white-box attack, demonstrating the feasibility of reward poisoning even in the most challenging black-box setting
- …