4,538 research outputs found
Efficient Robustness Assessment via Adversarial Spatial-Temporal Focus on Videos
Adversarial robustness assessment for video recognition models has raised
concerns owing to their wide applications on safety-critical tasks. Compared
with images, videos have much high dimension, which brings huge computational
costs when generating adversarial videos. This is especially serious for the
query-based black-box attacks where gradient estimation for the threat models
is usually utilized, and high dimensions will lead to a large number of
queries. To mitigate this issue, we propose to simultaneously eliminate the
temporal and spatial redundancy within the video to achieve an effective and
efficient gradient estimation on the reduced searching space, and thus query
number could decrease. To implement this idea, we design the novel Adversarial
spatial-temporal Focus (AstFocus) attack on videos, which performs attacks on
the simultaneously focused key frames and key regions from the inter-frames and
intra-frames in the video. AstFocus attack is based on the cooperative
Multi-Agent Reinforcement Learning (MARL) framework. One agent is responsible
for selecting key frames, and another agent is responsible for selecting key
regions. These two agents are jointly trained by the common rewards received
from the black-box threat models to perform a cooperative prediction. By
continuously querying, the reduced searching space composed of key frames and
key regions is becoming precise, and the whole query number becomes less than
that on the original video. Extensive experiments on four mainstream video
recognition models and three widely used action recognition datasets
demonstrate that the proposed AstFocus attack outperforms the SOTA methods,
which is prevenient in fooling rate, query number, time, and perturbation
magnitude at the same.Comment: accepted by TPAMI202
Procedural Noise Adversarial Examples for Black-Box Attacks on Deep Convolutional Networks
Deep Convolutional Networks (DCNs) have been shown to be vulnerable to
adversarial examples---perturbed inputs specifically designed to produce
intentional errors in the learning algorithms at test time. Existing
input-agnostic adversarial perturbations exhibit interesting visual patterns
that are currently unexplained. In this paper, we introduce a structured
approach for generating Universal Adversarial Perturbations (UAPs) with
procedural noise functions. Our approach unveils the systemic vulnerability of
popular DCN models like Inception v3 and YOLO v3, with single noise patterns
able to fool a model on up to 90% of the dataset. Procedural noise allows us to
generate a distribution of UAPs with high universal evasion rates using only a
few parameters. Additionally, we propose Bayesian optimization to efficiently
learn procedural noise parameters to construct inexpensive untargeted black-box
attacks. We demonstrate that it can achieve an average of less than 10 queries
per successful attack, a 100-fold improvement on existing methods. We further
motivate the use of input-agnostic defences to increase the stability of models
to adversarial perturbations. The universality of our attacks suggests that DCN
models may be sensitive to aggregations of low-level class-agnostic features.
These findings give insight on the nature of some universal adversarial
perturbations and how they could be generated in other applications.Comment: 16 pages, 10 figures. In Proceedings of the 2019 ACM SIGSAC
Conference on Computer and Communications Security (CCS '19
IMAP: Intrinsically Motivated Adversarial Policy
Reinforcement learning agents are susceptible to evasion attacks during
deployment. In single-agent environments, these attacks can occur through
imperceptible perturbations injected into the inputs of the victim policy
network. In multi-agent environments, an attacker can manipulate an adversarial
opponent to influence the victim policy's observations indirectly. While
adversarial policies offer a promising technique to craft such attacks, current
methods are either sample-inefficient due to poor exploration strategies or
require extra surrogate model training under the black-box assumption. To
address these challenges, in this paper, we propose Intrinsically Motivated
Adversarial Policy (IMAP) for efficient black-box adversarial policy learning
in both single- and multi-agent environments. We formulate four types of
adversarial intrinsic regularizers -- maximizing the adversarial state
coverage, policy coverage, risk, or divergence -- to discover potential
vulnerabilities of the victim policy in a principled way. We also present a
novel Bias-Reduction (BR) method to boost IMAP further. Our experiments
validate the effectiveness of the four types of adversarial intrinsic
regularizers and BR in enhancing black-box adversarial policy learning across a
variety of environments. Our IMAP successfully evades two types of defense
methods, adversarial training and robust regularizer, decreasing the
performance of the state-of-the-art robust WocaR-PPO agents by 34%-54% across
four single-agent tasks. IMAP also achieves a state-of-the-art attacking
success rate of 83.91% in the multi-agent game YouShallNotPass
- …