1,427 research outputs found
Adversarial Examples in the Physical World: A Survey
Deep neural networks (DNNs) have demonstrated high vulnerability to
adversarial examples. Besides the attacks in the digital world, the practical
implications of adversarial examples in the physical world present significant
challenges and safety concerns. However, current research on physical
adversarial examples (PAEs) lacks a comprehensive understanding of their unique
characteristics, leading to limited significance and understanding. In this
paper, we address this gap by thoroughly examining the characteristics of PAEs
within a practical workflow encompassing training, manufacturing, and
re-sampling processes. By analyzing the links between physical adversarial
attacks, we identify manufacturing and re-sampling as the primary sources of
distinct attributes and particularities in PAEs. Leveraging this knowledge, we
develop a comprehensive analysis and classification framework for PAEs based on
their specific characteristics, covering over 100 studies on physical-world
adversarial examples. Furthermore, we investigate defense strategies against
PAEs and identify open challenges and opportunities for future research. We aim
to provide a fresh, thorough, and systematic understanding of PAEs, thereby
promoting the development of robust adversarial learning and its application in
open-world scenarios.Comment: Adversarial examples, physical-world scenarios, attacks and defense
SentiNet: Detecting Physical Attacks Against Deep Learning Systems
SentiNet is a novel detection framework for physical attacks on neural
networks, a class of attacks that constrains an adversarial region to a visible
portion of an image. Physical attacks have been shown to be robust and flexible
techniques suited for deployment in real-world scenarios. Unlike most other
adversarial detection works, SentiNet does not require training a model or
preknowledge of an attack prior to detection. This attack-agnostic approach is
appealing due to the large number of possible mechanisms and vectors of attack
an attack-specific defense would have to consider. By leveraging the neural
network's susceptibility to attacks and by using techniques from model
interpretability and object detection as detection mechanisms, SentiNet turns a
weakness of a model into a strength. We demonstrate the effectiveness of
SentiNet on three different attacks - i.e., adversarial examples, data
poisoning attacks, and trojaned networks - that have large variations in
deployment mechanisms, and show that our defense is able to achieve very
competitive performance metrics for all three threats, even against strong
adaptive adversaries with full knowledge of SentiNet.Comment: 15 page
Simultaneously Optimizing Perturbations and Positions for Black-box Adversarial Patch Attacks
Adversarial patch is an important form of real-world adversarial attack that
brings serious risks to the robustness of deep neural networks. Previous
methods generate adversarial patches by either optimizing their perturbation
values while fixing the pasting position or manipulating the position while
fixing the patch's content. This reveals that the positions and perturbations
are both important to the adversarial attack. For that, in this paper, we
propose a novel method to simultaneously optimize the position and perturbation
for an adversarial patch, and thus obtain a high attack success rate in the
black-box setting. Technically, we regard the patch's position, the
pre-designed hyper-parameters to determine the patch's perturbations as the
variables, and utilize the reinforcement learning framework to simultaneously
solve for the optimal solution based on the rewards obtained from the target
model with a small number of queries. Extensive experiments are conducted on
the Face Recognition (FR) task, and results on four representative FR models
show that our method can significantly improve the attack success rate and
query efficiency. Besides, experiments on the commercial FR service and
physical environments confirm its practical application value. We also extend
our method to the traffic sign recognition task to verify its generalization
ability.Comment: Accepted by TPAMI 202
- …