1,313 research outputs found
BASAR:Black-box Attack on Skeletal Action Recognition
Skeletal motion plays a vital role in human activity recognition as either an
independent data source or a complement. The robustness of skeleton-based
activity recognizers has been questioned recently, which shows that they are
vulnerable to adversarial attacks when the full-knowledge of the recognizer is
accessible to the attacker. However, this white-box requirement is overly
restrictive in most scenarios and the attack is not truly threatening. In this
paper, we show that such threats do exist under black-box settings too. To this
end, we propose the first black-box adversarial attack method BASAR. Through
BASAR, we show that adversarial attack is not only truly a threat but also can
be extremely deceitful, because on-manifold adversarial samples are rather
common in skeletal motions, in contrast to the common belief that adversarial
samples only exist off-manifold. Through exhaustive evaluation and comparison,
we show that BASAR can deliver successful attacks across models, data, and
attack modes. Through harsh perceptual studies, we show that it achieves
effective yet imperceptible attacks. By analyzing the attack on different
activity recognizers, BASAR helps identify the potential causes of their
vulnerability and provides insights on what classifiers are likely to be more
robust against attack. Code is available at
https://github.com/realcrane/BASAR-Black-box-Attack-on-Skeletal-Action-Recognition.Comment: Accepted in CVPR 202
Fourier Analysis on Robustness of Graph Convolutional Neural Networks for Skeleton-based Action Recognition
Using Fourier analysis, we explore the robustness and vulnerability of graph
convolutional neural networks (GCNs) for skeleton-based action recognition. We
adopt a joint Fourier transform (JFT), a combination of the graph Fourier
transform (GFT) and the discrete Fourier transform (DFT), to examine the
robustness of adversarially-trained GCNs against adversarial attacks and common
corruptions. Experimental results with the NTU RGB+D dataset reveal that
adversarial training does not introduce a robustness trade-off between
adversarial attacks and low-frequency perturbations, which typically occurs
during image classification based on convolutional neural networks. This
finding indicates that adversarial training is a practical approach to
enhancing robustness against adversarial attacks and common corruptions in
skeleton-based action recognition. Furthermore, we find that the Fourier
approach cannot explain vulnerability against skeletal part occlusion
corruption, which highlights its limitations. These findings extend our
understanding of the robustness of GCNs, potentially guiding the development of
more robust learning methods for skeleton-based action recognition.Comment: 17 pages, 13 figure
Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack
Action recognition has been heavily employed in many applications such as
autonomous vehicles, surveillance, etc, where its robustness is a primary
concern. In this paper, we examine the robustness of state-of-the-art action
recognizers against adversarial attack, which has been rarely investigated so
far. To this end, we propose a new method to attack action recognizers that
rely on 3D skeletal motion. Our method involves an innovative perceptual loss
that ensures the imperceptibility of the attack. Empirical studies demonstrate
that our method is effective in both white-box and black-box scenarios. Its
generalizability is evidenced on a variety of action recognizers and datasets.
Its versatility is shown in different attacking strategies. Its deceitfulness
is proven in extensive perceptual studies. Our method shows that adversarial
attack on 3D skeletal motions, one type of time-series data, is significantly
different from traditional adversarial attack problems. Its success raises
serious concern on the robustness of action recognizers and provides insights
on potential improvements.Comment: Accepted in CVPR 2021. arXiv admin note: substantial text overlap
with arXiv:1911.0710
Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack
Human Activity Recognition (HAR) has been employed in a wide range of
applications, e.g. self-driving cars, where safety and lives are at stake.
Recently, the robustness of existing skeleton-based HAR methods has been
questioned due to their vulnerability to adversarial attacks, which causes
concerns considering the scale of the implication. However, the proposed
attacks require the full-knowledge of the attacked classifier, which is overly
restrictive. In this paper, we show such threats indeed exist, even when the
attacker only has access to the input/output of the model. To this end, we
propose the very first black-box adversarial attack approach in skeleton-based
HAR called BASAR. BASAR explores the interplay between the classification
boundary and the natural motion manifold. To our best knowledge, this is the
first time data manifold is introduced in adversarial attacks on time series.
Via BASAR, we find on-manifold adversarial samples are extremely deceitful and
rather common in skeletal motions, in contrast to the common belief that
adversarial samples only exist off-manifold. Through exhaustive evaluation, we
show that BASAR can deliver successful attacks across classifiers, datasets,
and attack modes. By attack, BASAR helps identify the potential causes of the
model vulnerability and provides insights on possible improvements. Finally, to
mitigate the newly identified threat, we propose a new adversarial training
approach by leveraging the sophisticated distributions of on/off-manifold
adversarial samples, called mixed manifold-based adversarial training (MMAT).
MMAT can successfully help defend against adversarial attacks without
compromising classification accuracy.Comment: arXiv admin note: substantial text overlap with arXiv:2103.0526
Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient
Recently, methods for skeleton-based human activity recognition have been
shown to be vulnerable to adversarial attacks. However, these attack methods
require either the full knowledge of the victim (i.e. white-box attacks),
access to training data (i.e. transfer-based attacks) or frequent model queries
(i.e. black-box attacks). All their requirements are highly restrictive,
raising the question of how detrimental the vulnerability is. In this paper, we
show that the vulnerability indeed exists. To this end, we consider a new
attack task: the attacker has no access to the victim model or the training
data or labels, where we coin the term hard no-box attack. Specifically, we
first learn a motion manifold where we define an adversarial loss to compute a
new gradient for the attack, named skeleton-motion-informed (SMI) gradient. Our
gradient contains information of the motion dynamics, which is different from
existing gradient-based attack methods that compute the loss gradient assuming
each dimension in the data is independent. The SMI gradient can augment many
gradient-based attack methods, leading to a new family of no-box attack
methods. Extensive evaluation and comparison show that our method imposes a
real threat to existing classifiers. They also show that the SMI gradient
improves the transferability and imperceptibility of adversarial samples in
both no-box and transfer-based black-box settings.Comment: Camera-ready version for ICCV 202
Defending Black-box Classifiers by Bayesian Boundary Correction
Classifiers based on deep neural networks have been recently challenged by
Adversarial Attack, where the widely existing vulnerability has invoked the
research in defending them from potential threats. Given a vulnerable
classifier, existing defense methods are mostly white-box and often require
re-training the victim under modified loss functions/training regimes. While
the model/data/training specifics of the victim are usually unavailable to the
user, re-training is unappealing, if not impossible for reasons such as limited
computational resources. To this end, we propose a new black-box defense
framework. It can turn any pre-trained classifier into a resilient one with
little knowledge of the model specifics. This is achieved by new joint Bayesian
treatments on the clean data, the adversarial examples and the classifier, for
maximizing their joint probability. It is further equipped with a new
post-train strategy which keeps the victim intact. We name our framework
Bayesian Boundary Correction (BBC). BBC is a general and flexible framework
that can easily adapt to different data types. We instantiate BBC for image
classification and skeleton-based human activity recognition, for both static
and dynamic data. Exhaustive evaluation shows that BBC has superior robustness
and can enhance robustness without severely hurting the clean accuracy,
compared with existing defense methods.Comment: arXiv admin note: text overlap with arXiv:2203.0471
- …