14 research outputs found
Defending Black-box Classifiers by Bayesian Boundary Correction
Classifiers based on deep neural networks have been recently challenged by
Adversarial Attack, where the widely existing vulnerability has invoked the
research in defending them from potential threats. Given a vulnerable
classifier, existing defense methods are mostly white-box and often require
re-training the victim under modified loss functions/training regimes. While
the model/data/training specifics of the victim are usually unavailable to the
user, re-training is unappealing, if not impossible for reasons such as limited
computational resources. To this end, we propose a new black-box defense
framework. It can turn any pre-trained classifier into a resilient one with
little knowledge of the model specifics. This is achieved by new joint Bayesian
treatments on the clean data, the adversarial examples and the classifier, for
maximizing their joint probability. It is further equipped with a new
post-train strategy which keeps the victim intact. We name our framework
Bayesian Boundary Correction (BBC). BBC is a general and flexible framework
that can easily adapt to different data types. We instantiate BBC for image
classification and skeleton-based human activity recognition, for both static
and dynamic data. Exhaustive evaluation shows that BBC has superior robustness
and can enhance robustness without severely hurting the clean accuracy,
compared with existing defense methods.Comment: arXiv admin note: text overlap with arXiv:2203.0471
BASAR:Black-box Attack on Skeletal Action Recognition
Skeletal motion plays a vital role in human activity recognition as either an
independent data source or a complement. The robustness of skeleton-based
activity recognizers has been questioned recently, which shows that they are
vulnerable to adversarial attacks when the full-knowledge of the recognizer is
accessible to the attacker. However, this white-box requirement is overly
restrictive in most scenarios and the attack is not truly threatening. In this
paper, we show that such threats do exist under black-box settings too. To this
end, we propose the first black-box adversarial attack method BASAR. Through
BASAR, we show that adversarial attack is not only truly a threat but also can
be extremely deceitful, because on-manifold adversarial samples are rather
common in skeletal motions, in contrast to the common belief that adversarial
samples only exist off-manifold. Through exhaustive evaluation and comparison,
we show that BASAR can deliver successful attacks across models, data, and
attack modes. Through harsh perceptual studies, we show that it achieves
effective yet imperceptible attacks. By analyzing the attack on different
activity recognizers, BASAR helps identify the potential causes of their
vulnerability and provides insights on what classifiers are likely to be more
robust against attack. Code is available at
https://github.com/realcrane/BASAR-Black-box-Attack-on-Skeletal-Action-Recognition.Comment: Accepted in CVPR 202
Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples
Safeguarding data from unauthorized exploitation is vital for privacy and security, especially in recent rampant research in security breach such as adversarial/membership attacks. To this end,unlearnable examples (UEs) have been recently proposed as a compelling protection, by adding imperceptible perturbation to data so that models trained on them cannot classify them accurately on original clean distribution. Unfortunately, we find UEs provide a false sense of security, because they cannot stop unauthorized users from utilizing other unprotected data to remove the protection, by turning unlearnable data into learnable again. Motivated by this observation, we formally define a new threat by introducinglearnable unauthorized examples (LEs) which are UEs with their protection removed. The core of this approach is a novel purification process that projects UEs onto the manifold of LEs. This is realized by a new joint-conditional diffusion model which denoises UEs conditioned on the pixel and perceptual similarity between UEs and LEs. Extensive experiments demonstrate that LE delivers state-of-the-art countering performance against both supervised UEs and unsupervised UEs in various scenarios, which is the first generalizable countermeasure to UEs across supervised learning and unsupervised learning. Our code is available at https://github.com/jiangw-0/LE_JCDP
Understanding the Vulnerability of Skeleton-based Human Activity Recognition via Black-box Attack
Human Activity Recognition (HAR) has been employed in a wide range of
applications, e.g. self-driving cars, where safety and lives are at stake.
Recently, the robustness of existing skeleton-based HAR methods has been
questioned due to their vulnerability to adversarial attacks, which causes
concerns considering the scale of the implication. However, the proposed
attacks require the full-knowledge of the attacked classifier, which is overly
restrictive. In this paper, we show such threats indeed exist, even when the
attacker only has access to the input/output of the model. To this end, we
propose the very first black-box adversarial attack approach in skeleton-based
HAR called BASAR. BASAR explores the interplay between the classification
boundary and the natural motion manifold. To our best knowledge, this is the
first time data manifold is introduced in adversarial attacks on time series.
Via BASAR, we find on-manifold adversarial samples are extremely deceitful and
rather common in skeletal motions, in contrast to the common belief that
adversarial samples only exist off-manifold. Through exhaustive evaluation, we
show that BASAR can deliver successful attacks across classifiers, datasets,
and attack modes. By attack, BASAR helps identify the potential causes of the
model vulnerability and provides insights on possible improvements. Finally, to
mitigate the newly identified threat, we propose a new adversarial training
approach by leveraging the sophisticated distributions of on/off-manifold
adversarial samples, called mixed manifold-based adversarial training (MMAT).
MMAT can successfully help defend against adversarial attacks without
compromising classification accuracy.Comment: arXiv admin note: substantial text overlap with arXiv:2103.0526
Chinese Farmers’ Preference for Rural Homestead Land Use: Mechanism, Knowledge and Perception
Defending Black-box Skeleton-based Human Activity Classifiers
Deep learning has been regarded as the `go to' solution for many tasks today,
but its intrinsic vulnerability to malicious attacks has become a major
concern. The vulnerability is affected by a variety of factors including
models, tasks, data, and attackers. Consequently, methods such as Adversarial
Training and Randomized Smoothing have been proposed to tackle the problem in a
wide range of applications. In this paper, we investigate skeleton-based Human
Activity Recognition, which is an important type of time-series data but
under-explored in defense against attacks. Our method is featured by (1) a new
Bayesian Energy-based formulation of robust discriminative classifiers, (2) a
new parameterization of the adversarial sample manifold of actions, and (3) a
new post-train Bayesian treatment on both the adversarial samples and the
classifier. We name our framework Bayesian Energy-based Adversarial Training or
BEAT. BEAT is straightforward but elegant, which turns vulnerable black-box
classifiers into robust ones without sacrificing accuracy. It demonstrates
surprising and universal effectiveness across a wide range of action
classifiers and datasets, under various attacks
Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples
Safeguarding data from unauthorized exploitation is vital for privacy and
security, especially in recent rampant research in security breach such as
adversarial/membership attacks. To this end, \textit{unlearnable examples}
(UEs) have been recently proposed as a compelling protection, by adding
imperceptible perturbation to data so that models trained on them cannot
classify them accurately on original clean distribution. Unfortunately, we find
UEs provide a false sense of security, because they cannot stop unauthorized
users from utilizing other unprotected data to remove the protection, by
turning unlearnable data into learnable again. Motivated by this observation,
we formally define a new threat by introducing \textit{learnable unauthorized
examples} (LEs) which are UEs with their protection removed. The core of this
approach is a novel purification process that projects UEs onto the manifold of
LEs. This is realized by a new joint-conditional diffusion model which denoises
UEs conditioned on the pixel and perceptual similarity between UEs and LEs.
Extensive experiments demonstrate that LE delivers state-of-the-art countering
performance against both supervised UEs and unsupervised UEs in various
scenarios, which is the first generalizable countermeasure to UEs across
supervised learning and unsupervised learning