10,911 research outputs found
Security Matters: A Survey on Adversarial Machine Learning
Adversarial machine learning is a fast growing research area, which considers
the scenarios when machine learning systems may face potential adversarial
attackers, who intentionally synthesize input data to make a well-trained model
to make mistake. It always involves a defending side, usually a classifier, and
an attacking side that aims to cause incorrect output. The earliest studies on
the adversarial examples for machine learning algorithms start from the
information security area, which considers a much wider varieties of attacking
methods. But recent research focus that popularized by the deep learning
community places strong emphasis on how the "imperceivable" perturbations on
the normal inputs may cause dramatic mistakes by the deep learning with
supposed super-human accuracy. This paper serves to give a comprehensive
introduction to a range of aspects of the adversarial deep learning topic,
including its foundations, typical attacking and defending strategies, and some
extended studies
Distributed Detection in Tree Topologies with Byzantines
In this paper, we consider the problem of distributed detection in tree
topologies in the presence of Byzantines. The expression for minimum attacking
power required by the Byzantines to blind the fusion center (FC) is obtained.
More specifically, we show that when more than a certain fraction of individual
node decisions are falsified, the decision fusion scheme becomes completely
incapable. We obtain closed form expressions for the optimal attacking
strategies that minimize the detection error exponent at the FC. We also look
at the possible counter-measures from the FC's perspective to protect the
network from these Byzantines. We formulate the robust topology design problem
as a bi-level program and provide an efficient algorithm to solve it. We also
provide some numerical results to gain insights into the solution
Query-Free Attacks on Industry-Grade Face Recognition Systems under Resource Constraints
To launch black-box attacks against a Deep Neural Network (DNN) based Face
Recognition (FR) system, one needs to build \textit{substitute} models to
simulate the target model, so the adversarial examples discovered from
substitute models could also mislead the target model. Such
\textit{transferability} is achieved in recent studies through querying the
target model to obtain data for training the substitute models. A real-world
target, likes the FR system of law enforcement, however, is less accessible to
the adversary. To attack such a system, a substitute model with similar quality
as the target model is needed to identify their common defects. This is hard
since the adversary often does not have the enough resources to train such a
powerful model (hundreds of millions of images and rooms of GPUs are needed to
train a commercial FR system).
We found in our research, however, that a resource-constrained adversary
could still effectively approximate the target model's capability to recognize
\textit{specific} individuals, by training \textit{biased} substitute models on
additional images of those victims whose identities the attacker want to cover
or impersonate. This is made possible by a new property we discovered, called
\textit{Nearly Local Linearity} (NLL), which models the observation that an
ideal DNN model produces the image representations (embeddings) whose distances
among themselves truthfully describe the human perception of the differences
among the input images. By simulating this property around the victim's images,
we significantly improve the transferability of black-box impersonation attacks
by nearly 50\%. Particularly, we successfully attacked a commercial system
trained over 20 million images, using 4 million images and 1/5 of the training
time but achieving 62\% transferability in an impersonation attack and 89\% in
a dodging attack
Towards Query Efficient Black-box Attacks: An Input-free Perspective
Recent studies have highlighted that deep neural networks (DNNs) are
vulnerable to adversarial attacks, even in a black-box scenario. However, most
of the existing black-box attack algorithms need to make a huge amount of
queries to perform attacks, which is not practical in the real world. We note
one of the main reasons for the massive queries is that the adversarial example
is required to be visually similar to the original image, but in many cases,
how adversarial examples look like does not matter much. It inspires us to
introduce a new attack called \emph{input-free} attack, under which an
adversary can choose an arbitrary image to start with and is allowed to add
perceptible perturbations on it. Following this approach, we propose two
techniques to significantly reduce the query complexity. First, we initialize
an adversarial example with a gray color image on which every pixel has roughly
the same importance for the target model. Then we shrink the dimension of the
attack space by perturbing a small region and tiling it to cover the input
image. To make our algorithm more effective, we stabilize a projected gradient
ascent algorithm with momentum, and also propose a heuristic approach for
region size selection. Through extensive experiments, we show that with only
1,701 queries on average, we can perturb a gray image to any target class of
ImageNet with a 100\% success rate on InceptionV3. Besides, our algorithm has
successfully defeated two real-world systems, the Clarifai food detection API
and the Baidu Animal Identification API.Comment: Accepted by 11th ACM Workshop on Artificial Intelligence and Security
(AISec) with the 25th ACM Conference on Computer and Communications Security
(CCS
Distributed Submodular Minimization And Motion Coordination Over Discrete State Space
We develop a framework for the distributed minimization of submodular
functions. Submodular functions are a discrete analog of convex functions and
are extensively used in large-scale combinatorial optimization problems. While
there has been significant interest in the distributed formulations of convex
optimization problems, distributed minimization of submodular functions has
received relatively little research attention. Our framework relies on an
equivalent convex reformulation of a submodular minimization problem, which is
efficiently computable. We then use this relaxation to exploit methods for the
distributed optimization of convex functions. The proposed framework is
applicable to submodular set functions as well as to a wider class of
submodular functions defined over certain lattices. We also propose an approach
for solving distributed motion coordination problems in discrete state space
based on submodular function minimization. We establish through a challenging
setup of capture the flag game that submodular functions over lattices can be
used to design artificial potential fields over discrete state space in which
the agents are attracted towards their goals and are repulsed from obstacles
and from each other for collision avoidance
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
The vulnerability of deep networks to adversarial attacks is a central
problem for deep learning from the perspective of both cognition and security.
The current most successful defense method is to train a classifier using
adversarial images created during learning. Another defense approach involves
transformation or purification of the original input to remove adversarial
signals before the image is classified. We focus on defending naturally-trained
classifiers using Markov Chain Monte Carlo (MCMC) sampling with an Energy-Based
Model (EBM) for adversarial purification. In contrast to adversarial training,
our approach is intended to secure pre-existing and highly vulnerable
classifiers.
The memoryless behavior of long-run MCMC sampling will eventually remove
adversarial signals, while metastable behavior preserves consistent appearance
of MCMC samples after many steps to allow accurate long-run prediction.
Balancing these factors can lead to effective purification and robust
classification. We evaluate adversarial defense with an EBM using the strongest
known attacks against purification. Our contributions are 1) an improved method
for training EBM's with realistic long-run MCMC samples, 2) an
Expectation-Over-Transformation (EOT) defense that resolves theoretical
ambiguities for stochastic defenses and from which the EOT attack naturally
follows, and 3) state-of-the-art adversarial defense for naturally-trained
classifiers and competitive defense compared to adversarially-trained
classifiers on Cifar-10, SVHN, and Cifar-100. Code and pre-trained models are
available at https://github.com/point0bar1/ebm-defense.Comment: ICLR 202
Securing Edge Networks with Securebox
The number of mobile and IoT devices connected to home and enterprise
networks is growing fast. These devices offer new services and experiences for
the users; however, they also present new classes of security threats
pertaining to data and device safety and user privacy. In this article, we
first analyze the potential threats presented by these devices connected to
edge networks. We then propose Securebox: a new cloud-driven, low cost
Security-as-a-Service solution that applies Software-Defined Networking (SDN)
to improve network monitoring, security and management. Securebox enables
remote management of networks through a cloud security service (CSS) with
minimal user intervention required. To reduce costs and improve the
scalability, Securebox is based on virtualized middleboxes provided by CSS. Our
proposal differs from the existing solutions by integrating the SDN and cloud
into a unified edge security solution, and by offering a collaborative
protection mechanism that enables rapid security policy dissemination across
all connected networks in mitigating new threats or attacks detected by the
system. We have implemented two Securebox prototypes, using a low-cost
Raspberry-PI and off-the-shelf fanless PC. Our system evaluation has shown that
Securebox can achieve automatic network security and be deployed incrementally
to the infrastructure with low management overhead
Simple Black-box Adversarial Attacks
We propose an intriguingly simple method for the construction of adversarial
images in the black-box setting. In constrast to the white-box scenario,
constructing black-box adversarial images has the additional constraint on
query budget, and efficient attacks remain an open problem to date. With only
the mild assumption of continuous-valued confidence scores, our highly
query-efficient algorithm utilizes the following simple iterative principle: we
randomly sample a vector from a predefined orthonormal basis and either add or
subtract it to the target image. Despite its simplicity, the proposed method
can be used for both untargeted and targeted attacks -- resulting in previously
unprecedented query efficiency in both settings. We demonstrate the efficacy
and efficiency of our algorithm on several real world settings including the
Google Cloud Vision API. We argue that our proposed algorithm should serve as a
strong baseline for future black-box attacks, in particular because it is
extremely fast and its implementation requires less than 20 lines of PyTorch
code.Comment: Published at ICML 201
Low Frequency Adversarial Perturbation
Adversarial images aim to change a target model's decision by minimally
perturbing a target image. In the black-box setting, the absence of gradient
information often renders this search problem costly in terms of query
complexity. In this paper we propose to restrict the search for adversarial
images to a low frequency domain. This approach is readily compatible with many
existing black-box attack frameworks and consistently reduces their query cost
by 2 to 4 times. Further, we can circumvent image transformation defenses even
when both the model and the defense strategy are unknown. Finally, we
demonstrate the efficacy of this technique by fooling the Google Cloud Vision
platform with an unprecedented low number of model queries.Comment: 9 pages, 9 figures. Accepted to UAI 201
Query-Efficient Black-Box Attack Against Sequence-Based Malware Classifiers
In this paper, we present a generic, query-efficient black-box attack against
API call-based machine learning malware classifiers. We generate adversarial
examples by modifying the malware's API call sequences and non-sequential
features (printable strings), and these adversarial examples will be
misclassified by the target malware classifier without affecting the malware's
functionality. In contrast to previous studies, our attack minimizes the number
of malware classifier queries required. In addition, in our attack, the
attacker must only know the class predicted by the malware classifier; attacker
knowledge of the malware classifier's confidence score is optional. We evaluate
the attack effectiveness when attacks are performed against a variety of
malware classifier architectures, including recurrent neural network (RNN)
variants, deep neural networks, support vector machines, and gradient boosted
decision trees. Our attack success rate is around 98% when the classifier's
confidence score is known and 64% when just the classifier's predicted class is
known. We implement four state-of-the-art query-efficient attacks and show that
our attack requires fewer queries and less knowledge about the attacked model's
architecture than other existing query-efficient attacks, making it practical
for attacking cloud-based malware classifiers at a minimal cost.Comment: Accepted as a conference paper at ACSAC 202
- …