20,159 research outputs found
Non-Negative Networks Against Adversarial Attacks
Adversarial attacks against neural networks are a problem of considerable
importance, for which effective defenses are not yet readily available. We make
progress toward this problem by showing that non-negative weight constraints
can be used to improve resistance in specific scenarios. In particular, we show
that they can provide an effective defense for binary classification problems
with asymmetric cost, such as malware or spam detection. We also show the
potential for non-negativity to be helpful to non-binary problems by applying
it to image classification
Adversarial Examples: Attacks and Defenses for Deep Learning
With rapid progress and significant successes in a wide spectrum of
applications, deep learning is being applied in many safety-critical
environments. However, deep neural networks have been recently found vulnerable
to well-designed input samples, called adversarial examples. Adversarial
examples are imperceptible to human but can easily fool deep neural networks in
the testing/deploying stage. The vulnerability to adversarial examples becomes
one of the major risks for applying deep neural networks in safety-critical
environments. Therefore, attacks and defenses on adversarial examples draw
great attention. In this paper, we review recent findings on adversarial
examples for deep neural networks, summarize the methods for generating
adversarial examples, and propose a taxonomy of these methods. Under the
taxonomy, applications for adversarial examples are investigated. We further
elaborate on countermeasures for adversarial examples and explore the
challenges and the potential solutions.Comment: Github: https://github.com/chbrian/awesome-adversarial-examples-d
Reinforcement Learning for Autonomous Defence in Software-Defined Networking
Despite the successful application of machine learning (ML) in a wide range
of domains, adaptability---the very property that makes machine learning
desirable---can be exploited by adversaries to contaminate training and evade
classification. In this paper, we investigate the feasibility of applying a
specific class of machine learning algorithms, namely, reinforcement learning
(RL) algorithms, for autonomous cyber defence in software-defined networking
(SDN). In particular, we focus on how an RL agent reacts towards different
forms of causative attacks that poison its training process, including
indiscriminate and targeted, white-box and black-box attacks. In addition, we
also study the impact of the attack timing, and explore potential
countermeasures such as adversarial training.Comment: 20 pages, 8 figure
A Robust Approach for Securing Audio Classification Against Adversarial Attacks
Adversarial audio attacks can be considered as a small perturbation
unperceptive to human ears that is intentionally added to the audio signal and
causes a machine learning model to make mistakes. This poses a security concern
about the safety of machine learning models since the adversarial attacks can
fool such models toward the wrong predictions. In this paper we first review
some strong adversarial attacks that may affect both audio signals and their 2D
representations and evaluate the resiliency of the most common machine learning
model, namely deep learning models and support vector machines (SVM) trained on
2D audio representations such as short time Fourier transform (STFT), discrete
wavelet transform (DWT) and cross recurrent plot (CRP) against several
state-of-the-art adversarial attacks. Next, we propose a novel approach based
on pre-processed DWT representation of audio signals and SVM to secure audio
systems against adversarial attacks. The proposed architecture has several
preprocessing modules for generating and enhancing spectrograms including
dimension reduction and smoothing. We extract features from small patches of
the spectrograms using speeded up robust feature (SURF) algorithm which are
further used to generate a codebook using the K-Means++ algorithm. Finally,
codewords are used to train a SVM on the codebook of the SURF-generated
vectors. All these steps yield to a novel approach for audio classification
that provides a good trade-off between accuracy and resilience. Experimental
results on three environmental sound datasets show the competitive performance
of proposed approach compared to the deep neural networks both in terms of
accuracy and robustness against strong adversarial attacks.Comment: Paper Accepted for Publication in IEEE Transactions on Information
Forensics and Securit
Adversarial Defense Framework for Graph Neural Network
Graph neural network (GNN), as a powerful representation learning model on
graph data, attracts much attention across various disciplines. However, recent
studies show that GNN is vulnerable to adversarial attacks. How to make GNN
more robust? What are the key vulnerabilities in GNN? How to address the
vulnerabilities and defense GNN against the adversarial attacks? In this paper,
we propose DefNet, an effective adversarial defense framework for GNNs. In
particular, we first investigate the latent vulnerabilities in every layer of
GNNs and propose corresponding strategies including dual-stage aggregation and
bottleneck perceptron. Then, to cope with the scarcity of training data, we
propose an adversarial contrastive learning method to train the GNN in a
conditional GAN manner by leveraging the high-level graph representation.
Extensive experiments on three public datasets demonstrate the effectiveness of
DefNet in improving the robustness of popular GNN variants, such as Graph
Convolutional Network and GraphSAGE, under various types of adversarial
attacks
Towards a Robust Deep Neural Network in Texts: A Survey
Deep neural networks (DNNs) have achieved remarkable success in various tasks
(e.g., image classification, speech recognition, and natural language
processing). However, researches have shown that DNN models are vulnerable to
adversarial examples, which cause incorrect predictions by adding imperceptible
perturbations into normal inputs. Studies on adversarial examples in image
domain have been well investigated, but in texts the research is not enough,
let alone a comprehensive survey in this field. In this paper, we aim at
presenting a comprehensive understanding of adversarial attacks and
corresponding mitigation strategies in texts. Specifically, we first give a
taxonomy of adversarial attacks and defenses in texts from the perspective of
different natural language processing (NLP) tasks, and then introduce how to
build a robust DNN model via testing and verification. Finally, we discuss the
existing challenges of adversarial attacks and defenses in texts and present
the future research directions in this emerging field
Adversarial Reinforcement Learning under Partial Observability in Autonomous Computer Network Defence
Recent studies have demonstrated that reinforcement learning (RL) agents are
susceptible to adversarial manipulation, similar to vulnerabilities previously
demonstrated in the supervised learning setting. While most existing work
studies the problem in the context of computer vision or console games, this
paper focuses on reinforcement learning in autonomous cyber defence under
partial observability. We demonstrate that under the black-box setting, where
the attacker has no direct access to the target RL model, causative
attacks---attacks that target the training process---can poison RL agents even
if the attacker only has partial observability of the environment. In addition,
we propose an inversion defence method that aims to apply the opposite
perturbation to that which an attacker might use to generate their adversarial
samples. Our experimental results illustrate that the countermeasure can
effectively reduce the impact of the causative attack, while not significantly
affecting the training process in non-attack scenarios.Comment: 8 pages, 4 figure
Enhancing Robustness of Deep Neural Networks Against Adversarial Malware Samples: Principles, Framework, and AICS'2019 Challenge
Malware continues to be a major cyber threat, despite the tremendous effort
that has been made to combat them. The number of malware in the wild steadily
increases over time, meaning that we must resort to automated defense
techniques. This naturally calls for machine learning based malware detection.
However, machine learning is known to be vulnerable to adversarial evasion
attacks that manipulate a small number of features to make classifiers wrongly
recognize a malware sample as a benign one. The state-of-the-art is that there
are no effective countermeasures against these attacks. Inspired by the
AICS'2019 Challenge, we systematize a number of principles for enhancing the
robustness of neural networks against adversarial malware evasion attacks. Some
of these principles have been scattered in the literature, but others are
proposed in this paper for the first time. Under the guidance of these
principles, we propose a framework and an accompanying training algorithm,
which are then applied to the AICS'2019 challenge. Our experimental results
have been submitted to the challenge organizer for evaluation.Comment: 8 pages, 4 figures, AICS 2019; for the fully-fledged version, please
see arxiv:2004.0791
Gradient Adversarial Training of Neural Networks
We propose gradient adversarial training, an auxiliary deep learning
framework applicable to different machine learning problems. In gradient
adversarial training, we leverage a prior belief that in many contexts,
simultaneous gradient updates should be statistically indistinguishable from
each other. We enforce this consistency using an auxiliary network that
classifies the origin of the gradient tensor, and the main network serves as an
adversary to the auxiliary network in addition to performing standard
task-based training. We demonstrate gradient adversarial training for three
different scenarios: (1) as a defense to adversarial examples we classify
gradient tensors and tune them to be agnostic to the class of their
corresponding example, (2) for knowledge distillation, we do binary
classification of gradient tensors derived from the student or teacher network
and tune the student gradient tensor to mimic the teacher's gradient tensor;
and (3) for multi-task learning we classify the gradient tensors derived from
different task loss functions and tune them to be statistically
indistinguishable. For each of the three scenarios we show the potential of
gradient adversarial training procedure. Specifically, gradient adversarial
training increases the robustness of a network to adversarial attacks, is able
to better distill the knowledge from a teacher network to a student network
compared to soft targets, and boosts multi-task learning by aligning the
gradient tensors derived from the task specific loss functions. Overall, our
experiments demonstrate that gradient tensors contain latent information about
whatever tasks are being trained, and can support diverse machine learning
problems when intelligently guided through adversarialization using a auxiliary
network.Comment: 13 pages, 4 figure
Characterizing Audio Adversarial Examples Using Temporal Dependency
Recent studies have highlighted adversarial examples as a ubiquitous threat
to different neural network models and many downstream applications.
Nonetheless, as unique data properties have inspired distinct and powerful
learning principles, this paper aims to explore their potentials towards
mitigating adversarial inputs. In particular, our results reveal the importance
of using the temporal dependency in audio data to gain discriminate power
against adversarial examples. Tested on the automatic speech recognition (ASR)
tasks and three recent audio adversarial attacks, we find that (i) input
transformation developed from image adversarial defense provides limited
robustness improvement and is subtle to advanced attacks; (ii) temporal
dependency can be exploited to gain discriminative power against audio
adversarial examples and is resistant to adaptive attacks considered in our
experiments. Our results not only show promising means of improving the
robustness of ASR systems, but also offer novel insights in exploiting
domain-specific data properties to mitigate negative effects of adversarial
examples
- …