9,514 research outputs found
DARTS: Deceiving Autonomous Cars with Toxic Signs
Sign recognition is an integral part of autonomous cars. Any
misclassification of traffic signs can potentially lead to a multitude of
disastrous consequences, ranging from a life-threatening accident to even a
large-scale interruption of transportation services relying on autonomous cars.
In this paper, we propose and examine security attacks against sign recognition
systems for Deceiving Autonomous caRs with Toxic Signs (we call the proposed
attacks DARTS). In particular, we introduce two novel methods to create these
toxic signs. First, we propose Out-of-Distribution attacks, which expand the
scope of adversarial examples by enabling the adversary to generate these
starting from an arbitrary point in the image space compared to prior attacks
which are restricted to existing training/test data (In-Distribution). Second,
we present the Lenticular Printing attack, which relies on an optical
phenomenon to deceive the traffic sign recognition system. We extensively
evaluate the effectiveness of the proposed attacks in both virtual and
real-world settings and consider both white-box and black-box threat models.
Our results demonstrate that the proposed attacks are successful under both
settings and threat models. We further show that Out-of-Distribution attacks
can outperform In-Distribution attacks on classifiers defended using the
adversarial training defense, exposing a new attack vector for these defenses.Comment: Submitted to ACM CCS 2018; Extended version of [1801.02780] Rogue
Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logo
Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logos
We propose a new real-world attack against the computer vision based systems
of autonomous vehicles (AVs). Our novel Sign Embedding attack exploits the
concept of adversarial examples to modify innocuous signs and advertisements in
the environment such that they are classified as the adversary's desired
traffic sign with high confidence. Our attack greatly expands the scope of the
threat posed to AVs since adversaries are no longer restricted to just
modifying existing traffic signs as in previous work. Our attack pipeline
generates adversarial samples which are robust to the environmental conditions
and noisy image transformations present in the physical world. We ensure this
by including a variety of possible image transformations in the optimization
problem used to generate adversarial samples. We verify the robustness of the
adversarial samples by printing them out and carrying out drive-by tests
simulating the conditions under which image capture would occur in a real-world
scenario. We experimented with physical attack samples for different distances,
lighting conditions and camera angles. In addition, extensive evaluations were
carried out in the virtual setting for a variety of image transformations. The
adversarial samples generated using our method have adversarial success rates
in excess of 95% in the physical as well as virtual settings.Comment: Extended abstract accepted for the 1st Deep Learning and Security
Workshop; 5 pages, 4 figure
TrISec: Training Data-Unaware Imperceptible Security Attacks on Deep Neural Networks
Most of the data manipulation attacks on deep neural networks (DNNs) during
the training stage introduce a perceptible noise that can be catered by
preprocessing during inference or can be identified during the validation
phase. Therefore, data poisoning attacks during inference (e.g., adversarial
attacks) are becoming more popular. However, many of them do not consider the
imperceptibility factor in their optimization algorithms, and can be detected
by correlation and structural similarity analysis, or noticeable (e.g., by
humans) in a multi-level security system. Moreover, the majority of the
inference attack relies on some knowledge about the training dataset. In this
paper, we propose a novel methodology which automatically generates
imperceptible attack images by using the back-propagation algorithm on
pre-trained DNNs, without requiring any information about the training dataset
(i.e., completely training data-unaware). We present a case study on traffic
sign detection using the VGGNet trained on the German Traffic Sign Recognition
Benchmarks dataset in an autonomous driving use case. Our results demonstrate
that the generated attack images successfully perform misclassification while
remaining imperceptible in both "subjective" and "objective" quality tests
Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward
Connected and autonomous vehicles (CAVs) will form the backbone of future
next-generation intelligent transportation systems (ITS) providing travel
comfort, road safety, along with a number of value-added services. Such a
transformation---which will be fuelled by concomitant advances in technologies
for machine learning (ML) and wireless communications---will enable a future
vehicular ecosystem that is better featured and more efficient. However, there
are lurking security problems related to the use of ML in such a critical
setting where an incorrect ML decision may not only be a nuisance but can lead
to loss of precious lives. In this paper, we present an in-depth overview of
the various challenges associated with the application of ML in vehicular
networks. In addition, we formulate the ML pipeline of CAVs and present various
potential security issues associated with the adoption of ML methods. In
particular, we focus on the perspective of adversarial ML attacks on CAVs and
outline a solution to defend against adversarial attacks in multiple settings
Robust Physical-World Attacks on Deep Learning Models
Recent studies show that the state-of-the-art deep neural networks (DNNs) are
vulnerable to adversarial examples, resulting from small-magnitude
perturbations added to the input. Given that that emerging physical systems are
using DNNs in safety-critical situations, adversarial examples could mislead
these systems and cause dangerous situations.Therefore, understanding
adversarial examples in the physical world is an important step towards
developing resilient learning algorithms. We propose a general attack
algorithm,Robust Physical Perturbations (RP2), to generate robust visual
adversarial perturbations under different physical conditions. Using the
real-world case of road sign classification, we show that adversarial examples
generated using RP2 achieve high targeted misclassification rates against
standard-architecture road sign classifiers in the physical world under various
environmental conditions, including viewpoints. Due to the current lack of a
standardized testing method, we propose a two-stage evaluation methodology for
robust physical adversarial examples consisting of lab and field tests. Using
this methodology, we evaluate the efficacy of physical adversarial
manipulations on real objects. Witha perturbation in the form of only black and
white stickers,we attack a real stop sign, causing targeted misclassification
in 100% of the images obtained in lab settings, and in 84.8%of the captured
video frames obtained on a moving vehicle(field test) for the target
classifier.Comment: Accepted to CVPR 201
Building Robust Deep Neural Networks for Road Sign Detection
Deep Neural Networks are built to generalize outside of training set in mind
by using techniques such as regularization, early stopping and dropout. But
considerations to make them more resilient to adversarial examples are rarely
taken. As deep neural networks become more prevalent in mission-critical and
real-time systems, miscreants start to attack them by intentionally making deep
neural networks to misclassify an object of one type to be seen as another
type. This can be catastrophic in some scenarios where the classification of a
deep neural network can lead to a fatal decision by a machine. In this work, we
used GTSRB dataset to craft adversarial samples by Fast Gradient Sign Method
and Jacobian Saliency Method, used those crafted adversarial samples to attack
another Deep Convolutional Neural Network and built the attacked network to be
more resilient against adversarial attacks by making it more robust by
Defensive Distillation and Adversarial Trainin
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
It has been shown that most machine learning algorithms are susceptible to
adversarial perturbations. Slightly perturbing an image in a carefully chosen
direction in the image space may cause a trained neural network model to
misclassify it. Recently, it was shown that physical adversarial examples
exist: printing perturbed images then taking pictures of them would still
result in misclassification. This raises security and safety concerns.
However, these experiments ignore a crucial property of physical objects: the
camera can view objects from different distances and at different angles. In
this paper, we show experiments that suggest that current constructions of
physical adversarial examples do not disrupt object detection from a moving
platform. Instead, a trained neural network classifies most of the pictures
taken from different distances and angles of a perturbed image correctly. We
believe this is because the adversarial property of the perturbation is
sensitive to the scale at which the perturbed picture is viewed, so (for
example) an autonomous car will misclassify a stop sign only from a small range
of distances.
Our work raises an important question: can one construct examples that are
adversarial for many or most viewing conditions? If so, the construction should
offer very significant insights into the internal representation of patterns by
deep networks. If not, there is a good prospect that adversarial examples can
be reduced to a curiosity with little practical impact.Comment: Accepted to CVPR 2017, Spotlight Oral Worksho
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
A large body of recent work has investigated the phenomenon of evasion
attacks using adversarial examples for deep learning systems, where the
addition of norm-bounded perturbations to the test inputs leads to incorrect
output classification. Previous work has investigated this phenomenon in
closed-world systems where training and test inputs follow a pre-specified
distribution. However, real-world implementations of deep learning
applications, such as autonomous driving and content classification are likely
to operate in the open-world environment. In this paper, we demonstrate the
success of open-world evasion attacks, where adversarial examples are generated
from out-of-distribution inputs (OOD adversarial examples). In our study, we
use 11 state-of-the-art neural network models trained on 3 image datasets of
varying complexity. We first demonstrate that state-of-the-art detectors for
out-of-distribution data are not robust against OOD adversarial examples. We
then consider 5 known defenses for adversarial examples, including
state-of-the-art robust training methods, and show that against these defenses,
OOD adversarial examples can achieve up to 4 higher target success
rates compared to adversarial examples generated from in-distribution data. We
also take a quantitative look at how open-world evasion attacks may affect
real-world systems. Finally, we present the first steps towards a robust
open-world machine learning system.Comment: 18 pages, 5 figures, 9 table
VerIDeep: Verifying Integrity of Deep Neural Networks through Sensitive-Sample Fingerprinting
Deep learning has become popular, and numerous cloud-based services are
provided to help customers develop and deploy deep learning applications.
Meanwhile, various attack techniques have also been discovered to stealthily
compromise the model's integrity. When a cloud customer deploys a deep learning
model in the cloud and serves it to end-users, it is important for him to be
able to verify that the deployed model has not been tampered with, and the
model's integrity is protected.
We propose a new low-cost and self-served methodology for customers to verify
that the model deployed in the cloud is intact, while having only black-box
access (e.g., via APIs) to the deployed model. Customers can detect arbitrary
changes to their deep learning models. Specifically, we define
\texttt{Sensitive-Sample} fingerprints, which are a small set of transformed
inputs that make the model outputs sensitive to the model's parameters. Even
small weight changes can be clearly reflected in the model outputs, and
observed by the customer. Our experiments on different types of model integrity
attacks show that we can detect model integrity breaches with high accuracy
(99\%) and low overhead (10 black-box model accesses)
Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
Deep neural networks (DNN) are known to be vulnerable to adversarial attacks.
Numerous efforts either try to patch weaknesses in trained models, or try to
make it difficult or costly to compute adversarial examples that exploit them.
In our work, we explore a new "honeypot" approach to protect DNN models. We
intentionally inject trapdoors, honeypot weaknesses in the classification
manifold that attract attackers searching for adversarial examples. Attackers'
optimization algorithms gravitate towards trapdoors, leading them to produce
attacks similar to trapdoors in the feature space. Our defense then identifies
attacks by comparing neuron activation signatures of inputs to those of
trapdoors. In this paper, we introduce trapdoors and describe an implementation
of a trapdoor-enabled defense. First, we analytically prove that trapdoors
shape the computation of adversarial attacks so that attack inputs will have
feature representations very similar to those of trapdoors. Second, we
experimentally show that trapdoor-protected models can detect, with high
accuracy, adversarial examples generated by state-of-the-art attacks (PGD,
optimization-based CW, Elastic Net, BPDA), with negligible impact on normal
classification. These results generalize across classification domains,
including image, facial, and traffic-sign recognition. We also present
significant results measuring trapdoors' robustness against customized adaptive
attacks (countermeasures)
- …