397 research outputs found
Fooling a Real Car with Adversarial Traffic Signs
The attacks on the neural-network-based classifiers using adversarial images
have gained a lot of attention recently. An adversary can purposely generate an
image that is indistinguishable from a innocent image for a human being but is
incorrectly classified by the neural networks. The adversarial images do not
need to be tuned to a particular architecture of the classifier - an image that
fools one network can fool another one with a certain success rate.The
published works mostly concentrate on the use of modified image files for
attacks against the classifiers trained on the model databases. Although there
exists a general understanding that such attacks can be carried in the real
world as well, the works considering the real-world attacks are scarce.
Moreover, to the best of our knowledge, there have been no reports on the
attacks against real production-grade image classification systems.In our work
we present a robust pipeline for reproducible production of adversarial traffic
signs that can fool a wide range of classifiers, both open-source and
production-grade in the real world. The efficiency of the attacks was checked
both with the neural-network-based classifiers and legacy computer vision
systems. Most of the attacks have been performed in the black-box mode, e.g.
the adversarial signs produced for a particular classifier were used to attack
a variety of other classifiers. The efficiency was confirmed in drive-by
experiments with a production-grade traffic sign recognition systems of a real
car
Securing Connected & Autonomous Vehicles: Challenges Posed by Adversarial Machine Learning and The Way Forward
Connected and autonomous vehicles (CAVs) will form the backbone of future
next-generation intelligent transportation systems (ITS) providing travel
comfort, road safety, along with a number of value-added services. Such a
transformation---which will be fuelled by concomitant advances in technologies
for machine learning (ML) and wireless communications---will enable a future
vehicular ecosystem that is better featured and more efficient. However, there
are lurking security problems related to the use of ML in such a critical
setting where an incorrect ML decision may not only be a nuisance but can lead
to loss of precious lives. In this paper, we present an in-depth overview of
the various challenges associated with the application of ML in vehicular
networks. In addition, we formulate the ML pipeline of CAVs and present various
potential security issues associated with the adoption of ML methods. In
particular, we focus on the perspective of adversarial ML attacks on CAVs and
outline a solution to defend against adversarial attacks in multiple settings
DARTS: Deceiving Autonomous Cars with Toxic Signs
Sign recognition is an integral part of autonomous cars. Any
misclassification of traffic signs can potentially lead to a multitude of
disastrous consequences, ranging from a life-threatening accident to even a
large-scale interruption of transportation services relying on autonomous cars.
In this paper, we propose and examine security attacks against sign recognition
systems for Deceiving Autonomous caRs with Toxic Signs (we call the proposed
attacks DARTS). In particular, we introduce two novel methods to create these
toxic signs. First, we propose Out-of-Distribution attacks, which expand the
scope of adversarial examples by enabling the adversary to generate these
starting from an arbitrary point in the image space compared to prior attacks
which are restricted to existing training/test data (In-Distribution). Second,
we present the Lenticular Printing attack, which relies on an optical
phenomenon to deceive the traffic sign recognition system. We extensively
evaluate the effectiveness of the proposed attacks in both virtual and
real-world settings and consider both white-box and black-box threat models.
Our results demonstrate that the proposed attacks are successful under both
settings and threat models. We further show that Out-of-Distribution attacks
can outperform In-Distribution attacks on classifiers defended using the
adversarial training defense, exposing a new attack vector for these defenses.Comment: Submitted to ACM CCS 2018; Extended version of [1801.02780] Rogue
Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logo
Standard detectors aren't (currently) fooled by physical adversarial stop signs
An adversarial example is an example that has been adjusted to produce the
wrong label when presented to a system at test time. If adversarial examples
existed that could fool a detector, they could be used to (for example) wreak
havoc on roads populated with smart vehicles. Recently, we described our
difficulties creating physical adversarial stop signs that fool a detector.
More recently, Evtimov et al. produced a physical adversarial stop sign that
fools a proxy model of a detector. In this paper, we show that these physical
adversarial stop signs do not fool two standard detectors (YOLO and Faster
RCNN) in standard configuration. Evtimov et al.'s construction relies on a crop
of the image to the stop sign; this crop is then resized and presented to a
classifier. We argue that the cropping and resizing procedure largely
eliminates the effects of rescaling and of view angle. Whether an adversarial
attack is robust under rescaling and change of view direction remains moot. We
argue that attacking a classifier is very different from attacking a detector,
and that the structure of detectors - which must search for their own bounding
box, and which cannot estimate that box very accurately - likely makes it
difficult to make adversarial patterns. Finally, an adversarial pattern on a
physical object that could fool a detector would have to be adversarial in the
face of a wide family of parametric distortions (scale; view angle; box shift
inside the detector; illumination; and so on). Such a pattern would be of great
theoretical and practical interest. There is currently no evidence that such
patterns exist.Comment: Follow up for previous adversarial stop sign pape
Robust Physical-World Attacks on Deep Learning Models
Recent studies show that the state-of-the-art deep neural networks (DNNs) are
vulnerable to adversarial examples, resulting from small-magnitude
perturbations added to the input. Given that that emerging physical systems are
using DNNs in safety-critical situations, adversarial examples could mislead
these systems and cause dangerous situations.Therefore, understanding
adversarial examples in the physical world is an important step towards
developing resilient learning algorithms. We propose a general attack
algorithm,Robust Physical Perturbations (RP2), to generate robust visual
adversarial perturbations under different physical conditions. Using the
real-world case of road sign classification, we show that adversarial examples
generated using RP2 achieve high targeted misclassification rates against
standard-architecture road sign classifiers in the physical world under various
environmental conditions, including viewpoints. Due to the current lack of a
standardized testing method, we propose a two-stage evaluation methodology for
robust physical adversarial examples consisting of lab and field tests. Using
this methodology, we evaluate the efficacy of physical adversarial
manipulations on real objects. Witha perturbation in the form of only black and
white stickers,we attack a real stop sign, causing targeted misclassification
in 100% of the images obtained in lab settings, and in 84.8%of the captured
video frames obtained on a moving vehicle(field test) for the target
classifier.Comment: Accepted to CVPR 201
PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks
Deep learning systems have become ubiquitous in many aspects of our lives.
Unfortunately, it has been shown that such systems are vulnerable to
adversarial attacks, making them prone to potential unlawful uses. Designing
deep neural networks that are robust to adversarial attacks is a fundamental
step in making such systems safer and deployable in a broader variety of
applications (e.g. autonomous driving), but more importantly is a necessary
step to design novel and more advanced architectures built on new computational
paradigms rather than marginally building on the existing ones. In this paper
we introduce PeerNets, a novel family of convolutional networks alternating
classical Euclidean convolutions with graph convolutions to harness information
from a graph of peer samples. This results in a form of non-local forward
propagation in the model, where latent features are conditioned on the global
structure induced by the graph, that is up to 3 times more robust to a variety
of white- and black-box adversarial attacks compared to conventional
architectures with almost no drop in accuracy
Building Robust Deep Neural Networks for Road Sign Detection
Deep Neural Networks are built to generalize outside of training set in mind
by using techniques such as regularization, early stopping and dropout. But
considerations to make them more resilient to adversarial examples are rarely
taken. As deep neural networks become more prevalent in mission-critical and
real-time systems, miscreants start to attack them by intentionally making deep
neural networks to misclassify an object of one type to be seen as another
type. This can be catastrophic in some scenarios where the classification of a
deep neural network can lead to a fatal decision by a machine. In this work, we
used GTSRB dataset to craft adversarial samples by Fast Gradient Sign Method
and Jacobian Saliency Method, used those crafted adversarial samples to attack
another Deep Convolutional Neural Network and built the attacked network to be
more resilient against adversarial attacks by making it more robust by
Defensive Distillation and Adversarial Trainin
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
It has been shown that most machine learning algorithms are susceptible to
adversarial perturbations. Slightly perturbing an image in a carefully chosen
direction in the image space may cause a trained neural network model to
misclassify it. Recently, it was shown that physical adversarial examples
exist: printing perturbed images then taking pictures of them would still
result in misclassification. This raises security and safety concerns.
However, these experiments ignore a crucial property of physical objects: the
camera can view objects from different distances and at different angles. In
this paper, we show experiments that suggest that current constructions of
physical adversarial examples do not disrupt object detection from a moving
platform. Instead, a trained neural network classifies most of the pictures
taken from different distances and angles of a perturbed image correctly. We
believe this is because the adversarial property of the perturbation is
sensitive to the scale at which the perturbed picture is viewed, so (for
example) an autonomous car will misclassify a stop sign only from a small range
of distances.
Our work raises an important question: can one construct examples that are
adversarial for many or most viewing conditions? If so, the construction should
offer very significant insights into the internal representation of patterns by
deep networks. If not, there is a good prospect that adversarial examples can
be reduced to a curiosity with little practical impact.Comment: Accepted to CVPR 2017, Spotlight Oral Worksho
LaVAN: Localized and Visible Adversarial Noise
Most works on adversarial examples for deep-learning based image classifiers
use noise that, while small, covers the entire image. We explore the case where
the noise is allowed to be visible but confined to a small, localized patch of
the image, without covering any of the main object(s) in the image. We show
that it is possible to generate localized adversarial noises that cover only 2%
of the pixels in the image, none of them over the main object, and that are
transferable across images and locations, and successfully fool a
state-of-the-art Inception v3 model with very high success rates
Universal Physical Camouflage Attacks on Object Detectors
In this paper, we study physical adversarial attacks on object detectors in
the wild. Previous works mostly craft instance-dependent perturbations only for
rigid or planar objects. To this end, we propose to learn an adversarial
pattern to effectively attack all instances belonging to the same object
category, referred to as Universal Physical Camouflage Attack (UPC).
Concretely, UPC crafts camouflage by jointly fooling the region proposal
network, as well as misleading the classifier and the regressor to output
errors. In order to make UPC effective for non-rigid or non-planar objects, we
introduce a set of transformations for mimicking deformable properties. We
additionally impose optimization constraint to make generated patterns look
natural to human observers. To fairly evaluate the effectiveness of different
physical-world attacks, we present the first standardized virtual database,
AttackScenes, which simulates the real 3D world in a controllable and
reproducible environment. Extensive experiments suggest the superiority of our
proposed UPC compared with existing physical adversarial attackers not only in
virtual environments (AttackScenes), but also in real-world physical
environments. Code and dataset are available at
https://mesunhlf.github.io/index_physical.html.Comment: CVPR 2020; codes, models, and demos are available at
https://mesunhlf.github.io/index_physical.htm
- …