11,915 research outputs found
Principal Component Properties of Adversarial Samples
Deep Neural Networks for image classification have been found to be
vulnerable to adversarial samples, which consist of sub-perceptual noise added
to a benign image that can easily fool trained neural networks, posing a
significant risk to their commercial deployment. In this work, we analyze
adversarial samples through the lens of their contributions to the principal
components of each image, which is different than prior works in which authors
performed PCA on the entire dataset. We investigate a number of
state-of-the-art deep neural networks trained on ImageNet as well as several
attacks for each of the networks. Our results demonstrate empirically that
adversarial samples across several attacks have similar properties in their
contributions to the principal components of neural network inputs. We propose
a new metric for neural networks to measure their robustness to adversarial
samples, termed the (k,p) point. We utilize this metric to achieve 93.36%
accuracy in detecting adversarial samples independent of architecture and
attack type for models trained on ImageNet
Classification regions of deep neural networks
The goal of this paper is to analyze the geometric properties of deep neural
network classifiers in the input space. We specifically study the topology of
classification regions created by deep networks, as well as their associated
decision boundary. Through a systematic empirical investigation, we show that
state-of-the-art deep nets learn connected classification regions, and that the
decision boundary in the vicinity of datapoints is flat along most directions.
We further draw an essential connection between two seemingly unrelated
properties of deep networks: their sensitivity to additive perturbations in the
inputs, and the curvature of their decision boundary. The directions where the
decision boundary is curved in fact remarkably characterize the directions to
which the classifier is the most vulnerable. We finally leverage a fundamental
asymmetry in the curvature of the decision boundary of deep nets, and propose a
method to discriminate between original images, and images perturbed with small
adversarial examples. We show the effectiveness of this purely geometric
approach for detecting small adversarial perturbations in images, and for
recovering the labels of perturbed images
Parametrization and generation of geological models with generative adversarial networks
One of the main challenges in the parametrization of geological models is the
ability to capture complex geological structures often observed in the
subsurface. In recent years, generative adversarial networks (GAN) were
proposed as an efficient method for the generation and parametrization of
complex data, showing state-of-the-art performances in challenging computer
vision tasks such as reproducing natural images (handwritten digits, human
faces, etc.). In this work, we study the application of Wasserstein GAN for the
parametrization of geological models. The effectiveness of the method is
assessed for uncertainty propagation tasks using several test cases involving
different permeability patterns and subsurface flow problems. Results show that
GANs are able to generate samples that preserve the multipoint statistical
features of the geological models both visually and quantitatively. The
generated samples reproduce both the geological structures and the flow
statistics of the reference geology
Optimizing the Latent Space of Generative Networks
Generative Adversarial Networks (GANs) have achieved remarkable results in
the task of generating realistic natural images. In most successful
applications, GAN models share two common aspects: solving a challenging saddle
point optimization problem, interpreted as an adversarial game between a
generator and a discriminator functions; and parameterizing the generator and
the discriminator as deep convolutional neural networks. The goal of this paper
is to disentangle the contribution of these two factors to the success of GANs.
In particular, we introduce Generative Latent Optimization (GLO), a framework
to train deep convolutional generators using simple reconstruction losses.
Throughout a variety of experiments, we show that GLO enjoys many of the
desirable properties of GANs: synthesizing visually-appealing samples,
interpolating meaningfully between samples, and performing linear arithmetic
with noise vectors; all of this without the adversarial optimization scheme
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Neural networks are known to be vulnerable to adversarial examples: inputs
that are close to natural inputs but classified incorrectly. In order to better
understand the space of adversarial examples, we survey ten recent proposals
that are designed for detection and compare their efficacy. We show that all
can be defeated by constructing new loss functions. We conclude that
adversarial examples are significantly harder to detect than previously
appreciated, and the properties believed to be intrinsic to adversarial
examples are in fact not. Finally, we propose several simple guidelines for
evaluating future proposed defenses
On GANs and GMMs
A longstanding problem in machine learning is to find unsupervised methods
that can learn the statistical structure of high dimensional signals. In recent
years, GANs have gained much attention as a possible solution to the problem,
and in particular have shown the ability to generate remarkably realistic high
resolution sampled images. At the same time, many authors have pointed out that
GANs may fail to model the full distribution ("mode collapse") and that using
the learned models for anything other than generating samples may be very
difficult. In this paper, we examine the utility of GANs in learning
statistical models of images by comparing them to perhaps the simplest
statistical model, the Gaussian Mixture Model. First, we present a simple
method to evaluate generative models based on relative proportions of samples
that fall into predetermined bins. Unlike previous automatic methods for
evaluating models, our method does not rely on an additional neural network nor
does it require approximating intractable computations. Second, we compare the
performance of GANs to GMMs trained on the same datasets. While GMMs have
previously been shown to be successful in modeling small patches of images, we
show how to train them on full sized images despite the high dimensionality.
Our results show that GMMs can generate realistic samples (although less sharp
than those of GANs) but also capture the full distribution, which GANs fail to
do. Furthermore, GMMs allow efficient inference and explicit representation of
the underlying statistical structure. Finally, we discuss how GMMs can be used
to generate sharp images.Comment: Accepted to NIPS 201
Why is the Mahalanobis Distance Effective for Anomaly Detection?
The Mahalanobis distance-based confidence score, a recently proposed anomaly
detection method for pre-trained neural classifiers, achieves state-of-the-art
performance on both out-of-distribution (OoD) and adversarial examples
detection. This work analyzes why this method exhibits such strong performance
in practical settings while imposing an implausible assumption; namely, that
class conditional distributions of pre-trained features have tied covariance.
Although the Mahalanobis distance-based method is claimed to be motivated by
classification prediction confidence, we find that its superior performance
stems from information not useful for classification. This suggests that the
reason the Mahalanobis confidence score works so well is mistaken, and makes
use of different information from ODIN, another popular OoD detection method
based on prediction confidence. This perspective motivates us to combine these
two methods, and the combined detector exhibits improved performance and
robustness. These findings provide insight into the behavior of neural
classifiers in response to anomalous inputs
A Generative Model for Sampling High-Performance and Diverse Weights for Neural Networks
Recent work on mode connectivity in the loss landscape of deep neural
networks has demonstrated that the locus of (sub-)optimal weight vectors lies
on continuous paths. In this work, we train a neural network that serves as a
hypernetwork, mapping a latent vector into high-performance (low-loss) weight
vectors, generalizing recent findings of mode connectivity to higher
dimensional manifolds. We formulate the training objective as a compromise
between accuracy and diversity, where the diversity takes into account trivial
symmetry transformations of the target network. We demonstrate how to reduce
the number of parameters in the hypernetwork by parameter sharing. Once
learned, the hypernetwork allows for a computationally efficient, ancestral
sampling of neural network weights, which we recruit to form large ensembles.
The improvement in classification accuracy obtained by this ensembling
indicates that the generated manifold extends in dimensions other than
directions implied by trivial symmetries. For computational efficiency, we
distill an ensemble into a single classifier while retaining generalization.Comment: arXiv admin note: substantial text overlap with arXiv:1801.0195
Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
Active subspace is a model reduction method widely used in the uncertainty
quantification community. In this paper, we propose analyzing the internal
structure and vulnerability and deep neural networks using active subspace.
Firstly, we employ the active subspace to measure the number of "active
neurons" at each intermediate layer and reduce the number of neurons from
several thousands to several dozens. This motivates us to change the network
structure and to develop a new and more compact network, referred to as
{ASNet}, that has significantly fewer model parameters. Secondly, we propose
analyzing the vulnerability of a neural network using active subspace and
finding an additive universal adversarial attack vector that can misclassify a
dataset with a high probability. Our experiments on CIFAR-10 show that ASNet
can achieve 23.98 parameter and 7.30 flops reduction. The
universal active subspace attack vector can achieve around 20% higher attack
ratio compared with the existing approach in all of our numerical experiments.
The PyTorch codes for this paper are available online
Adversarial Attacks on Deep-Learning Based Radio Signal Classification
Deep learning (DL), despite its enormous success in many computer vision and
language processing applications, is exceedingly vulnerable to adversarial
attacks. We consider the use of DL for radio signal (modulation) classification
tasks, and present practical methods for the crafting of white-box and
universal black-box adversarial attacks in that application. We show that these
attacks can considerably reduce the classification performance, with extremely
small perturbations of the input. In particular, these attacks are
significantly more powerful than classical jamming attacks, which raises
significant security and robustness concerns in the use of DL-based algorithms
for the wireless physical layer.Comment: 4 page
- …