8,109 research outputs found
Generative Adversarial Trainer: Defense to Adversarial Perturbations with GAN
We propose a novel technique to make neural network robust to adversarial
examples using a generative adversarial network. We alternately train both
classifier and generator networks. The generator network generates an
adversarial perturbation that can easily fool the classifier network by using a
gradient of each image. Simultaneously, the classifier network is trained to
classify correctly both original and adversarial images generated by the
generator. These procedures help the classifier network to become more robust
to adversarial perturbations. Furthermore, our adversarial training framework
efficiently reduces overfitting and outperforms other regularization methods
such as Dropout. We applied our method to supervised learning for CIFAR
datasets, and experimantal results show that our method significantly lowers
the generalization error of the network. To the best of our knowledge, this is
the first method which uses GAN to improve supervised learning
Adversarial Dropout for Supervised and Semi-supervised Learning
Recently, the training with adversarial examples, which are generated by
adding a small but worst-case perturbation on input examples, has been proved
to improve generalization performance of neural networks. In contrast to the
individually biased inputs to enhance the generality, this paper introduces
adversarial dropout, which is a minimal set of dropouts that maximize the
divergence between the outputs from the network with the dropouts and the
training supervisions. The identified adversarial dropout are used to
reconfigure the neural network to train, and we demonstrated that training on
the reconfigured sub-network improves the generalization performance of
supervised and semi-supervised learning tasks on MNIST and CIFAR-10. We
analyzed the trained model to reason the performance improvement, and we found
that adversarial dropout increases the sparsity of neural networks more than
the standard dropout does.Comment: submitted to AAAI-1
Regularizing deep networks using efficient layerwise adversarial training
Adversarial training has been shown to regularize deep neural networks in
addition to increasing their robustness to adversarial examples. However, its
impact on very deep state of the art networks has not been fully investigated.
In this paper, we present an efficient approach to perform adversarial training
by perturbing intermediate layer activations and study the use of such
perturbations as a regularizer during training. We use these perturbations to
train very deep models such as ResNets and show improvement in performance both
on adversarial and original test data. Our experiments highlight the benefits
of perturbing intermediate layer activations compared to perturbing only the
inputs. The results on CIFAR-10 and CIFAR-100 datasets show the merits of the
proposed adversarial training approach. Additional results on WideResNets show
that our approach provides significant improvement in classification accuracy
for a given base model, outperforming dropout and other base models of larger
size.Comment: Published at the Thirty-Second AAAI Conference on Artificial
Intelligence (AAAI-18). Official link:
https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/1663
Learning Robust Representations of Text
Deep neural networks have achieved remarkable results across many language
processing tasks, however these methods are highly sensitive to noise and
adversarial attacks. We present a regularization based method for limiting
network sensitivity to its inputs, inspired by ideas from computer vision, thus
learning models that are more robust. Empirical evaluation over a range of
sentiment datasets with a convolutional neural network shows that, compared to
a baseline model and the dropout method, our method achieves superior
performance over noisy inputs and out-of-domain data.Comment: 5 pages with 2 pages reference, 2 tables, 1 figur
Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
Despite being impactful on a variety of problems and applications, the
generative adversarial nets (GANs) are remarkably difficult to train. This
issue is formally analyzed by \cite{arjovsky2017towards}, who also propose an
alternative direction to avoid the caveats in the minmax two-player training of
GANs. The corresponding algorithm, called Wasserstein GAN (WGAN), hinges on the
1-Lipschitz continuity of the discriminator. In this paper, we propose a novel
approach to enforcing the Lipschitz continuity in the training procedure of
WGANs. Our approach seamlessly connects WGAN with one of the recent
semi-supervised learning methods. As a result, it gives rise to not only better
photo-realistic samples than the previous methods but also state-of-the-art
semi-supervised learning results. In particular, our approach gives rise to the
inception score of more than 5.0 with only 1,000 CIFAR-10 images and is the
first that exceeds the accuracy of 90% on the CIFAR-10 dataset using only 4,000
labeled images, to the best of our knowledge.Comment: Accepted as a conference paper in International Conference on
Learning Representation(ICLR). Xiang Wei and Boqing Gong contributed equally
in this wor
- …