889 research outputs found
Adversarial Examples Might be Avoidable: The Role of Data Concentration in Adversarial Robustness
The susceptibility of modern machine learning classifiers to adversarial
examples has motivated theoretical results suggesting that these might be
unavoidable. However, these results can be too general to be applicable to
natural data distributions. Indeed, humans are quite robust for tasks involving
vision. This apparent conflict motivates a deeper dive into the question: Are
adversarial examples truly unavoidable? In this work, we theoretically
demonstrate that a key property of the data distribution -- concentration on
small-volume subsets of the input space -- determines whether a robust
classifier exists. We further demonstrate that, for a data distribution
concentrated on a union of low-dimensional linear subspaces, exploiting data
structure naturally leads to classifiers that enjoy good robustness guarantees,
improving upon methods for provable certification in certain regimes.Comment: Accepted to Neural Information Processing Systems (NeurIPS) 202
Scaling in Depth: Unlocking Robustness Certification on ImageNet
Despite the promise of Lipschitz-based methods for provably-robust deep
learning with deterministic guarantees, current state-of-the-art results are
limited to feed-forward Convolutional Networks (ConvNets) on low-dimensional
data, such as CIFAR-10. This paper investigates strategies for expanding
certifiably robust training to larger, deeper models. A key challenge in
certifying deep networks is efficient calculation of the Lipschitz bound for
residual blocks found in ResNet and ViT architectures. We show that fast ways
of bounding the Lipschitz constant for conventional ResNets are loose, and show
how to address this by designing a new residual block, leading to the
\emph{Linear ResNet} (LiResNet) architecture. We then introduce \emph{Efficient
Margin MAximization} (EMMA), a loss function that stabilizes robust training by
simultaneously penalizing worst-case adversarial examples from \emph{all}
classes. Together, these contributions yield new \emph{state-of-the-art} robust
accuracy on CIFAR-10/100 and Tiny-ImageNet under perturbations.
Moreover, for the first time, we are able to scale up fast deterministic
robustness guarantees to ImageNet, demonstrating that this approach to robust
learning can be applied to real-world applications.
We release our code on Github: \url{https://github.com/klasleino/gloro}
- …