11 research outputs found
On the Adversarial Robustness of Vision Transformers
Following the success in advancing natural language processing and
understanding, transformers are expected to bring revolutionary changes to
computer vision. This work provides the first and comprehensive study on the
robustness of vision transformers (ViTs) against adversarial perturbations.
Tested on various white-box and transfer attack settings, we find that ViTs
possess better adversarial robustness when compared with convolutional neural
networks (CNNs). This observation also holds for certified robustness. We
summarize the following main observations contributing to the improved
robustness of ViTs:
1) Features learned by ViTs contain less low-level information and are more
generalizable, which contributes to superior robustness against adversarial
perturbations.
2) Introducing convolutional or tokens-to-token blocks for learning low-level
features in ViTs can improve classification accuracy but at the cost of
adversarial robustness.
3) Increasing the proportion of transformers in the model structure (when the
model consists of both transformer and CNN blocks) leads to better robustness.
But for a pure transformer model, simply increasing the size or adding layers
cannot guarantee a similar effect.
4) Pre-training on larger datasets does not significantly improve adversarial
robustness though it is critical for training ViTs.
5) Adversarial training is also applicable to ViT for training robust models.
Furthermore, feature visualization and frequency analysis are conducted for
explanation. The results show that ViTs are less sensitive to high-frequency
perturbations than CNNs and there is a high correlation between how well the
model learns low-level features and its robustness against different
frequency-based perturbations
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data
"Effective robustness" measures the extra out-of-distribution (OOD)
robustness beyond what can be predicted from the in-distribution (ID)
performance. Existing effective robustness evaluations typically use a single
test set such as ImageNet to evaluate the ID accuracy. This becomes problematic
when evaluating models trained on different data distributions, e.g., comparing
models trained on ImageNet vs. zero-shot language-image pre-trained models
trained on LAION. In this paper, we propose a new evaluation metric to evaluate
and compare the effective robustness of models trained on different data. To do
this, we control for the accuracy on multiple ID test sets that cover the
training distributions for all the evaluated models. Our new evaluation metric
provides a better estimate of effective robustness when there are models with
different training data. It may also explain the surprising effective
robustness gains of zero-shot CLIP-like models exhibited in prior works that
used ImageNet as the only ID test set, while the gains diminish under our new
evaluation. Additional artifacts including interactive visualizations are
provided at https://shizhouxing.github.io/effective-robustness.Comment: NeurIPS 202
Red Teaming Language Model Detectors with Language Models
The prevalence and strong capability of large language models (LLMs) present
significant safety and ethical risks if exploited by malicious users. To
prevent the potentially deceptive usage of LLMs, recent works have proposed
algorithms to detect LLM-generated text and protect LLMs. In this paper, we
investigate the robustness and reliability of these LLM detectors under
adversarial attacks. We study two types of attack strategies: 1) replacing
certain words in an LLM's output with their synonyms given the context; 2)
automatically searching for an instructional prompt to alter the writing style
of the generation. In both strategies, we leverage an auxiliary LLM to generate
the word replacements or the instructional prompt. Different from previous
works, we consider a challenging setting where the auxiliary LLM can also be
protected by a detector. Experiments reveal that our attacks effectively
compromise the performance of all detectors in the study with plausible
generations, underscoring the urgent need to improve the robustness of
LLM-generated text detection systems.Comment: Preprint. Accepted by TAC