51 research outputs found
Investigating Catastrophic Overfitting in Fast Adversarial Training: A Self-fitting Perspective
Although fast adversarial training provides an efficient approach for
building robust networks, it may suffer from a serious problem known as
catastrophic overfitting (CO), where multi-step robust accuracy suddenly
collapses to zero. In this paper, we for the first time decouple single-step
adversarial examples into data-information and self-information, which reveals
an interesting phenomenon called "self-fitting". Self-fitting, i.e., the
network learns the self-information embedded in single-step perturbations,
naturally leads to the occurrence of CO. When self-fitting occurs, the network
experiences an obvious "channel differentiation" phenomenon that some
convolution channels accounting for recognizing self-information become
dominant, while others for data-information are suppressed. In this way, the
network can only recognize images with sufficient self-information and loses
generalization ability to other types of data. Based on self-fitting, we
provide new insights into the existing methods to mitigate CO and extend CO to
multi-step adversarial training. Our findings reveal a self-learning mechanism
in adversarial training and open up new perspectives for suppressing different
kinds of information to mitigate CO.Comment: Comment: The camera-ready version (accepted at CVPR Workshop of
Adversarial Machine Learning on Computer Vision: Art of Robustness, 2023
QueryNet: Attack by Multi-Identity Surrogates
Deep Neural Networks (DNNs) are acknowledged as vulnerable to adversarial
attacks, while the existing black-box attacks require extensive queries on the
victim DNN to achieve high success rates. For query-efficiency, surrogate
models of the victim are used to generate transferable Adversarial Examples
(AEs) because of their Gradient Similarity (GS), i.e., surrogates' attack
gradients are similar to the victim's ones. However, it is generally neglected
to exploit their similarity on outputs, namely the Prediction Similarity (PS),
to filter out inefficient queries by surrogates without querying the victim. To
jointly utilize and also optimize surrogates' GS and PS, we develop QueryNet, a
unified attack framework that can significantly reduce queries. QueryNet
creatively attacks by multi-identity surrogates, i.e., crafts several AEs for
one sample by different surrogates, and also uses surrogates to decide on the
most promising AE for the query. After that, the victim's query feedback is
accumulated to optimize not only surrogates' parameters but also their
architectures, enhancing both the GS and the PS. Although QueryNet has no
access to pre-trained surrogates' prior, it reduces queries by averagely about
an order of magnitude compared to alternatives within an acceptable time,
according to our comprehensive experiments: 11 victims (including two
commercial models) on MNIST/CIFAR10/ImageNet, allowing only 8-bit image
queries, and no access to the victim's training data. The code is available at
https://github.com/Sizhe-Chen/QueryNet.Comment: QueryNet reduces queries by about an order of magnitude against SOTA
black-box attack
Going Far Boosts Attack Transferability, but Do Not Do It
Deep Neural Networks (DNNs) could be easily fooled by Adversarial Examples
(AEs) with an imperceptible difference to original ones in human eyes. Also,
the AEs from attacking one surrogate DNN tend to cheat other black-box DNNs as
well, i.e., the attack transferability. Existing works reveal that adopting
certain optimization algorithms in attack improves transferability, but the
underlying reasons have not been thoroughly studied. In this paper, we
investigate the impacts of optimization on attack transferability by
comprehensive experiments concerning 7 optimization algorithms, 4 surrogates,
and 9 black-box models. Through the thorough empirical analysis from three
perspectives, we surprisingly find that the varied transferability of AEs from
optimization algorithms is strongly related to the corresponding Root Mean
Square Error (RMSE) from their original samples. On such a basis, one could
simply approach high transferability by attacking until RMSE decreases, which
motives us to propose a LArge RMSE Attack (LARA). Although LARA significantly
improves transferability by 20%, it is insufficient to exploit the
vulnerability of DNNs, leading to a natural urge that the strength of all
attacks should be measured by both the widely used bound and the
RMSE addressed in this paper, so that tricky enhancement of transferability
would be avoided
Unifying Gradients to Improve Real-world Robustness for Deep Networks
The wide application of deep neural networks (DNNs) demands an increasing
amount of attention to their real-world robustness, i.e., whether a DNN resists
black-box adversarial attacks, among which score-based query attacks (SQAs) are
most threatening since they can effectively hurt a victim network with the only
access to model outputs. Defending against SQAs requires a slight but artful
variation of outputs due to the service purpose for users, who share the same
output information with SQAs. In this paper, we propose a real-world defense by
Unifying Gradients (UniG) of different data so that SQAs could only probe a
much weaker attack direction that is similar for different samples. Since such
universal attack perturbations have been validated as less aggressive than the
input-specific perturbations, UniG protects real-world DNNs by indicating
attackers a twisted and less informative attack direction. We implement UniG
efficiently by a Hadamard product module which is plug-and-play. According to
extensive experiments on 5 SQAs, 2 adaptive attacks and 7 defense baselines,
UniG significantly improves real-world robustness without hurting clean
accuracy on CIFAR10 and ImageNet. For instance, UniG maintains a model of
77.80% accuracy under 2500-query Square attack while the state-of-the-art
adversarially-trained model only has 67.34% on CIFAR10. Simultaneously, UniG
outperforms all compared baselines in terms of clean accuracy and achieves the
smallest modification of the model output. The code is released at
https://github.com/snowien/UniG-pytorch
The Expression Levels of XLF and Mutant P53 Are Inversely Correlated in Head and Neck Cancer Cells.
XRCC4-like factor (XLF), also known as Cernunnos, is a protein encoded by the human NHEJ1 gene and an important repair factor for DNA double-strand breaks. In this study, we have found that XLF is over-expressed in HPV(+) versus HPV(-) head and neck squamous cell carcinoma (HNSCC) and significantly down-regulated in the HNSCC cell lines expressing high level of mutant p53 protein versus those cell lines harboring wild-type TP53 gene with low p53 protein expression. We have also demonstrated that Werner syndrome protein (WRN), a member of the NHEJ repair pathway, binds to both mutant p53 protein and NHEJ1 gene promoter, and siRNA knockdown of WRN leads to the inhibition of XLF expression in the HNSCC cells. Collectively, these findings suggest that WRN and p53 are involved in the regulation of XLF expression and the activity of WRN might be affected by mutant p53 protein in the HNSCC cells with aberrant TP53 gene mutations, due to the interaction of mutant p53 with WRN. As a result, the expression of XLF in these cancer cells is significantly suppressed. Our study also suggests that XLF is over-expressed in HPV(+) HNSCC with low expression of wild type p53, and might serve as a potential biomarker for HPV(+) HNSCC. Further studies are warranted to investigate the mechanisms underlying the interactive role of WRN and XLF in NHEJ repair pathway
- …