119 research outputs found
Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness
Evaluating the robustness of a defense model is a challenging task in
adversarial robustness research. Obfuscated gradients, a type of gradient
masking, have previously been found to exist in many defense methods and cause
a false signal of robustness. In this paper, we identify a more subtle
situation called Imbalanced Gradients that can also cause overestimated
adversarial robustness. The phenomenon of imbalanced gradients occurs when the
gradient of one term of the margin loss dominates and pushes the attack towards
to a suboptimal direction. To exploit imbalanced gradients, we formulate a
Margin Decomposition (MD) attack that decomposes a margin loss into individual
terms and then explores the attackability of these terms separately via a
two-stage process. We also propose a MultiTargeted and an ensemble version of
our MD attack. By investigating 17 defense models proposed since 2018, we find
that 6 models are susceptible to imbalanced gradients and our MD attack can
decrease their robustness evaluated by the best baseline standalone attack by
another 2%. We also provide an in-depth analysis of the likely causes of
imbalanced gradients and effective countermeasures.Comment: 19 pages, 7 figue
Recommended from our members
LRID: A new metric of multi-class imbalance degree based on likelihood-ratio test
In this paper, we introduce a new likelihood ratio imbalance degree (LRID) to measure the class-imbalance extent of multi-class data. Imbalance ratio (IR) is usually used to measure class-imbalance extent in imbalanced learning problems. However, IR cannot capture the detailed information in the class distribution of multi-class data, because it only utilises the information of the largest majority class and the smallest minority class. Imbalance degree (ID) has been proposed to solve the problem of IR for multi-class data. However, we note that improper use of distance metric in ID can have harmful effect on the results. In addition, ID assumes that data with more minority classes are more imbalanced than data with less minority classes, which is not always true in practice. Thus ID cannot provide reliable measurement when the assumption is violated. In this paper, we propose a new metric based on the likelihood-ratio test, LRID, to provide a more reliable measurement of class-imbalance extent for multi-class data. Experiments on both simulated and real data show that LRID is competitive with IR and ID, and can reduce the negative correlation with F1 scores by up to 0.55
- …