33 research outputs found
Echoes: Unsupervised Debiasing via Pseudo-bias Labeling in an Echo Chamber
Neural networks often learn spurious correlations when exposed to biased
training data, leading to poor performance on out-of-distribution data. A
biased dataset can be divided, according to biased features, into bias-aligned
samples (i.e., with biased features) and bias-conflicting samples (i.e.,
without biased features). Recent debiasing works typically assume that no bias
label is available during the training phase, as obtaining such information is
challenging and labor-intensive. Following this unsupervised assumption,
existing methods usually train two models: a biased model specialized to learn
biased features and a target model that uses information from the biased model
for debiasing. This paper first presents experimental analyses revealing that
the existing biased models overfit to bias-conflicting samples in the training
data, which negatively impacts the debiasing performance of the target models.
To address this issue, we propose a straightforward and effective method called
Echoes, which trains a biased model and a target model with a different
strategy. We construct an "echo chamber" environment by reducing the weights of
samples which are misclassified by the biased model, to ensure the biased model
fully learns the biased features without overfitting to the bias-conflicting
samples. The biased model then assigns lower weights on the bias-conflicting
samples. Subsequently, we use the inverse of the sample weights of the biased
model for training the target model. Experiments show that our approach
achieves superior debiasing results compared to the existing baselines on both
synthetic and real-world datasets. Our code is available at
https://github.com/isruihu/Echoes.Comment: Accepted by ACM Multimedia 202
Fair Visual Recognition via Intervention with Proxy Features
Deep learning models often learn to make predictions that rely on sensitive
social attributes like gender and race, which poses significant fairness risks,
especially in societal applications, e.g., hiring, banking, and criminal
justice. Existing work tackles this issue by minimizing information about
social attributes in models for debiasing. However, the high correlation
between target task and social attributes makes bias mitigation incompatible
with target task accuracy. Recalling that model bias arises because the
learning of features in regard to bias attributes (i.e., bias features) helps
target task optimization, we explore the following research question: \emph{Can
we leverage proxy features to replace the role of bias feature in target task
optimization for debiasing?} To this end, we propose \emph{Proxy Debiasing}, to
first transfer the target task's learning of bias information from bias
features to artificial proxy features, and then employ causal intervention to
eliminate proxy features in inference. The key idea of \emph{Proxy Debiasing}
is to design controllable proxy features to on one hand replace bias features
in contributing to target task during the training stage, and on the other hand
easily to be removed by intervention during the inference stage. This
guarantees the elimination of bias features without affecting the target
information, thus addressing the fairness-accuracy paradox in previous
debiasing solutions. We apply \emph{Proxy Debiasing} to several benchmark
datasets, and achieve significant improvements over the state-of-the-art
debiasing methods in both of accuracy and fairness
Introducing Foundation Models as Surrogate Models: Advancing Towards More Practical Adversarial Attacks
Recently, the no-box adversarial attack, in which the attacker lacks access
to the model's architecture, weights, and training data, become the most
practical and challenging attack setup. However, there is an unawareness of the
potential and flexibility inherent in the surrogate model selection process on
no-box setting. Inspired by the burgeoning interest in utilizing foundational
models to address downstream tasks, this paper adopts an innovative idea that
1) recasting adversarial attack as a downstream task. Specifically, image noise
generation to meet the emerging trend and 2) introducing foundational models as
surrogate models. Harnessing the concept of non-robust features, we elaborate
on two guiding principles for surrogate model selection to explain why the
foundational model is an optimal choice for this role. However, paradoxically,
we observe that these foundational models underperform. Analyzing this
unexpected behavior within the feature space, we attribute the lackluster
performance of foundational models (e.g., CLIP) to their significant
representational capacity and, conversely, their lack of discriminative
prowess. To mitigate this issue, we propose the use of a margin-based loss
strategy for the fine-tuning of foundational models on target images. The
experimental results verify that our approach, which employs the basic Fast
Gradient Sign Method (FGSM) attack algorithm, outstrips the performance of
other, more convoluted algorithms. We conclude by advocating for the research
community to consider surrogate models as crucial determinants in the
effectiveness of adversarial attacks in no-box settings. The implications of
our work bear relevance for improving the efficacy of such adversarial attacks
and the overall robustness of AI systems
Attention, Please! Adversarial Defense via Attention Rectification and Preservation
This study provides a new understanding of the adversarial attack problem by
examining the correlation between adversarial attack and visual attention
change. In particular, we observed that: (1) images with incomplete attention
regions are more vulnerable to adversarial attacks; and (2) successful
adversarial attacks lead to deviated and scattered attention map. Accordingly,
an attention-based adversarial defense framework is designed to simultaneously
rectify the attention map for prediction and preserve the attention area
between adversarial and original images. The problem of adding iteratively
attacked samples is also discussed in the context of visual attention change.
We hope the attention-related data analysis and defense solution in this study
will shed some light on the mechanism behind the adversarial attack and also
facilitate future adversarial defense/attack model design
Benign Adversarial Attack: Tricking Models for Goodness
In spite of the successful application in many fields, machine learning
models today suffer from notorious problems like vulnerability to adversarial
examples. Beyond falling into the cat-and-mouse game between adversarial attack
and defense, this paper provides alternative perspective to consider
adversarial example and explore whether we can exploit it in benign
applications. We first attribute adversarial example to the human-model
disparity on employing non-semantic features. While largely ignored in
classical machine learning mechanisms, non-semantic feature enjoys three
interesting characteristics as (1) exclusive to model, (2) critical to affect
inference, and (3) utilizable as features. Inspired by this, we present brave
new idea of benign adversarial attack to exploit adversarial examples for
goodness in three directions: (1) adversarial Turing test, (2) rejecting
malicious model application, and (3) adversarial data augmentation. Each
direction is positioned with motivation elaboration, justification analysis and
prototype applications to showcase its potential.Comment: ACM MM2022 Brave New Ide
Towards Alleviating the Object Bias in Prompt Tuning-based Factual Knowledge Extraction
Many works employed prompt tuning methods to automatically optimize prompt
queries and extract the factual knowledge stored in Pretrained Language Models.
In this paper, we observe that the optimized prompts, including discrete
prompts and continuous prompts, exhibit undesirable object bias. To handle this
problem, we propose a novel prompt tuning method called MeCoD. consisting of
three modules: Prompt Encoder, Object Equalization and Biased Object
Obstruction. Experimental results show that MeCoD can significantly reduce the
object bias and at the same time improve accuracy of factual knowledge
extraction
Low-Mid Adversarial Perturbation against Unauthorized Face Recognition System
In light of the growing concerns regarding the unauthorized use of facial
recognition systems and its implications on individual privacy, the exploration
of adversarial perturbations as a potential countermeasure has gained traction.
However, challenges arise in effectively deploying this approach against
unauthorized facial recognition systems due to the effects of JPEG compression
on image distribution across the internet, which ultimately diminishes the
efficacy of adversarial perturbations. Existing JPEG compression-resistant
techniques struggle to strike a balance between resistance, transferability,
and attack potency. To address these limitations, we propose a novel solution
referred to as \emph{low frequency adversarial perturbation} (LFAP). This
method conditions the source model to leverage low-frequency characteristics
through adversarial training. To further enhance the performance, we introduce
an improved \emph{low-mid frequency adversarial perturbation} (LMFAP) that
incorporates mid-frequency components for an additive benefit. Our study
encompasses a range of settings to replicate genuine application scenarios,
including cross backbones, supervisory heads, training datasets, and testing
datasets. Moreover, we evaluated our approaches on a commercial black-box API,
\texttt{Face++}. The empirical results validate the cutting-edge performance
achieved by our proposed solutions.Comment: published in Information Science