35 research outputs found
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?
While a substantial body of prior work has explored adversarial example
generation for natural language understanding tasks, these examples are often
unrealistic and diverge from the real-world data distributions. In this work,
we introduce a two-stage adversarial example generation framework
(NaturalAdversaries), for designing adversaries that are effective at fooling a
given classifier and demonstrate natural-looking failure cases that could
plausibly occur during in-the-wild deployment of the models.
At the first stage a token attribution method is used to summarize a given
classifier's behaviour as a function of the key tokens in the input. In the
second stage a generative model is conditioned on the key tokens from the first
stage. NaturalAdversaries is adaptable to both black-box and white-box
adversarial attacks based on the level of access to the model parameters. Our
results indicate these adversaries generalize across domains, and offer
insights for future research on improving robustness of neural text
classification models.Comment: Findings of EMNLP 202
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Large-scale Pre-Trained Language Models (PTLMs) capture knowledge from
massive human-written data which contains latent societal biases and toxic
contents. In this paper, we leverage the primary task of PTLMs, i.e., language
modeling, and propose a new metric to quantify manifested implicit
representational harms in PTLMs towards 13 marginalized demographics. Using
this metric, we conducted an empirical analysis of 24 widely used PTLMs. Our
analysis provides insights into the correlation between the proposed metric in
this work and other related metrics for representational harm. We observe that
our metric correlates with most of the gender-specific metrics in the
literature. Through extensive experiments, we explore the connections between
PTLMs architectures and representational harms across two dimensions: depth and
width of the networks. We found that prioritizing depth over width, mitigates
representational harms in some PTLMs. Our code and data can be found at
https://github.com/microsoft/SafeNLP.Comment: 17 pages
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning
Spurious correlations that degrade model generalization or lead the model to
be right for the wrong reasons are one of the main robustness concerns for
real-world deployments. However, mitigating these correlations during
pre-training for large-scale models can be costly and impractical, particularly
for those without access to high-performance computing resources. This paper
proposes a novel approach to address spurious correlations during fine-tuning
for a given domain of interest. With a focus on multi-modal models (e.g.,
CLIP), the proposed method leverages different modalities in these models to
detect and explicitly set apart spurious attributes from the affected class,
achieved through a multi-modal contrastive loss function that expresses
spurious relationships through language. Our experimental results and in-depth
visualizations on CLIP show that such an intervention can effectively i)
improve the model's accuracy when spurious attributes are not present, and ii)
directs the model's activation maps towards the actual class rather than the
spurious attribute when present. In particular, on the Waterbirds dataset, our
algorithm achieved a worst-group accuracy 23% higher than ERM on CLIP with a
ResNet-50 backbone, and 32% higher on CLIP with a ViT backbone, while
maintaining the same average accuracy as ERM
Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
Large pre-trained models decay over long-term deployment as input
distributions shift, user requirements change, or crucial knowledge gaps are
discovered. Recently, model editors have been proposed to modify a model's
behavior by adjusting its weights during deployment. However, when editing the
same model multiple times, these approaches quickly decay a model's performance
on upstream data and forget how to fix previous errors. We propose and study a
novel Lifelong Model Editing setting, where streaming errors are identified for
a deployed model and we update the model to correct its predictions without
influencing unrelated inputs without access to training edits, exogenous
datasets, or any upstream data for the edited model. To approach this problem,
we introduce General Retrieval Adaptors for Continual Editing, or GRACE, which
learns to cache a chosen layer's activations in an adaptive codebook as edits
stream in, leaving original model weights frozen. GRACE can thus edit models
thousands of times in a row using only streaming errors, while minimally
influencing unrelated inputs. Experimentally, we show that GRACE improves over
recent model editors and generalizes to unseen inputs. Our code is available at
https://www.github.com/thartvigsen/grace
Improving Pre-trained Language Models' Generalization
The reusability of state-of-the-art Pre-trained Language Models (PLMs) is
often limited by their generalization problem, where their performance
drastically decreases when evaluated on examples that differ from the training
dataset, known as Out-of-Distribution (OOD)/unseen examples. This limitation
arises from PLMs' reliance on spurious correlations, which work well for
frequent example types but not for general examples. To address this issue, we
propose a training approach called Mask-tuning, which integrates Masked
Language Modeling (MLM) training objectives into the fine-tuning process to
enhance PLMs' generalization. Comprehensive experiments demonstrate that
Mask-tuning surpasses current state-of-the-art techniques and enhances PLMs'
generalization on OOD datasets while improving their performance on
in-distribution datasets. The findings suggest that Mask-tuning improves the
reusability of PLMs on unseen data, making them more practical and effective
for real-world applications