371 research outputs found
On Measuring and Mitigating Biased Inferences of Word Embeddings
Word embeddings carry stereotypical connotations from the text they are
trained on, which can lead to invalid inferences in downstream models that rely
on them. We use this observation to design a mechanism for measuring
stereotypes using the task of natural language inference. We demonstrate a
reduction in invalid inferences via bias mitigation strategies on static word
embeddings (GloVe). Further, we show that for gender bias, these techniques
extend to contextualized embeddings when applied selectively only to the static
components of contextualized embeddings (ELMo, BERT)
Language (Technology) is Power: A Critical Survey of "Bias" in NLP
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities
Survey on Sociodemographic Bias in Natural Language Processing
Deep neural networks often learn unintended biases during training, which
might have harmful effects when deployed in real-world settings. This paper
surveys 209 papers on bias in NLP models, most of which address
sociodemographic bias. To better understand the distinction between bias and
real-world harm, we turn to ideas from psychology and behavioral economics to
propose a definition for sociodemographic bias. We identify three main
categories of NLP bias research: types of bias, quantifying bias, and
debiasing. We conclude that current approaches on quantifying bias face
reliability issues, that many of the bias metrics do not relate to real-world
biases, and that current debiasing techniques are superficial and hide bias
rather than removing it. Finally, we provide recommendations for future work.Comment: 23 pages, 1 figur
Bridging Fairness and Environmental Sustainability in Natural Language Processing
Fairness and environmental impact are important research directions for the
sustainable development of artificial intelligence. However, while each topic
is an active research area in natural language processing (NLP), there is a
surprising lack of research on the interplay between the two fields. This
lacuna is highly problematic, since there is increasing evidence that an
exclusive focus on fairness can actually hinder environmental sustainability,
and vice versa. In this work, we shed light on this crucial intersection in NLP
by (1) investigating the efficiency of current fairness approaches through
surveying example methods for reducing unfair stereotypical bias from the
literature, and (2) evaluating a common technique to reduce energy consumption
(and thus environmental impact) of English NLP models, knowledge distillation
(KD), for its impact on fairness. In this case study, we evaluate the effect of
important KD factors, including layer and dimensionality reduction, with
respect to: (a) performance on the distillation task (natural language
inference and semantic similarity prediction), and (b) multiple measures and
dimensions of stereotypical bias (e.g., gender bias measured via the Word
Embedding Association Test). Our results lead us to clarify current assumptions
regarding the effect of KD on unfair bias: contrary to other findings, we show
that KD can actually decrease model fairness.Comment: Accepted for publication at EMNLP 202
Survey of Social Bias in Vision-Language Models
In recent years, the rapid advancement of machine learning (ML) models,
particularly transformer-based pre-trained models, has revolutionized Natural
Language Processing (NLP) and Computer Vision (CV) fields. However, researchers
have discovered that these models can inadvertently capture and reinforce
social biases present in their training datasets, leading to potential social
harms, such as uneven resource allocation and unfair representation of specific
social groups. Addressing these biases and ensuring fairness in artificial
intelligence (AI) systems has become a critical concern in the ML community.
The recent introduction of pre-trained vision-and-language (VL) models in the
emerging multimodal field demands attention to the potential social biases
present in these models as well. Although VL models are susceptible to social
bias, there is a limited understanding compared to the extensive discussions on
bias in NLP and CV. This survey aims to provide researchers with a high-level
insight into the similarities and differences of social bias studies in
pre-trained models across NLP, CV, and VL. By examining these perspectives, the
survey aims to offer valuable guidelines on how to approach and mitigate social
bias in both unimodal and multimodal settings. The findings and recommendations
presented here can benefit the ML community, fostering the development of
fairer and non-biased AI models in various applications and research endeavors
A Survey on Fairness in Large Language Models
Large language models (LLMs) have shown powerful performance and development
prospect and are widely deployed in the real world. However, LLMs can capture
social biases from unprocessed training data and propagate the biases to
downstream tasks. Unfair LLM systems have undesirable social impacts and
potential harms. In this paper, we provide a comprehensive review of related
research on fairness in LLMs. First, for medium-scale LLMs, we introduce
evaluation metrics and debiasing methods from the perspectives of intrinsic
bias and extrinsic bias, respectively. Then, for large-scale LLMs, we introduce
recent fairness research, including fairness evaluation, reasons for bias, and
debiasing methods. Finally, we discuss and provide insight on the challenges
and future directions for the development of fairness in LLMs.Comment: 12 pages, 2 figures, 101 reference
- …