309 research outputs found
Language (Technology) is Power: A Critical Survey of "Bias" in NLP
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities
No Word Embedding Model Is Perfect: Evaluating the Representation Accuracy for Social Bias in the Media
News articles both shape and reflect public opinion across the political
spectrum. Analyzing them for social bias can thus provide valuable insights,
such as prevailing stereotypes in society and the media, which are often
adopted by NLP models trained on respective data. Recent work has relied on
word embedding bias measures, such as WEAT. However, several representation
issues of embeddings can harm the measures' accuracy, including low-resource
settings and token frequency differences. In this work, we study what kind of
embedding algorithm serves best to accurately measure types of social bias
known to exist in US online news articles. To cover the whole spectrum of
political bias in the US, we collect 500k articles and review psychology
literature with respect to expected social bias. We then quantify social bias
using WEAT along with embedding algorithms that account for the aforementioned
issues. We compare how models trained with the algorithms on news articles
represent the expected social bias. Our results suggest that the standard way
to quantify bias does not align well with knowledge from psychology. While the
proposed algorithms reduce the~gap, they still do not fully match the
literature.Comment: Accepted to Findings of the Association for Computational
Linguistics: EMNLP 202
Gender Stereotype Reinforcement: Measuring the Gender Bias Conveyed by Ranking Algorithms
Search Engines (SE) have been shown to perpetuate well-known gender
stereotypes identified in psychology literature and to influence users
accordingly. Similar biases were found encoded in Word Embeddings (WEs) learned
from large online corpora. In this context, we propose the Gender Stereotype
Reinforcement (GSR) measure, which quantifies the tendency of a SE to support
gender stereotypes, leveraging gender-related information encoded in WEs.
Through the critical lens of construct validity, we validate the proposed
measure on synthetic and real collections. Subsequently, we use GSR to compare
widely-used Information Retrieval ranking algorithms, including lexical,
semantic, and neural models. We check if and how ranking algorithms based on
WEs inherit the biases of the underlying embeddings. We also consider the most
common debiasing approaches for WEs proposed in the literature and test their
impact in terms of GSR and common performance measures. To the best of our
knowledge, GSR is the first specifically tailored measure for IR, capable of
quantifying representational harms.Comment: To appear in Information Processing & Managemen
Evaluating Biased Attitude Associations of Language Models in an Intersectional Context
Language models are trained on large-scale corpora that embed implicit biases
documented in psychology. Valence associations (pleasantness/unpleasantness) of
social groups determine the biased attitudes towards groups and concepts in
social cognition. Building on this established literature, we quantify how
social groups are valenced in English language models using a sentence template
that provides an intersectional context. We study biases related to age,
education, gender, height, intelligence, literacy, race, religion, sex, sexual
orientation, social class, and weight. We present a concept projection approach
to capture the valence subspace through contextualized word embeddings of
language models. Adapting the projection-based approach to embedding
association tests that quantify bias, we find that language models exhibit the
most biased attitudes against gender identity, social class, and sexual
orientation signals in language. We find that the largest and better-performing
model that we study is also more biased as it effectively captures bias
embedded in sociocultural data. We validate the bias evaluation method by
overperforming on an intrinsic valence evaluation task. The approach enables us
to measure complex intersectional biases as they are known to manifest in the
outputs and applications of language models that perpetuate historical biases.
Moreover, our approach contributes to design justice as it studies the
associations of groups underrepresented in language such as transgender and
homosexual individuals.Comment: to be published in AIES 202
A World Full of Stereotypes? Further Investigation on Origin and Gender Bias in Multi-Lingual Word Embeddings
Publicly available off-the-shelf word embeddings that are often used in productive applications for natural language processing have been proven to be biased. We have previously shown that this bias can come in a different form, depending on the language and the cultural context. In this work we extend our previous work and further investigate how bias varies in different languages. We examine Italian and Swedish word embeddings for gender and origin bias, and demonstrate how an origin bias concerning local migration groups in Switzerland is included in German word embeddings. We propose BiasWords, a method to automatically detect new forms of bias. Finally, we discuss how cultural and language aspects are relevant to the impact of bias on the application, and to potential mitigation measures
- …