345 research outputs found
Dictionary-based Debiasing of Pre-trained Word Embeddings.
Word embeddings trained on large corpora have shown to encode high levels of
unfair discriminatory gender, racial, religious and ethnic biases.
In contrast, human-written dictionaries describe the meanings of words in a
concise, objective and an unbiased manner.
We propose a method for debiasing pre-trained word embeddings using
dictionaries, without requiring access to the original training resources or
any knowledge regarding the word embedding algorithms used.
Unlike prior work, our proposed method does not require the types of biases
to be pre-defined in the form of word lists, and learns the constraints that
must be satisfied by unbiased word embeddings automatically from dictionary
definitions of the words.
Specifically, we learn an encoder to generate a debiased version of an input
word embedding such that it
(a) retains the semantics of the pre-trained word embeddings,
(b) agrees with the unbiased definition of the word according to the
dictionary, and
(c) remains orthogonal to the vector space spanned by any biased basis
vectors in the pre-trained word embedding space.
Experimental results on standard benchmark datasets show that the proposed
method can accurately remove unfair biases encoded in pre-trained word
embeddings, while preserving useful semantics.Comment: EACL 202
Quantifying and Reducing Stereotypes in Word Embeddings
Machine learning algorithms are optimized to model statistical properties of
the training data. If the input data reflects stereotypes and biases of the
broader society, then the output of the learning algorithm also captures these
stereotypes. In this paper, we initiate the study of gender stereotypes in {\em
word embedding}, a popular framework to represent text data. As their use
becomes increasingly common, applications can inadvertently amplify unwanted
stereotypes. We show across multiple datasets that the embeddings contain
significant gender stereotypes, especially with regard to professions. We
created a novel gender analogy task and combined it with crowdsourcing to
systematically quantify the gender bias in a given embedding. We developed an
efficient algorithm that reduces gender stereotype using just a handful of
training examples while preserving the useful geometric properties of the
embedding. We evaluated our algorithm on several metrics. While we focus on
male/female stereotypes, our framework may be applicable to other types of
embedding biases.Comment: presented at 2016 ICML Workshop on #Data4Good: Machine Learning in
Social Good Applications, New York, N
Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning
Human society had a long history of suffering from cognitive biases leading
to social prejudices and mass injustice. The prevalent existence of cognitive
biases in large volumes of historical data can pose a threat of being
manifested as unethical and seemingly inhuman predictions as outputs of AI
systems trained on such data. To alleviate this problem, we propose a
bias-aware multi-objective learning framework that given a set of identity
attributes (e.g. gender, ethnicity etc.) and a subset of sensitive categories
of the possible classes of prediction outputs, learns to reduce the frequency
of predicting certain combinations of them, e.g. predicting stereotypes such as
`most blacks use abusive language', or `fear is a virtue of women'. Our
experiments conducted on an emotion prediction task with balanced class priors
shows that a set of baseline bias-agnostic models exhibit cognitive biases with
respect to gender, such as women are prone to be afraid whereas men are more
prone to be angry. In contrast, our proposed bias-aware multi-objective
learning methodology is shown to reduce such biases in the predictied emotions
A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations
Word embedding has become essential for natural language processing as it
boosts empirical performances of various tasks. However, recent research
discovers that gender bias is incorporated in neural word embeddings, and
downstream tasks that rely on these biased word vectors also produce
gender-biased results. While some word-embedding gender-debiasing methods have
been developed, these methods mainly focus on reducing gender bias associated
with gender direction and fail to reduce the gender bias presented in word
embedding relations. In this paper, we design a causal and simple approach for
mitigating gender bias in word vector relation by utilizing the statistical
dependency between gender-definition word embeddings and gender-biased word
embeddings. Our method attains state-of-the-art results on gender-debiasing
tasks, lexical- and sentence-level evaluation tasks, and downstream coreference
resolution tasks.Comment: Accepted by AAAI 202
Do Neural Ranking Models Intensify Gender Bias?
Concerns regarding the footprint of societal biases in information retrieval
(IR) systems have been raised in several previous studies. In this work, we
examine various recent IR models from the perspective of the degree of gender
bias in their retrieval results. To this end, we first provide a bias
measurement framework which includes two metrics to quantify the degree of the
unbalanced presence of gender-related concepts in a given IR model's ranking
list. To examine IR models by means of the framework, we create a dataset of
non-gendered queries, selected by human annotators. Applying these queries to
the MS MARCO Passage retrieval collection, we then measure the gender bias of a
BM25 model and several recent neural ranking models. The results show that
while all models are strongly biased toward male, the neural models, and in
particular the ones based on contextualized embedding models, significantly
intensify gender bias. Our experiments also show an overall increase in the
gender bias of neural models when they exploit transfer learning, namely when
they use (already biased) pre-trained embeddings.Comment: In Proceedings of ACM SIGIR 202
- …