100 research outputs found
Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning
Human society had a long history of suffering from cognitive biases leading
to social prejudices and mass injustice. The prevalent existence of cognitive
biases in large volumes of historical data can pose a threat of being
manifested as unethical and seemingly inhuman predictions as outputs of AI
systems trained on such data. To alleviate this problem, we propose a
bias-aware multi-objective learning framework that given a set of identity
attributes (e.g. gender, ethnicity etc.) and a subset of sensitive categories
of the possible classes of prediction outputs, learns to reduce the frequency
of predicting certain combinations of them, e.g. predicting stereotypes such as
`most blacks use abusive language', or `fear is a virtue of women'. Our
experiments conducted on an emotion prediction task with balanced class priors
shows that a set of baseline bias-agnostic models exhibit cognitive biases with
respect to gender, such as women are prone to be afraid whereas men are more
prone to be angry. In contrast, our proposed bias-aware multi-objective
learning methodology is shown to reduce such biases in the predictied emotions
An Empirical Study on the Fairness of Pre-trained Word Embeddings
Pre-trained word embedding models are easily distributed and applied, as they alleviate
users from the effort to train models themselves.
With widely distributed models, it is important to ensure that they do not exhibit undesired behaviour, such as biases against population groups. For this purpose, we carry out
an empirical study on evaluating the bias of
15 publicly available, pre-trained word embeddings model based on three training algorithms
(GloVe, word2vec, and fastText) with
regard to four bias metrics (WEAT, SEMBIAS,
DIRECT BIAS, and ECT). The choice of word
embedding models and bias metrics is motivated by a literature survey over 37 publications
which quantified bias on pre-trained word embeddings. Our results indicate that fastText
is the least biased model (in 8 out of 12 cases)
and small vector lengths lead to a higher bias
Fair Is Better than Sensational:Man Is to Doctor as Woman Is to Doctor
Analogies such as "man is to king as woman is to X" are often used to
illustrate the amazing power of word embeddings. Concurrently, they have also
been used to expose how strongly human biases are encoded in vector spaces
built on natural language, like "man is to computer programmer as woman is to
homemaker". Recent work has shown that analogies are in fact not such a
diagnostic for bias, and other methods have been proven to be more apt to the
task. However, beside the intrinsic problems with the analogy task as a bias
detection tool, in this paper we show that a series of issues related to how
analogies have been implemented and used might have yielded a distorted picture
of bias in word embeddings. Human biases are present in word embeddings and
need to be addressed. Analogies, though, are probably not the right tool to do
so. Also, the way they have been most often used has exacerbated some possibly
non-existing biases and perhaps hid others. Because they are still widely
popular, and some of them have become classics within and outside the NLP
community, we deem it important to provide a series of clarifications that
should put well-known, and potentially new cases into the right perspective
Language (Technology) is Power: A Critical Survey of "Bias" in NLP
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities
Exploring Gender Bias in Semantic Representations for Occupational Classification in NLP: Techniques and Mitigation Strategies
Gender bias in Natural Language Processing (NLP) models is a non-trivial problem that can perpetuate and amplify existing societal biases. This thesis investigates gender bias in occupation classification and explores the effectiveness of different debiasing methods for language models to reduce the impact of bias in the model’s representations. The study employs a data-driven empirical methodology focusing heavily on experimentation and result investigation. The study uses five distinct semantic representations and models with varying levels of complexity to classify the occupation of individuals based on their biographies
- …