3 research outputs found
Limitations of Pinned AUC for Measuring Unintended Bias
This report examines the Pinned AUC metric introduced and highlights some of
its limitations. Pinned AUC provides a threshold-agnostic measure of unintended
bias in a classification model, inspired by the ROC-AUC metric. However, as we
highlight in this report, there are ways that the metric can obscure different
kinds of unintended biases when the underlying class distributions on which
bias is being measured are not carefully controlled
Debiasing Personal Identities in Toxicity Classification
As Machine Learning models continue to be relied upon for making automated
decisions, the issue of model bias becomes more and more prevalent. In this
paper, we approach training a text classifica-tion model and optimize on bias
minimization by measuring not only the models performance on our dataset as a
whole, but also how it performs across different subgroups. This requires
measuring per-formance independently for different demographic subgroups and
measuring bias by comparing them to results from the rest of our data. We show
how unintended bias can be detected using these metrics and how removing bias
from a dataset completely can result in worse results
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Distribution shifts -- where the training distribution differs from the test
distribution -- can substantially degrade the accuracy of machine learning (ML)
systems deployed in the wild. Despite their ubiquity in the real-world
deployments, these distribution shifts are under-represented in the datasets
widely used in the ML community today. To address this gap, we present WILDS, a
curated benchmark of 10 datasets reflecting a diverse range of distribution
shifts that naturally arise in real-world applications, such as shifts across
hospitals for tumor identification; across camera traps for wildlife
monitoring; and across time and location in satellite imaging and poverty
mapping. On each dataset, we show that standard training yields substantially
lower out-of-distribution than in-distribution performance. This gap remains
even with models trained by existing methods for tackling distribution shifts,
underscoring the need for new methods for training models that are more robust
to the types of distribution shifts that arise in practice. To facilitate
method development, we provide an open-source package that automates dataset
loading, contains default model architectures and hyperparameters, and
standardizes evaluations. Code and leaderboards are available at
https://wilds.stanford.edu