15 research outputs found
Fair Meta-Learning: Learning How to Learn Fairly
Data sets for fairness relevant tasks can lack examples or be biased
according to a specific label in a sensitive attribute. We demonstrate the
usefulness of weight based meta-learning approaches in such situations. For
models that can be trained through gradient descent, we demonstrate that there
are some parameter configurations that allow models to be optimized from a few
number of gradient steps and with minimal data which are both fair and
accurate. To learn such weight sets, we adapt the popular MAML algorithm to
Fair-MAML by the inclusion of a fairness regularization term. In practice,
Fair-MAML allows practitioners to train fair machine learning models from only
a few examples when data from related tasks is available. We empirically
exhibit the value of this technique by comparing to relevant baselines.Comment: arXiv admin note: substantial text overlap with arXiv:1908.0909
Fairness Under Demographic Scarce Regime
Most existing works on fairness assume the model has full access to
demographic information. However, there exist scenarios where demographic
information is partially available because a record was not maintained
throughout data collection or due to privacy reasons. This setting is known as
demographic scarce regime. Prior research have shown that training an attribute
classifier to replace the missing sensitive attributes (proxy) can still
improve fairness. However, the use of proxy-sensitive attributes worsens
fairness-accuracy trade-offs compared to true sensitive attributes. To address
this limitation, we propose a framework to build attribute classifiers that
achieve better fairness-accuracy trade-offs. Our method introduces uncertainty
awareness in the attribute classifier and enforces fairness on samples with
demographic information inferred with the lowest uncertainty. We show
empirically that enforcing fairness constraints on samples with uncertain
sensitive attributes is detrimental to fairness and accuracy. Our experiments
on two datasets showed that the proposed framework yields models with
significantly better fairness-accuracy trade-offs compared to classic attribute
classifiers. Surprisingly, our framework outperforms models trained with
constraints on the true sensitive attributes.Comment: 14 pages, 7 page
Unbiased Decisions Reduce Regret: Adversarial Domain Adaptation for the Bank Loan Problem
In many real world settings binary classification decisions are made based on
limited data in near real-time, e.g. when assessing a loan application. We
focus on a class of these problems that share a common feature: the true label
is only observed when a data point is assigned a positive label by the
principal, e.g. we only find out whether an applicant defaults if we accepted
their loan application. As a consequence, the false rejections become
self-reinforcing and cause the labelled training set, that is being
continuously updated by the model decisions, to accumulate bias. Prior work
mitigates this effect by injecting optimism into the model, however this comes
at the cost of increased false acceptance rate. We introduce adversarial
optimism (AdOpt) to directly address bias in the training set using adversarial
domain adaptation. The goal of AdOpt is to learn an unbiased but informative
representation of past data, by reducing the distributional shift between the
set of accepted data points and all data points seen thus far. AdOpt
significantly exceeds state-of-the-art performance on a set of challenging
benchmark problems. Our experiments also provide initial evidence that the
introduction of adversarial domain adaptation improves fairness in this
setting
Within-group fairness: A guidance for more sound between-group fairness
As they have a vital effect on social decision-making, AI algorithms not only
should be accurate and but also should not pose unfairness against certain
sensitive groups (e.g., non-white, women). Various specially designed AI
algorithms to ensure trained AI models to be fair between sensitive groups have
been developed. In this paper, we raise a new issue that between-group fair AI
models could treat individuals in a same sensitive group unfairly. We introduce
a new concept of fairness so-called within-group fairness which requires that
AI models should be fair for those in a same sensitive group as well as those
in different sensitive groups. We materialize the concept of within-group
fairness by proposing corresponding mathematical definitions and developing
learning algorithms to control within-group fairness and between-group fairness
simultaneously. Numerical studies show that the proposed learning algorithms
improve within-group fairness without sacrificing accuracy as well as
between-group fairness
Robust and Fair Machine Learning under Distribution Shift
Machine learning algorithms have been widely used in real world applications. The development of these techniques has brought huge benefits for many AI-related tasks, such as natural language processing, image classification, video analysis, and so forth. In traditional machine learning algorithms, we usually assume that the training data and test data are independently and identically distributed (iid), indicating that the model learned from the training data can be well applied to the test data with good prediction performance. However, this assumption is quite restrictive because the distribution shift can exist from the training data to the test data in many scenarios. In addition, the goal of traditional machine learning model is to maximize the prediction performance, e.g., accuracy, based on the historical training data, which may tend to make unfair predictions for some particular individual or groups. In the literature, researchers either focus on building robust machine learning models under data distribution shift or achieving fairness separately, without considering to solve them simultaneously.
The goal of this dissertation is to solve the above challenging issues in fair machine learning under distribution shift. We start from building an agnostic fair framework in federated learning as the data distribution is more diversified and distribution shift exists from the training data to the test data. Then we build a robust framework to address the sample selection bias for fair classification. Next we solve the sample selection bias issue for fair regression. Finally, we propose an adversarial framework to build a personalized model in the distributed setting where the distribution shift exists between different users.
In this dissertation, we conduct the following research for fair machine learning under distribution shift. • We develop a fairness-aware agnostic federated learning framework (AgnosticFair) to deal with the challenge of unknown testing distribution; • We propose a framework for robust and fair learning under sample selection bias; • We develop a framework for fair regression under sample selection bias when dependent variable values of a set of samples from the training data are missing as a result of another hidden process; • We propose a learning framework that allows an individual user to build a personalized model in a distributed setting, where the distribution shift exists among different users