43,012 research outputs found
Dynamic Fair Federated Learning Based on Reinforcement Learning
Federated learning enables a collaborative training and optimization of
global models among a group of devices without sharing local data samples.
However, the heterogeneity of data in federated learning can lead to unfair
representation of the global model across different devices. To address the
fairness issue in federated learning, we propose a dynamic q fairness federated
learning algorithm with reinforcement learning, called DQFFL. DQFFL aims to
mitigate the discrepancies in device aggregation and enhance the fairness of
treatment for all groups involved in federated learning. To quantify fairness,
DQFFL leverages the performance of the global federated model on each device
and incorporates {\alpha}-fairness to transform the preservation of fairness
during federated aggregation into the distribution of client weights in the
aggregation process. Considering the sensitivity of parameters in measuring
fairness, we propose to utilize reinforcement learning for dynamic parameters
during aggregation. Experimental results demonstrate that our DQFFL outperforms
the state-of-the-art methods in terms of overall performance, fairness and
convergence speed
Reinforcement Learning with Stepwise Fairness Constraints
AI methods are used in societally important settings, ranging from credit to
employment to housing, and it is crucial to provide fairness in regard to
algorithmic decision making. Moreover, many settings are dynamic, with
populations responding to sequential decision policies. We introduce the study
of reinforcement learning (RL) with stepwise fairness constraints, requiring
group fairness at each time step. Our focus is on tabular episodic RL, and we
provide learning algorithms with strong theoretical guarantees in regard to
policy optimality and fairness violation. Our framework provides useful tools
to study the impact of fairness constraints in sequential settings and brings
up new challenges in RL.Comment: Fairness, Reinforcement Learnin
Fairness in Preference-based Reinforcement Learning
In this paper, we address the issue of fairness in preference-based
reinforcement learning (PbRL) in the presence of multiple objectives. The main
objective is to design control policies that can optimize multiple objectives
while treating each objective fairly. Toward this objective, we design a new
fairness-induced preference-based reinforcement learning or FPbRL. The main
idea of FPbRL is to learn vector reward functions associated with multiple
objectives via new welfare-based preferences rather than reward-based
preference in PbRL, coupled with policy learning via maximizing a generalized
Gini welfare function. Finally, we provide experiment studies on three
different environments to show that the proposed FPbRL approach can achieve
both efficiency and equity for learning effective and fair policies.Comment: Accepted to The Many Facets of Preference Learning Workshop at the
International Conference on Machine Learning (ICML
Fairness Through Counterfactual Utilities
Group fairness definitions such as Demographic Parity and Equal Opportunity
make assumptions about the underlying decision-problem that restrict them to
classification problems. Prior work has translated these definitions to other
machine learning environments, such as unsupervised learning and reinforcement
learning, by implementing their closest mathematical equivalent. As a result,
there are numerous bespoke interpretations of these definitions. Instead, we
provide a generalized set of group fairness definitions that unambiguously
extend to all machine learning environments while still retaining their
original fairness notions. We derive two fairness principles that enable such a
generalized framework. First, our framework measures outcomes in terms of
utilities, rather than predictions, and does so for both the decision-algorithm
and the individual. Second, our framework considers counterfactual outcomes,
rather than just observed outcomes, thus preventing loopholes where fairness
criteria are satisfied through self-fulfilling prophecies. We provide concrete
examples of how our counterfactual utility fairness framework resolves known
fairness issues in classification, clustering, and reinforcement learning
problems. We also show that many of the bespoke interpretations of Demographic
Parity and Equal Opportunity fit nicely as special cases of our framework
Long-Term Fairness with Unknown Dynamics
While machine learning can myopically reinforce social inequalities, it may
also be used to dynamically seek equitable outcomes. In this paper, we
formalize long-term fairness in the context of online reinforcement learning.
This formulation can accommodate dynamical control objectives, such as driving
equity inherent in the state of a population, that cannot be incorporated into
static formulations of fairness. We demonstrate that this framing allows an
algorithm to adapt to unknown dynamics by sacrificing short-term incentives to
drive a classifier-population system towards more desirable equilibria. For the
proposed setting, we develop an algorithm that adapts recent work in online
learning. We prove that this algorithm achieves simultaneous probabilistic
bounds on cumulative loss and cumulative violations of fairness (as statistical
regularities between demographic groups). We compare our proposed algorithm to
the repeated retraining of myopic classifiers, as a baseline, and to a deep
reinforcement learning algorithm that lacks safety guarantees. Our experiments
model human populations according to evolutionary game theory and integrate
real-world datasets
Striking a Balance in Fairness for Dynamic Systems Through Reinforcement Learning
While significant advancements have been made in the field of fair machine
learning, the majority of studies focus on scenarios where the decision model
operates on a static population. In this paper, we study fairness in dynamic
systems where sequential decisions are made. Each decision may shift the
underlying distribution of features or user behavior. We model the dynamic
system through a Markov Decision Process (MDP). By acknowledging that
traditional fairness notions and long-term fairness are distinct requirements
that may not necessarily align with one another, we propose an algorithmic
framework to integrate various fairness considerations with reinforcement
learning using both pre-processing and in-processing approaches. Three case
studies show that our method can strike a balance between traditional fairness
notions, long-term fairness, and utility
- …