7,417 research outputs found
Dynamic Fair Federated Learning Based on Reinforcement Learning
Federated learning enables a collaborative training and optimization of
global models among a group of devices without sharing local data samples.
However, the heterogeneity of data in federated learning can lead to unfair
representation of the global model across different devices. To address the
fairness issue in federated learning, we propose a dynamic q fairness federated
learning algorithm with reinforcement learning, called DQFFL. DQFFL aims to
mitigate the discrepancies in device aggregation and enhance the fairness of
treatment for all groups involved in federated learning. To quantify fairness,
DQFFL leverages the performance of the global federated model on each device
and incorporates {\alpha}-fairness to transform the preservation of fairness
during federated aggregation into the distribution of client weights in the
aggregation process. Considering the sensitivity of parameters in measuring
fairness, we propose to utilize reinforcement learning for dynamic parameters
during aggregation. Experimental results demonstrate that our DQFFL outperforms
the state-of-the-art methods in terms of overall performance, fairness and
convergence speed
Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing
Recently, open radio access network (O-RAN) has become a promising technology
to provide an open environment for network vendors and operators. Coordinating
the x-applications (xAPPs) is critical to increase flexibility and guarantee
high overall network performance in O-RAN. Meanwhile, federated reinforcement
learning has been proposed as a promising technique to enhance the
collaboration among distributed reinforcement learning agents and improve
learning efficiency. In this paper, we propose a federated deep reinforcement
learning algorithm to coordinate multiple independent xAPPs in O-RAN for
network slicing. We design two xAPPs, namely a power control xAPP and a
slice-based resource allocation xAPP, and we use a federated learning model to
coordinate two xAPP agents to enhance learning efficiency and improve network
performance. Compared with conventional deep reinforcement learning, our
proposed algorithm can achieve 11% higher throughput for enhanced mobile
broadband (eMBB) slices and 33% lower delay for ultra-reliable low-latency
communication (URLLC) slices
Federated Offline Reinforcement Learning
Evidence-based or data-driven dynamic treatment regimes are essential for
personalized medicine, which can benefit from offline reinforcement learning
(RL). Although massive healthcare data are available across medical
institutions, they are prohibited from sharing due to privacy constraints.
Besides, heterogeneity exists in different sites. As a result, federated
offline RL algorithms are necessary and promising to deal with the problems. In
this paper, we propose a multi-site Markov decision process model that allows
for both homogeneous and heterogeneous effects across sites. The proposed model
makes the analysis of the site-level features possible. We design the first
federated policy optimization algorithm for offline RL with sample complexity.
The proposed algorithm is communication-efficient, which requires only a single
round of communication interaction by exchanging summary statistics. We give a
theoretical guarantee for the proposed algorithm, where the suboptimality for
the learned policies is comparable to the rate as if data is not distributed.
Extensive simulations demonstrate the effectiveness of the proposed algorithm.
The method is applied to a sepsis dataset in multiple sites to illustrate its
use in clinical settings
Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
This paper was motivated by the problem of how to make robots fuse and
transfer their experience so that they can effectively use prior knowledge and
quickly adapt to new environments. To address the problem, we present a
learning architecture for navigation in cloud robotic systems: Lifelong
Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge
fusion algorithm for upgrading a shared model deployed on the cloud. Then,
effective transfer learning methods in LFRL are introduced. LFRL is consistent
with human cognitive science and fits well in cloud robotic systems.
Experiments show that LFRL greatly improves the efficiency of reinforcement
learning for robot navigation. The cloud robotic system deployment also shows
that LFRL is capable of fusing prior knowledge. In addition, we release a cloud
robotic navigation-learning website based on LFRL
Federated TD Learning Over Finite-Rate Erasure Channels: Linear Speedup Under Markovian Sampling
Federated learning (FL) has recently gained much attention due to its effectiveness in speeding up supervised learning tasks under communication and privacy constraints. However, whether similar speedups can be established for reinforcement learning remains much less understood theoretically. Towards this direction, we study a federated policy evaluation problem where agents communicate via a central aggregator to expedite the evaluation of a common policy. To capture typical communication constraints in FL, we consider finite capacity up-link channels that can drop packets based on a Bernoulli erasure model. Given this setting, we propose and analyze QFedTD - a quantized federated temporal difference learning algorithm with linear function approximation. Our main technical contribution is to provide a finite-sample analysis of QFedTD that (i) highlights the effect of quantization and erasures on the convergence rate; and (ii) establishes a linear speedup w.r.t. the number of agents under Markovian sampling. Notably, while different quantization mechanisms and packet drop models have been extensively studied in the FL, distributed optimization, and networked control systems literature, our work is the first to provide a non-asymptotic analysis of their effects in multi-agent and federated reinforcement learning
Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations
In the growing world of artificial intelligence, federated learning is a
distributed learning framework enhanced to preserve the privacy of individuals'
data. Federated learning lays the groundwork for collaborative research in
areas where the data is sensitive. Federated learning has several implications
for real-world problems. In times of crisis, when real-time decision-making is
critical, federated learning allows multiple entities to work collectively
without sharing sensitive data. This distributed approach enables us to
leverage information from multiple sources and gain more diverse insights. This
paper is a systematic review of the literature on privacy-preserving machine
learning in the last few years based on the Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Specifically, we have
presented an extensive review of supervised/unsupervised machine learning
algorithms, ensemble methods, meta-heuristic approaches, blockchain technology,
and reinforcement learning used in the framework of federated learning, in
addition to an overview of federated learning applications. This paper reviews
the literature on the components of federated learning and its applications in
the last few years. The main purpose of this work is to provide researchers and
practitioners with a comprehensive overview of federated learning from the
machine learning point of view. A discussion of some open problems and future
research directions in federated learning is also provided
- …