5,770 research outputs found
Continual Local Training for Better Initialization of Federated Models
Federated learning (FL) refers to the learning paradigm that trains machine
learning models directly in the decentralized systems consisting of smart edge
devices without transmitting the raw data, which avoids the heavy communication
costs and privacy concerns. Given the typical heterogeneous data distributions
in such situations, the popular FL algorithm \emph{Federated Averaging}
(FedAvg) suffers from weight divergence and thus cannot achieve a competitive
performance for the global model (denoted as the \emph{initial performance} in
FL) compared to centralized methods. In this paper, we propose the local
continual training strategy to address this problem. Importance weights are
evaluated on a small proxy dataset on the central server and then used to
constrain the local training. With this additional term, we alleviate the
weight divergence and continually integrate the knowledge on different local
clients into the global model, which ensures a better generalization ability.
Experiments on various FL settings demonstrate that our method significantly
improves the initial performance of federated models with few extra
communication costs.Comment: This paper has been accepted to 2020 IEEE International Conference on
Image Processing (ICIP 2020
Peer to Peer Information Retrieval: An Overview
Peer-to-peer technology is widely used for file sharing. In the past decade a number of prototype peer-to-peer information retrieval systems have been developed. Unfortunately, none of these have seen widespread real- world adoption and thus, in contrast with file sharing, information retrieval is still dominated by centralised solutions. In this paper we provide an overview of the key challenges for peer-to-peer information retrieval and the work done so far. We want to stimulate and inspire further research to overcome these challenges. This will open the door to the development and large-scale deployment of real-world peer-to-peer information retrieval systems that rival existing centralised client-server solutions in terms of scalability, performance, user satisfaction and freedom
A Survey on Secure and Private Federated Learning Using Blockchain: Theory and Application in Resource-constrained Computing
Federated Learning (FL) has gained widespread popularity in recent years due
to the fast booming of advanced machine learning and artificial intelligence
along with emerging security and privacy threats. FL enables efficient model
generation from local data storage of the edge devices without revealing the
sensitive data to any entities. While this paradigm partly mitigates the
privacy issues of users' sensitive data, the performance of the FL process can
be threatened and reached a bottleneck due to the growing cyber threats and
privacy violation techniques. To expedite the proliferation of FL process, the
integration of blockchain for FL environments has drawn prolific attention from
the people of academia and industry. Blockchain has the potential to prevent
security and privacy threats with its decentralization, immutability,
consensus, and transparency characteristic. However, if the blockchain
mechanism requires costly computational resources, then the
resource-constrained FL clients cannot be involved in the training. Considering
that, this survey focuses on reviewing the challenges, solutions, and future
directions for the successful deployment of blockchain in resource-constrained
FL environments. We comprehensively review variant blockchain mechanisms that
are suitable for FL process and discuss their trade-offs for a limited resource
budget. Further, we extensively analyze the cyber threats that could be
observed in a resource-constrained FL environment, and how blockchain can play
a key role to block those cyber attacks. To this end, we highlight some
potential solutions towards the coupling of blockchain and federated learning
that can offer high levels of reliability, data privacy, and distributed
computing performance
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions
The intersection of the Foundation Model (FM) and Federated Learning (FL)
provides mutual benefits, presents a unique opportunity to unlock new
possibilities in AI research, and address critical challenges in AI and
real-world applications. FL expands the availability of data for FMs and
enables computation sharing, distributing the training process and reducing the
burden on FL participants. It promotes collaborative FM development,
democratizing the process and fostering inclusivity and innovation. On the
other hand, FM, with its enormous size, pre-trained knowledge, and exceptional
performance, serves as a robust starting point for FL, facilitating faster
convergence and better performance under non-iid data. Additionally, leveraging
FM to generate synthetic data enriches data diversity, reduces overfitting, and
preserves privacy. By examining the interplay between FL and FM, this paper
aims to deepen the understanding of their synergistic relationship,
highlighting the motivations, challenges, and future directions. Through an
exploration of the challenges faced by FL and FM individually and their
interconnections, we aim to inspire future research directions that can further
enhance both fields, driving advancements and propelling the development of
privacy-preserving and scalable AI systems
Towards Fairness-Aware Federated Learning
Recent advances in Federated Learning (FL) have brought large-scale
collaborative machine learning opportunities for massively distributed clients
with performance and data privacy guarantees. However, most current works focus
on the interest of the central controller in FL,and overlook the interests of
the FL clients. This may result in unfair treatment of clients which
discourages them from actively participating in the learning process and
damages the sustainability of the FL ecosystem. Therefore, the topic of
ensuring fairness in FL is attracting a great deal of research interest. In
recent years, diverse Fairness-Aware FL (FAFL) approaches have been proposed in
an effort to achieve fairness in FL from different perspectives. However, there
is no comprehensive survey which helps readers gain insight into this
interdisciplinary field. This paper aims to provide such a survey. By examining
the fundamental and simplifying assumptions, as well as the notions of fairness
adopted by existing literature in this field, we propose a taxonomy of FAFL
approaches covering major steps in FL, including client selection,
optimization, contribution evaluation and incentive distribution. In addition,
we discuss the main metrics for experimentally evaluating the performance of
FAFL approaches, and suggest promising future research directions towards
fairness-aware federated learning.Comment: 16 pages, 4 figure
A Comprehensive Survey On Client Selections in Federated Learning
Federated Learning (FL) is a rapidly growing field in machine learning that
allows data to be trained across multiple decentralized devices. The selection
of clients to participate in the training process is a critical factor for the
performance of the overall system. In this survey, we provide a comprehensive
overview of the state-of-the-art client selection techniques in FL, including
their strengths and limitations, as well as the challenges and open issues that
need to be addressed. We cover conventional selection techniques such as random
selection where all or partial random of clients is used for the trained. We
also cover performance-aware selections and as well as resource-aware
selections for resource-constrained networks and heterogeneous networks. We
also discuss the usage of client selection in model security enhancement.
Lastly, we discuss open issues and challenges related to clients selection in
dynamic constrained, and heterogeneous networks
- …