377 research outputs found
Decentralized Federated Learning: Fundamentals, State-of-the-art, Frameworks, Trends, and Challenges
In the last decade, Federated Learning (FL) has gained relevance in training
collaborative models without sharing sensitive data. Since its birth,
Centralized FL (CFL) has been the most common approach in the literature, where
a central entity creates a global model. However, a centralized approach leads
to increased latency due to bottlenecks, heightened vulnerability to system
failures, and trustworthiness concerns affecting the entity responsible for the
global model creation. Decentralized Federated Learning (DFL) emerged to
address these concerns by promoting decentralized model aggregation and
minimizing reliance on centralized architectures. However, despite the work
done in DFL, the literature has not (i) studied the main aspects
differentiating DFL and CFL; (ii) analyzed DFL frameworks to create and
evaluate new solutions; and (iii) reviewed application scenarios using DFL.
Thus, this article identifies and analyzes the main fundamentals of DFL in
terms of federation architectures, topologies, communication mechanisms,
security approaches, and key performance indicators. Additionally, the paper at
hand explores existing mechanisms to optimize critical DFL fundamentals. Then,
the most relevant features of the current DFL frameworks are reviewed and
compared. After that, it analyzes the most used DFL application scenarios,
identifying solutions based on the fundamentals and frameworks previously
defined. Finally, the evolution of existing DFL solutions is studied to provide
a list of trends, lessons learned, and open challenges
Concept-aware clustering for decentralized deep learning under temporal shift
Decentralized deep learning requires dealing with non-iid data across
clients, which may also change over time due to temporal shifts. While non-iid
data has been extensively studied in distributed settings, temporal shifts have
received no attention. To the best of our knowledge, we are first with tackling
the novel and challenging problem of decentralized learning with non-iid and
dynamic data. We propose a novel algorithm that can automatically discover and
adapt to the evolving concepts in the network, without any prior knowledge or
estimation of the number of concepts. We evaluate our algorithm on standard
benchmark datasets and demonstrate that it outperforms previous methods for
decentralized learning.Comment: 4 pages, 2 figure
On the (In)security of Peer-to-Peer Decentralized Machine Learning
In this work, we carry out the first, in-depth, privacy analysis of
Decentralized Learning -- a collaborative machine learning framework aimed at
addressing the main limitations of federated learning. We introduce a suite of
novel attacks for both passive and active decentralized adversaries. We
demonstrate that, contrary to what is claimed by decentralized learning
proposers, decentralized learning does not offer any security advantage over
federated learning. Rather, it increases the attack surface enabling any user
in the system to perform privacy attacks such as gradient inversion, and even
gain full control over honest users' local model. We also show that, given the
state of the art in protections, privacy-preserving configurations of
decentralized learning require fully connected networks, losing any practical
advantage over the federated setup and therefore completely defeating the
objective of the decentralized approach.Comment: IEEE S&P'23 (Previous title: "On the Privacy of Decentralized Machine
Learning"
Gossip Consensus Algorithm Based on Time-Varying Influence Factors and Weakly Connected Graph for Opinion Evolution in Social Networks
We provide a new gossip algorithm to investigate the problem of opinion consensus with the time-varying influence factors and weakly connected graph among multiple agents. What is more, we discuss not only the effect of the time-varying factors and the randomized topological structure but also the spread of misinformation and communication constrains described by probabilistic quantized communication in the social network. Under the underlying weakly connected graph, we first denote that all opinion states converge to a stochastic consensus almost surely; that is, our algorithm indeed achieves the consensus with probability one. Furthermore, our results show that the mean of all the opinion states converges to the average of the initial states when time-varying influence factors satisfy some conditions. Finally, we give a result about the square mean error between the dynamic opinion states and the benchmark without quantized communication
Hiding in the Crowd: A Massively Distributed Algorithm for Private Averaging with Malicious Adversaries
The amount of personal data collected in our everyday interactions with
connected devices offers great opportunities for innovative services fueled by
machine learning, as well as raises serious concerns for the privacy of
individuals. In this paper, we propose a massively distributed protocol for a
large set of users to privately compute averages over their joint data, which
can then be used to learn predictive models. Our protocol can find a solution
of arbitrary accuracy, does not rely on a third party and preserves the privacy
of users throughout the execution in both the honest-but-curious and malicious
adversary models. Specifically, we prove that the information observed by the
adversary (the set of maliciours users) does not significantly reduce the
uncertainty in its prediction of private values compared to its prior belief.
The level of privacy protection depends on a quantity related to the Laplacian
matrix of the network graph and generally improves with the size of the graph.
Furthermore, we design a verification procedure which offers protection against
malicious users joining the service with the goal of manipulating the outcome
of the algorithm.Comment: 17 page
- …