37,281 research outputs found
On the (In)security of Peer-to-Peer Decentralized Machine Learning
In this work, we carry out the first, in-depth, privacy analysis of
Decentralized Learning -- a collaborative machine learning framework aimed at
addressing the main limitations of federated learning. We introduce a suite of
novel attacks for both passive and active decentralized adversaries. We
demonstrate that, contrary to what is claimed by decentralized learning
proposers, decentralized learning does not offer any security advantage over
federated learning. Rather, it increases the attack surface enabling any user
in the system to perform privacy attacks such as gradient inversion, and even
gain full control over honest users' local model. We also show that, given the
state of the art in protections, privacy-preserving configurations of
decentralized learning require fully connected networks, losing any practical
advantage over the federated setup and therefore completely defeating the
objective of the decentralized approach.Comment: IEEE S&P'23 (Previous title: "On the Privacy of Decentralized Machine
Learning"
Stability and Generalization of the Decentralized Stochastic Gradient Descent
The stability and generalization of stochastic gradient-based methods provide
valuable insights into understanding the algorithmic performance of machine
learning models. As the main workhorse for deep learning, stochastic gradient
descent has received a considerable amount of studies. Nevertheless, the
community paid little attention to its decentralized variants. In this paper,
we provide a novel formulation of the decentralized stochastic gradient
descent. Leveraging this formulation together with (non)convex optimization
theory, we establish the first stability and generalization guarantees for the
decentralized stochastic gradient descent. Our theoretical results are built on
top of a few common and mild assumptions and reveal that the decentralization
deteriorates the stability of SGD for the first time. We verify our theoretical
findings by using a variety of decentralized settings and benchmark machine
learning models
- …