1 research outputs found
SphereFed: Hyperspherical Federated Learning
Federated Learning aims at training a global model from multiple
decentralized devices (i.e. clients) without exchanging their private local
data. A key challenge is the handling of non-i.i.d. (independent identically
distributed) data across multiple clients that may induce disparities of their
local features. We introduce the Hyperspherical Federated Learning (SphereFed)
framework to address the non-i.i.d. issue by constraining learned
representations of data points to be on a unit hypersphere shared by clients.
Specifically, all clients learn their local representations by minimizing the
loss with respect to a fixed classifier whose weights span the unit
hypersphere. After federated training in improving the global model, this
classifier is further calibrated with a closed-form solution by minimizing a
mean squared loss. We show that the calibration solution can be computed
efficiently and distributedly without direct access of local data. Extensive
experiments indicate that our SphereFed approach is able to improve the
accuracy of multiple existing federated learning algorithms by a considerable
margin (up to 6% on challenging datasets) with enhanced computation and
communication efficiency across datasets and model architectures.Comment: European Conference on Computer Vision 202