6 research outputs found
Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis
A wireless federated learning system is investigated by allowing a server and
workers to exchange uncoded information via orthogonal wireless channels. Since
the workers frequently upload local gradients to the server via
bandwidth-limited channels, the uplink transmission from the workers to the
server becomes a communication bottleneck. Therefore, a one-shot distributed
principle component analysis (PCA) is leveraged to reduce the dimension of
uploaded gradients such that the communication bottleneck is relieved. A
PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated
version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients
and the Nesterov's momentum. For the non-convex loss functions, a finite-time
analysis is performed to quantify the impacts of system hyper-parameters on the
convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is
theoretically certified to converge faster than the PCA-WFL algorithm. Besides,
the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal
the linear speedup with respect to the number of workers over the vanilla
gradient descent algorithm. Numerical results are used to demonstrate the
improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over
the benchmarks