5 research outputs found
Federated Learning with Downlink Device Selection
We study federated edge learning, where a global model is trained
collaboratively using privacy-sensitive data at the edge of a wireless network.
A parameter server (PS) keeps track of the global model and shares it with the
wireless edge devices for training using their private local data. The devices
then transmit their local model updates, which are used to update the global
model, to the PS. The algorithm, which involves transmission over PS-to-device
and device-to-PS links, continues until the convergence of the global model or
lack of any participating devices. In this study, we consider device selection
based on downlink channels over which the PS shares the global model with the
devices. Performing digital downlink transmission, we design a partial device
participation framework where a subset of the devices is selected for training
at each iteration. Therefore, the participating devices can have a better
estimate of the global model compared to the full device participation case
which is due to the shared nature of the broadcast channel with the price of
updating the global model with respect to a smaller set of data. At each
iteration, the PS broadcasts different quantized global model updates to
different participating devices based on the last global model estimates
available at the devices. We investigate the best number of participating
devices through experimental results for image classification using the MNIST
dataset with biased distribution.Comment: accepted in IEEE International Workshop on Signal Processing Advances
in Wireless Communications (SPAWC), 202
XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning
User-generated data distributions are often imbalanced across devices and
labels, hampering the performance of federated learning (FL). To remedy to this
non-independent and identically distributed (non-IID) data problem, in this
work we develop a privacy-preserving XOR based mixup data augmentation
technique, coined XorMixup, and thereby propose a novel one-shot FL framework,
termed XorMixFL. The core idea is to collect other devices' encoded data
samples that are decoded only using each device's own data samples. The
decoding provides synthetic-but-realistic samples until inducing an IID
dataset, used for model training. Both encoding and decoding procedures follow
the bit-wise XOR operations that intentionally distort raw samples, thereby
preserving data privacy. Simulation results corroborate that XorMixFL achieves
up to 17.6% higher accuracy than Vanilla FL under a non-IID MNIST dataset
Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data
This study develops a federated learning (FL) framework overcoming largely
incremental communication costs due to model sizes in typical frameworks
without compromising model performance. To this end, based on the idea of
leveraging an unlabeled open dataset, we propose a distillation-based
semi-supervised FL (DS-FL) algorithm that exchanges the outputs of local models
among mobile devices, instead of model parameter exchange employed by the
typical frameworks. In DS-FL, the communication cost depends only on the output
dimensions of the models and does not scale up according to the model size. The
exchanged model outputs are used to label each sample of the open dataset,
which creates an additionally labeled dataset. Based on the new dataset, local
models are further trained, and model performance is enhanced owing to the data
augmentation effect. We further highlight that in DS-FL, the heterogeneity of
the devices' dataset leads to ambiguous of each data sample and lowing of the
training convergence. To prevent this, we propose entropy reduction averaging,
where the aggregated model outputs are intentionally sharpened. Moreover,
extensive experiments show that DS-FL reduces communication costs up to 99%
relative to those of the FL benchmark while achieving similar or higher
classification accuracy
Convergence of Federated Learning over a Noisy Downlink
We study federated learning (FL), where power-limited wireless devices
utilize their local datasets to collaboratively train a global model with the
help of a remote parameter server (PS). The PS has access to the global model
and shares it with the devices for local training, and the devices return the
result of their local updates to the PS to update the global model. This
framework requires downlink transmission from the PS to the devices and uplink
transmission from the devices to the PS. The goal of this study is to
investigate the impact of the bandwidth-limited shared wireless medium in both
the downlink and uplink on the performance of FL with a focus on the downlink.
To this end, the downlink and uplink channels are modeled as fading broadcast
and multiple access channels, respectively, both with limited bandwidth. For
downlink transmission, we first introduce a digital approach, where a
quantization technique is employed at the PS to broadcast the global model
update at a common rate such that all the devices can decode it. Next, we
propose analog downlink transmission, where the global model is broadcast by
the PS in an uncoded manner. We consider analog transmission over the uplink in
both cases. We further analyze the convergence behavior of the proposed analog
approach assuming that the uplink transmission is error-free. Numerical
experiments show that the analog downlink approach provides significant
improvement over the digital one, despite a significantly lower transmit power
at the PS. The experimental results corroborate the convergence results, and
show that a smaller number of local iterations should be used when the data
distribution is more biased, and also when the devices have a better estimate
of the global model in the analog downlink approach.Comment: submitted for publicatio
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM
In this paper, we propose a communication-efficiently decentralized machine
learning framework that solves a consensus optimization problem defined over a
network of inter-connected workers. The proposed algorithm, Censored and
Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and
decentralized learning ideas of Group Alternating Direction Method of
Multipliers (GADMM), and pushes the frontier in communication efficiency by
extending its applicability to generalized network topologies, while
incorporating link censoring for negligible updates after quantization. We
theoretically prove that CQ-GGADMM achieves the linear convergence rate when
the local objective functions are strongly convex under some mild assumptions.
Numerical simulations corroborate that CQ-GGADMM exhibits higher communication
efficiency in terms of the number of communication rounds and transmit energy
consumption without compromising the accuracy and convergence speed, compared
to the censored decentralized ADMM, and the worker grouping method of GADMM.Comment: 14 pages, 5 figure