19 research outputs found
FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning
Federated learning (FL) has emerged as a new paradigm for privacy-preserving
computation in recent years. Unfortunately, FL faces two critical challenges
that hinder its actual performance: data distribution heterogeneity and high
resource costs brought by large foundation models. Specifically, the non-IID
data in different clients make existing FL algorithms hard to converge while
the high resource costs, including computational and communication costs that
increase the deployment difficulty in real-world scenarios. In this paper, we
propose an effective yet simple method, named FedCLIP, to achieve fast
generalization and personalization for CLIP in federated learning. Concretely,
we design an attention-based adapter for the large model, CLIP, and the rest
operations merely depend on adapters. Lightweight adapters can make the most
use of pretrained model information and ensure models be adaptive for clients
in specific tasks. Simultaneously, small-scale operations can mitigate the
computational burden and communication burden caused by large models. Extensive
experiments are conducted on three datasets with distribution shifts.
Qualitative and quantitative results demonstrate that FedCLIP significantly
outperforms other baselines (9% overall improvements on PACS) and effectively
reduces computational and communication costs (283x faster than FedAVG). Our
code will be available at: https://github.com/microsoft/PersonalizedFL.Comment: Accepted by IEEE Data Engineering Bulletin; code is at:
https://github.com/microsoft/PersonalizedF
SCEI: A Smart-Contract Driven Edge Intelligence Framework for IoT Systems
Federated learning (FL) utilizes edge computing devices to collaboratively
train a shared model while each device can fully control its local data access.
Generally, FL techniques focus on learning model on independent and identically
distributed (iid) dataset and cannot achieve satisfiable performance on non-iid
datasets (e.g. learning a multi-class classifier but each client only has a
single class dataset). Some personalized approaches have been proposed to
mitigate non-iid issues. However, such approaches cannot handle underlying data
distribution shift, namely data distribution skew, which is quite common in
real scenarios (e.g. recommendation systems learn user behaviors which change
over time). In this work, we provide a solution to the challenge by leveraging
smart-contract with federated learning to build optimized, personalized deep
learning models. Specifically, our approach utilizes smart contract to reach
consensus among distributed trainers on the optimal weights of personalized
models. We conduct experiments across multiple models (CNN and MLP) and
multiple datasets (MNIST and CIFAR-10). The experimental results demonstrate
that our personalized learning models can achieve better accuracy and faster
convergence compared to classic federated and personalized learning. Compared
with the model given by baseline FedAvg algorithm, the average accuracy of our
personalized learning models is improved by 2% to 20%, and the convergence rate
is about 2 faster. Moreover, we also illustrate that our approach is
secure against recent attack on distributed learning.Comment: 12 pages, 9 figure
Federated Robustness Propagation: Sharing Adversarial Robustness in Federated Learning
Federated learning (FL) emerges as a popular distributed learning schema that
learns a model from a set of participating users without requiring raw data to
be shared. One major challenge of FL comes from heterogeneity in users, which
may have distributionally different (or non-iid) data and varying computation
resources. Just like in centralized learning, FL users also desire model
robustness against malicious attackers at test time. Whereas adversarial
training (AT) provides a sound solution for centralized learning, extending its
usage for FL users has imposed significant challenges, as many users may have
very limited training data as well as tight computational budgets, to afford
the data-hungry and costly AT. In this paper, we study a novel learning setting
that propagates adversarial robustness from high-resource users that can afford
AT, to those low-resource users that cannot afford it, during the FL process.
We show that existing FL techniques cannot effectively propagate adversarial
robustness among non-iid users, and propose a simple yet effective propagation
approach that transfers robustness through carefully designed
batch-normalization statistics. We demonstrate the rationality and
effectiveness of our method through extensive experiments. Especially, the
proposed method is shown to grant FL remarkable robustness even when only a
small portion of users afford AT during learning. Codes will be published upon
acceptance