8 research outputs found
SplitFed: When Federated Learning Meets Split Learning
Federated learning (FL) and split learning (SL) are two recent distributed
machine learning (ML) approaches that have gained attention due to their
inherent privacy-preserving capabilities. Both approaches follow a
model-to-data scenario, in that an ML model is sent to clients for network
training and testing. However, FL and SL show contrasting strengths and
weaknesses. For example, while FL performs faster than SL due to its parallel
client-side model generation strategy, SL provides better privacy than FL due
to the ML model architecture split between clients and the server. In contrast
to FL, SL enables ML training with clients having low computing resources as
the client trains only the first few layers of the split ML network model. In
this paper, we present a novel approach, named splitfed (SFL), that amalgamates
the two approaches eliminating their inherent drawbacks. SFL splits the network
architecture between the clients and server as in SL to provide a higher level
of privacy than FL. Moreover, it offers better efficiency than SL by
incorporating the parallel ML model update paradigm of FL. Our empirical
results, on uniformly distributed horizontally partitioned HAM10000 and MNIST
datasets with multiple clients, show that SFL provides similar communication
efficiency and test accuracy as SL, while significantly decreasing - by four to
six times - its computation time per global epoch than in SL for both datasets.
Furthermore, as in SL, its communication efficiency over FL improves with the
number of clients. To further enhance privacy, we integrate a differentially
private local model training mechanism to SFL and test its performance on
AlexNet with the MNIST dataset under various privacy levels
FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks
Federated Learning (FL) and Split Learning (SL) are privacy-preserving
Machine-Learning (ML) techniques that enable training ML models over data
distributed among clients without requiring direct access to their raw data.
Existing FL and SL approaches work on horizontally or vertically partitioned
data and cannot handle sequentially partitioned data where segments of
multiple-segment sequential data are distributed across clients. In this paper,
we propose a novel federated split learning framework, FedSL, to train models
on distributed sequential data. The most common ML models to train on
sequential data are Recurrent Neural Networks (RNNs). Since the proposed
framework is privacy preserving, segments of multiple-segment sequential data
cannot be shared between clients or between clients and server. To circumvent
this limitation, we propose a novel SL approach tailored for RNNs. A RNN is
split into sub-networks, and each sub-network is trained on one client
containing single segments of multiple-segment training sequences. During local
training, the sub-networks on different clients communicate with each other to
capture latent dependencies between consecutive segments of multiple-segment
sequential data on different clients, but without sharing raw data or complete
model parameters. After training local sub-networks with local sequential data
segments, all clients send their sub-networks to a federated server where
sub-networks are aggregated to generate a global model. The experimental
results on simulated and real-world datasets demonstrate that the proposed
method successfully train models on distributed sequential data, while
preserving privacy, and outperforms previous FL and centralized learning
approaches in terms of achieving higher accuracy in fewer communication rounds