37 research outputs found
Personalized Federated Learning with Multi-branch Architecture
Federated learning (FL) is a decentralized machine learning technique that
enables multiple clients to collaboratively train models without requiring
clients to reveal their raw data to each other. Although traditional FL trains
a single global model with average performance among clients, statistical data
heterogeneity across clients has resulted in the development of personalized FL
(PFL), which trains personalized models with good performance on each client's
data. A key challenge with PFL is how to facilitate clients with similar data
to collaborate more in a situation where each client has data from complex
distribution and cannot determine one another's distribution. In this paper, we
propose a new PFL method (pFedMB) using multi-branch architecture, which
achieves personalization by splitting each layer of a neural network into
multiple branches and assigning client-specific weights to each branch. We also
design an aggregation method to improve the communication efficiency and the
model performance, with which each branch is globally updated with weighted
averaging by client-specific weights assigned to the branch. pFedMB is simple
but effective in facilitating each client to share knowledge with similar
clients by adjusting the weights assigned to each branch. We experimentally
show that pFedMB performs better than the state-of-the-art PFL methods using
the CIFAR10 and CIFAR100 datasets.Comment: Accepted by IJCNN 202
Heterogeneous Domain Adaptation with Positive and Unlabeled Data
Heterogeneous unsupervised domain adaptation (HUDA) is the most challenging
domain adaptation setting where the feature spaces of source and target domains
are heterogeneous, and the target domain has only unlabeled data. Existing HUDA
methods assume that both positive and negative examples are available in the
source domain, which may not be satisfied in some real applications. This paper
addresses a new challenging setting called positive and unlabeled heterogeneous
unsupervised domain adaptation (PU-HUDA), a HUDA setting where the source
domain only has positives. PU-HUDA can also be viewed as an extension of PU
learning where the positive and unlabeled examples are sampled from different
domains. A naive combination of existing HUDA and PU learning methods is
ineffective in PU-HUDA due to the gap in label distribution between the source
and target domains. To overcome this issue, we propose a novel method,
predictive adversarial domain adaptation (PADA), which can predict likely
positive examples from the unlabeled target data and simultaneously align the
feature spaces to reduce the distribution divergence between the whole source
data and the likely positive target data. PADA achieves this by a unified
adversarial training framework for learning a classifier to predict positive
examples and a feature transformer to transform the target feature space to
that of the source. Specifically, they are both trained to fool a common
discriminator that determines whether the likely positive examples are from the
target or source domain. We experimentally show that PADA outperforms several
baseline methods, such as the naive combination of HUDA and PU learning.Comment: Accepted by IEEE Big Data 2023 as a regular pape