Efficient Personalized Federated Learning via Sparse Model-Adaptation

Federated Learning (FL) aims to train machine learning models for multiple clients without sharing their own private data. Due to the heterogeneity of clients' local data distribution, recent studies explore the personalized FL that learns and deploys distinct local models with the help of auxiliary global models. However, the clients can be heterogeneous in terms of not only local data distribution, but also their computation and communication resources. The capacity and efficiency of personalized models are restricted by the lowest-resource clients, leading to sub-optimal performance and limited practicality of personalized FL. To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models. With a lightweight trainable gating layer, pFedGate enables clients to reach their full potential in model capacity by generating different sparse models accounting for both the heterogeneous data distributions and resource constraints. Meanwhile, the computation and communication efficiency are both improved thanks to the adaptability between the model sparsity and clients' resources. Further, we theoretically show that the proposed pFedGate has superior complexity with guaranteed convergence and generalization error. Extensive experiments show that pFedGate achieves superior global accuracy, individual accuracy and efficiency simultaneously over state-of-the-art methods. We also demonstrate that pFedGate performs better than competitors in the novel clients participation and partial clients participation scenarios, and can learn meaningful sparse local models adapted to different data distributions.Comment: Accepted to ICML 202

Revisiting Personalized Federated Learning: Robustness Against Backdoor Attacks

In this work, besides improving prediction accuracy, we study whether personalization could bring robustness benefits to backdoor attacks. We conduct the first study of backdoor attacks in the pFL framework, testing 4 widely used backdoor attacks against 6 pFL methods on benchmark datasets FEMNIST and CIFAR-10, a total of 600 experiments. The study shows that pFL methods with partial model-sharing can significantly boost robustness against backdoor attacks. In contrast, pFL methods with full model-sharing do not show robustness. To analyze the reasons for varying robustness performances, we provide comprehensive ablation studies on different pFL methods. Based on our findings, we further propose a lightweight defense method, Simple-Tuning, which empirically improves defense performance against backdoor attacks. We believe that our work could provide both guidance for pFL application in terms of its robustness and offer valuable insights to design more robust FL methods in the future. We open-source our code to establish the first benchmark for black-box backdoor attacks in pFL: https://github.com/alibaba/FederatedScope/tree/backdoor-bench.Comment: KDD 202

Matching in Selective and Balanced Representation Space for Treatment Effects Estimation

The dramatically growing availability of observational data is being witnessed in various domains of science and technology, which facilitates the study of causal inference. However, estimating treatment effects from observational data is faced with two major challenges, missing counterfactual outcomes and treatment selection bias. Matching methods are among the most widely used and fundamental approaches to estimating treatment effects, but existing matching methods have poor performance when facing data with high dimensional and complicated variables. We propose a feature selection representation matching (FSRM) method based on deep representation learning and matching, which maps the original covariate space into a selective, nonlinear, and balanced representation space, and then conducts matching in the learned representation space. FSRM adopts deep feature selection to minimize the influence of irrelevant variables for estimating treatment effects and incorporates a regularizer based on the Wasserstein distance to learn balanced representations. We evaluate the performance of our FSRM method on three datasets, and the results demonstrate superiority over the state-of-the-art methods.Comment: Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM '20

DA-FSOD: A Novel Data Augmentation Scheme for Few-Shot Object Detection

Deep learning techniques continue to be used in various applications in recent years. However, when it is difficult to obtain adequate training samples, the performance of the depth model will degrade. Although few-shot learning and data enhancement techniques can relieve this dilemma, the diversity of real data is too large to simulate. To tackle this challenge, we study a novel method, Data Augmentation Scheme For Few-Shot Object Detection (DA-FSOD), to improve the efficiency of model training on visual tasks. Specifically, to expand data augmentation space, we build a data augmentation operation pool (DAOP) based on several common-applied image process operations. Then we propose a novel data augmentation scheme, the series and parallel connection scheme, which superimposes the effects of different operations to generate diverse variants. To further explore and utilize the deep feature information, we leverage the semantic information of input image in model and propose imposed semantic data augmentation which augments training set semantically via deep features of augmented variants. The proposed method successfully enhanced the model performance. We validated our approach using extensive experiments on the domain of few-shot object detection. The results showed remarkable gains compared to state-of-the-art methods

EC-GAN: Inferring Brain Effective Connectivity via Generative Adversarial Networks

Inferring effective connectivity between different brain regions from functional magnetic resonance imaging (fMRI) data is an important advanced study in neuroinformatics in recent years. However, current methods have limited usage in effective connectivity studies due to the high noise and small sample size of fMRI data. In this paper, we propose a novel framework for inferring effective connectivity based on generative adversarial networks (GAN), named as EC-GAN. The proposed framework EC-GAN infers effective connectivity via an adversarial process, in which we simultaneously train two models: a generator and a discriminator. The generator consists of a set of effective connectivity generators based on structural equation models which can generate the fMRI time series of each brain region via effective connectivity. Meanwhile, the discriminator is employed to distinguish between the joint distributions of the real and generated fMRI time series. Experimental results on simulated data show that EC-GAN can better infer effective connectivity compared to other state-of-the-art methods. The real-world experiments indicate that EC-GAN can provide a new and reliable perspective analyzing the effective connectivity of fMRI data

The effects of family structure and function on mental health during China’s transition: a cross-sectional analysis

Abstract Background Social change, intensified by industrialization and globalization, has not only changed people’s work lives but also their personal lives, especially in developing countries. The aim of this study was to provide evidence and recommendations regarding family structure, function, and mental health to actively respond to rapid social change. Methods A cross-sectional survey was conducted face-to-face and door-to-door from July 2011 to September 2012 in Hubei Province, central China. Family structure comprised alone, couple, nuclear family, and extended family; family function was measured using the family APGAR (Adaptation, Partnership, Growth, Affection, and Resolve) scale, and mental health was measured using the Chinese version of the 12-item General Health Questionnaire (GHQ-12). Results The urban-vs-rural difference of family structure among alone, couple, nuclear family, and extended family was statistically significant (5.21% vs 4.62%; 27.36% vs 13.14%; 33.22% vs 27.74%; 34.20% vs 54.50%, respectively; p < 0.0001); and those difference of family function was not statistically significant (8.11 ± 2.13 vs 8.09 ± 2.27, p = 0.9372). The general linear regression showed that the effect of family structure on mental health, whether urban or rural, was not significant, however, the effect of family function was significant, especially regarding better family functioning with better mental health. Conclusions Combined the effects of family structure and function on mental health, the external form of family (family structure) may not be important; while the internal quality of role (family function) might be key. Improving the residents’ family function would be a priority strategy for family practice with their mental health

FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning

The incredible development of federated learning (FL) has benefited various tasks in the domains of computer vision and natural language processing, and the existing frameworks such as TFF and FATE has made the deployment easy in real-world applications. However, federated graph learning (FGL), even though graph data are prevalent, has not been well supported due to its unique characteristics and requirements. The lack of FGL-related framework increases the efforts for accomplishing reproducible research and deploying in real-world applications. Motivated by such strong demand, in this paper, we first discuss the challenges in creating an easy-to-use FGL package and accordingly present our implemented package FederatedScope-GNN (FS-G), which provides (1) a unified view for modularizing and expressing FGL algorithms; (2) comprehensive DataZoo and ModelZoo for out-of-the-box FGL capability; (3) an efficient model auto-tuning component; and (4) off-the-shelf privacy attack and defense abilities. We validate the effectiveness of FS-G by conducting extensive experiments, which simultaneously gains many valuable insights about FGL for the community. Moreover, we employ FS-G to serve the FGL application in real-world E-commerce scenarios, where the attained improvements indicate great potential business benefits. We publicly release FS-G, as submodules of FederatedScope, at https://github.com/alibaba/FederatedScope to promote FGL's research and enable broad applications that would otherwise be infeasible due to the lack of a dedicated package.Comment: Accpeted by KDD'2022; We have released FederatedScope for users on https://github.com/alibaba/FederatedScop

FederatedScope: A Flexible Federated Learning Platform for Heterogeneity

Although remarkable progress has been made by existing federated learning (FL) platforms to provide infrastructures for development, these platforms may not well tackle the challenges brought by various types of heterogeneity, including the heterogeneity in participants' local data, resources, behaviors and learning goals. To fill this gap, in this paper, we propose a novel FL platform, named FederatedScope, which employs an event-driven architecture to provide users with great flexibility to independently describe the behaviors of different participants. Such a design makes it easy for users to describe participants with various local training processes, learning goals and backends, and coordinate them into an FL course with synchronous or asynchronous training strategies. Towards an easy-to-use and flexible platform, FederatedScope enables rich types of plug-in operations and components for efficient further development, and we have implemented several important components to better help users with privacy protection, attack simulation and auto-tuning. We have released FederatedScope at https://github.com/alibaba/FederatedScope to promote academic research and industrial deployment of federated learning in a wide range of scenarios.Comment: We have released FederatedScope for users on https://github.com/alibaba/FederatedScop