Efficient Personalized Federated Learning via Sparse Model-Adaptation

Federated Learning (FL) aims to train machine learning models for multiple clients without sharing their own private data. Due to the heterogeneity of clients' local data distribution, recent studies explore the personalized FL that learns and deploys distinct local models with the help of auxiliary global models. However, the clients can be heterogeneous in terms of not only local data distribution, but also their computation and communication resources. The capacity and efficiency of personalized models are restricted by the lowest-resource clients, leading to sub-optimal performance and limited practicality of personalized FL. To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models. With a lightweight trainable gating layer, pFedGate enables clients to reach their full potential in model capacity by generating different sparse models accounting for both the heterogeneous data distributions and resource constraints. Meanwhile, the computation and communication efficiency are both improved thanks to the adaptability between the model sparsity and clients' resources. Further, we theoretically show that the proposed pFedGate has superior complexity with guaranteed convergence and generalization error. Extensive experiments show that pFedGate achieves superior global accuracy, individual accuracy and efficiency simultaneously over state-of-the-art methods. We also demonstrate that pFedGate performs better than competitors in the novel clients participation and partial clients participation scenarios, and can learn meaningful sparse local models adapted to different data distributions.Comment: Accepted to ICML 202

Revisiting Personalized Federated Learning: Robustness Against Backdoor Attacks

In this work, besides improving prediction accuracy, we study whether personalization could bring robustness benefits to backdoor attacks. We conduct the first study of backdoor attacks in the pFL framework, testing 4 widely used backdoor attacks against 6 pFL methods on benchmark datasets FEMNIST and CIFAR-10, a total of 600 experiments. The study shows that pFL methods with partial model-sharing can significantly boost robustness against backdoor attacks. In contrast, pFL methods with full model-sharing do not show robustness. To analyze the reasons for varying robustness performances, we provide comprehensive ablation studies on different pFL methods. Based on our findings, we further propose a lightweight defense method, Simple-Tuning, which empirically improves defense performance against backdoor attacks. We believe that our work could provide both guidance for pFL application in terms of its robustness and offer valuable insights to design more robust FL methods in the future. We open-source our code to establish the first benchmark for black-box backdoor attacks in pFL: https://github.com/alibaba/FederatedScope/tree/backdoor-bench.Comment: KDD 202

Matching in Selective and Balanced Representation Space for Treatment Effects Estimation

The dramatically growing availability of observational data is being witnessed in various domains of science and technology, which facilitates the study of causal inference. However, estimating treatment effects from observational data is faced with two major challenges, missing counterfactual outcomes and treatment selection bias. Matching methods are among the most widely used and fundamental approaches to estimating treatment effects, but existing matching methods have poor performance when facing data with high dimensional and complicated variables. We propose a feature selection representation matching (FSRM) method based on deep representation learning and matching, which maps the original covariate space into a selective, nonlinear, and balanced representation space, and then conducts matching in the learned representation space. FSRM adopts deep feature selection to minimize the influence of irrelevant variables for estimating treatment effects and incorporates a regularizer based on the Wasserstein distance to learn balanced representations. We evaluate the performance of our FSRM method on three datasets, and the results demonstrate superiority over the state-of-the-art methods.Comment: Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM '20

DA-FSOD: A Novel Data Augmentation Scheme for Few-Shot Object Detection

Deep learning techniques continue to be used in various applications in recent years. However, when it is difficult to obtain adequate training samples, the performance of the depth model will degrade. Although few-shot learning and data enhancement techniques can relieve this dilemma, the diversity of real data is too large to simulate. To tackle this challenge, we study a novel method, Data Augmentation Scheme For Few-Shot Object Detection (DA-FSOD), to improve the efficiency of model training on visual tasks. Specifically, to expand data augmentation space, we build a data augmentation operation pool (DAOP) based on several common-applied image process operations. Then we propose a novel data augmentation scheme, the series and parallel connection scheme, which superimposes the effects of different operations to generate diverse variants. To further explore and utilize the deep feature information, we leverage the semantic information of input image in model and propose imposed semantic data augmentation which augments training set semantically via deep features of augmented variants. The proposed method successfully enhanced the model performance. We validated our approach using extensive experiments on the domain of few-shot object detection. The results showed remarkable gains compared to state-of-the-art methods

FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning

The incredible development of federated learning (FL) has benefited various tasks in the domains of computer vision and natural language processing, and the existing frameworks such as TFF and FATE has made the deployment easy in real-world applications. However, federated graph learning (FGL), even though graph data are prevalent, has not been well supported due to its unique characteristics and requirements. The lack of FGL-related framework increases the efforts for accomplishing reproducible research and deploying in real-world applications. Motivated by such strong demand, in this paper, we first discuss the challenges in creating an easy-to-use FGL package and accordingly present our implemented package FederatedScope-GNN (FS-G), which provides (1) a unified view for modularizing and expressing FGL algorithms; (2) comprehensive DataZoo and ModelZoo for out-of-the-box FGL capability; (3) an efficient model auto-tuning component; and (4) off-the-shelf privacy attack and defense abilities. We validate the effectiveness of FS-G by conducting extensive experiments, which simultaneously gains many valuable insights about FGL for the community. Moreover, we employ FS-G to serve the FGL application in real-world E-commerce scenarios, where the attained improvements indicate great potential business benefits. We publicly release FS-G, as submodules of FederatedScope, at https://github.com/alibaba/FederatedScope to promote FGL's research and enable broad applications that would otherwise be infeasible due to the lack of a dedicated package.Comment: Accpeted by KDD'2022; We have released FederatedScope for users on https://github.com/alibaba/FederatedScop

Analysis of Air Purification Methods in Operating Rooms of Chinese Hospitals

This research demonstrates the current use of air purification methods in the operating rooms (ORs) in China. 154 hospitals from 6 provinces were included in this survey to reflect the air purification methods of ORs in 2017. Air cleaning technology (ACT) is used in 124 (80.52%) hospitals. We find that the rates of using grade I, III, or IV clean operating room (COR) in tertiary hospitals are all higher than in lower level hospitals; the rate of using ACT in the ORs is higher, too. In addition, general hospitals have higher rate in using ACT in the ORs than specialized hospitals. The highest rate of using ACT in the ORs is in the eastern region of China. The number of hospitals using ACT, ultraviolet light disinfection, and air sterilizers (such as circulating air UV sterilizer) increased yearly. All grades of CORs can be maintained as required by more than 90% hospitals except grade II COR. In this research, we found air purification methods, especially the ACT, are widely used in hospitals’ ORs. However, finding the way to select and use different air purification methods correctly is an urgent problem to be solved next

Characteristics and potential exposure risks of environmentally persistent free radicals in PM2.5 in the three gorges reservoir area, Southwestern China

Environmentally persistent free radicals (EPFRs) are a novel class of hazardous substances that can exist stably in airborne particles for a period ranging from days to weeks and are potentially toxic to human health. Electron paramagnetic resonance spectroscopy (EPR) was used to characterize particulate EPFRs in Wanzhou in the Three Gorges Reservoir area in 2017. During the whole of 2017, the average concentration of particulate EPFRs was 7.0 x 10(13) +/- 1.7 x 10(13) spins/m(3). The seasonal concentration of EPFRs in PM2.5 showed a trend of autumn > winter > spring > summer. The maxima and minima of EPFRs occurred in spring with concentrations of 2.1 x 10(14) spins/m(3 )and 9.4 x 10(12) spins/m(3)respectively. The EPFRs in PM2.5 were mainly carbon-centered radicals with adjacent oxygen atoms. Significant positive correlations were found between EPFRs and SO42-, NO3- and NH4+ (r> 0.55, n = 111), indicating that EPFRs are associated with secondary sources. The atmospheric processing of particles from coal combustion, traffic, and agriculture were important sources of EPFRs. They were also particularly well correlated with K+ and Cl- in winter, suggesting that EPFRs may also be derived from wintertime biomass burning emissions. The amount of inhalable EPFRs in Wanzhou was equivalent to the range of 2.3-6.8 cigarettes per capita per day. This study provides evidence of the potential health risks of EPFRs in PM2.5, and references for air pollution control in the Three Gorges Reservoir area. (C) 2020 Elsevier Ltd. All rights reserved