FedALA: Adaptive Local Aggregation for Personalized Federated Learning

A key challenge in federated learning (FL) is the statistical heterogeneity that impairs the generalization of the global model on each client. To address this, we propose a method Federated learning with Adaptive Local Aggregation (FedALA) by capturing the desired information in the global model for client models in personalized FL. The key component of FedALA is an Adaptive Local Aggregation (ALA) module, which can adaptively aggregate the downloaded global model and local model towards the local objective on each client to initialize the local model before training in each iteration. To evaluate the effectiveness of FedALA, we conduct extensive experiments with five benchmark datasets in computer vision and natural language processing domains. FedALA outperforms eleven state-of-the-art baselines by up to 3.27% in test accuracy. Furthermore, we also apply ALA module to other federated learning methods and achieve up to 24.19% improvement in test accuracy.Comment: Accepted by AAAI 202

FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy

Recently, personalized federated learning (pFL) has attracted increasing attention in privacy protection, collaborative learning, and tackling statistical heterogeneity among clients, e.g., hospitals, mobile smartphones, etc. Most existing pFL methods focus on exploiting the global information and personalized information in the client-level model parameters while neglecting that data is the source of these two kinds of information. To address this, we propose the Federated Conditional Policy (FedCP) method, which generates a conditional policy for each sample to separate the global information and personalized information in its features and then processes them by a global head and a personalized head, respectively. FedCP is more fine-grained to consider personalization in a sample-specific manner than existing pFL methods. Extensive experiments in computer vision and natural language processing domains show that FedCP outperforms eleven state-of-the-art methods by up to 6.69%. Furthermore, FedCP maintains its superiority when some clients accidentally drop out, which frequently happens in mobile settings. Our code is public at https://github.com/TsingZ0/FedCP.Comment: Accepted by KDD 202

GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning

Federated Learning (FL) is popular for its privacy-preserving and collaborative learning capabilities. Recently, personalized FL (pFL) has received attention for its ability to address statistical heterogeneity and achieve personalization in FL. However, from the perspective of feature extraction, most existing pFL methods only focus on extracting global or personalized feature information during local training, which fails to meet the collaborative learning and personalization goals of pFL. To address this, we propose a new pFL method, named GPFL, to simultaneously learn global and personalized feature information on each client. We conduct extensive experiments on six datasets in three statistically heterogeneous settings and show the superiority of GPFL over ten state-of-the-art methods regarding effectiveness, scalability, fairness, stability, and privacy. Besides, GPFL mitigates overfitting and outperforms the baselines by up to 8.99% in accuracy.Comment: Accepted by ICCV202

Eliminating Domain Bias for Federated Learning in Representation Space

Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE.Comment: Accepted by NeurIPS 2023, 24 page

Tracking-assisted Weakly Supervised Online Visual Object Segmentation in Unconstrained Videos

This paper tackles the task of online video object segmentation with weak supervision, i.e., labeling the target object and background with pixel-level accuracy in unconstrained videos, given only one bounding box information in the first frame. We present a novel tracking-assisted visual object segmentation framework to achieve this. On the one hand, initialized with a given bounding box in the first frame, the auxiliary object tracking module guides the segmentation module frame by frame by providing motion and region information, which is usually missing in semi-supervised methods. Moreover, compared with the unsupervised approach, our approach with such minimum supervision can focus on the target object without bringing unrelated objects into the final results. On the other hand, the video object segmentation module also improves the robustness of the visual object tracking module by pixel-level localization and objectness information. Thus, segmentation and tracking in our framework can mutually help each other in an online manner. To verify the generality and effectiveness of the proposed framework, we evaluate our weakly supervised method on two cross-domain datasets, i.e., the DAVIS and VOT2016 datasets, with the same configuration and parameter setting. Experimental results show the top performance of our method, which is even better than the leading semi-supervised methods. Furthermore, we conduct the extensive ablation study on our approach to investigate the influence of each component and main parameters

Adult Type 3 Adenylyl Cyclase–Deficient Mice Are Obese

Background: A recent study of obesity in Swedish men found that polymorphisms in the type 3 adenylyl cyclase (AC3) are associated with obesity, suggesting the interesting possibility that AC3 may play a role in weight control. Therefore, we examined the weight of AC3 mice over an extended period of time. Methodology/Principal Findings: We discovered that AC3 2/2 mice become obese as they age. Adult male AC3 2/2 mice are about 40 % heavier than wild type male mice while female AC3 2/2 are 70 % heavier. The additional weight of AC3 2/2 mice is due to increased fat mass and larger adipocytes. Before the onset of obesity, young AC3 2/2 mice exhibit reduced physical activity, increased food consumption, and leptin insensitivity. Surprisingly, the obesity of AC3 2/2 mice is not due to a loss of AC3 from white adipose and a decrease in lipolysis. Conclusions/Significance: We conclude that mice lacking AC3 exhibit obesity that is apparently caused by low locomotor activity, hyperphagia, and leptin insensitivity. The presence of AC3 in primary cilia of neurons of the hypothalamus suggests that cAMP signals generated by AC3 in the hypothalamus may play a critical role in regulation of body weight

ERK5 MAP Kinase Regulates Neurogenin1 during Cortical Neurogenesis

The commitment of multi-potent cortical progenitors to a neuronal fate depends on the transient induction of the basic-helix-loop-helix (bHLH) family of transcription factors including Neurogenin 1 (Neurog1). Previous studies have focused on mechanisms that control the expression of these proteins while little is known about whether their pro-neural activities can be regulated by kinase signaling pathways. Using primary cultures and ex vivo slice cultures, here we report that both the transcriptional and pro-neural activities of Neurog1 are regulated by extracellular signal-regulated kinase (ERK) 5 signaling in cortical progenitors. Activation of ERK5 potentiated, while blocking ERK5 inhibited Neurog1-induced neurogenesis. Furthermore, endogenous ERK5 activity was required for Neurog1-initiated transcription. Interestingly, ERK5 activation was sufficient to induce Neurog1 phosphorylation and ERK5 directly phosphorylated Neurog1 in vitro. We identified S179/S208 as putative ERK5 phosphorylation sites in Neurog1. Mutations of S179/S208 to alanines inhibited the transcriptional and pro-neural activities of Neurog1. Our data identify ERK5 phosphorylation of Neurog1 as a novel mechanism regulating neuronal fate commitment of cortical progenitors