Multi-Level Branched Regularization for Federated Learning

A critical challenge of federated learning is data heterogeneity and imbalance across clients, which leads to inconsistency between local networks and unstable convergence of global models. To alleviate the limitations, we propose a novel architectural regularization technique that constructs multiple auxiliary branches in each local model by grafting local and global subnetworks at several different levels and that learns the representations of the main pathway in the local model congruent to the auxiliary hybrid pathways via online knowledge distillation. The proposed technique is effective to robustify the global model even in the non-iid setting and is applicable to various federated learning frameworks conveniently without incurring extra communication costs. We perform comprehensive empirical studies and demonstrate remarkable performance gains in terms of accuracy and efficiency compared to existing methods. The source code is available at our project page.Comment: ICML 202

Communication-Efficient Federated Learning with Accelerated Client Gradient

Federated learning often suffers from slow and unstable convergence due to the heterogeneous characteristics of participating client datasets. Such a tendency is aggravated when the client participation ratio is low since the information collected from the clients has large variations. To address this challenge, we propose a simple but effective federated learning framework, which improves the consistency across clients and facilitates the convergence of the server model. This is achieved by making the server broadcast a global model with a lookahead gradient. This strategy enables the proposed approach to convey the projected global update information to participants effectively without additional client memory and extra communication costs. We also regularize local updates by aligning each client with the overshot global model to reduce bias and improve the stability of our algorithm. We provide the theoretical convergence rate of our algorithm and demonstrate remarkable performance gains in terms of accuracy and communication efficiency compared to the state-of-the-art methods, especially with low client participation rates. The source code is available at our project page.Comment: CVPR 202

Learning to Optimize Domain Specific Normalization for Domain Generalization

We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers that are specific to individual domains. Our approach employs multiple normalization methods while learning separate affine parameters per domain. For each domain, the activations are normalized by a weighted average of multiple normalization statistics. The normalization statistics are kept track of separately for each normalization type if necessary. Specifically, we employ batch and instance normalizations in our implementation to identify the best combination of these two normalization methods in each domain. The optimized normalization layers are effective to enhance the generalizability of the learned model. We demonstrate the state-of-the-art accuracy of our algorithm in the standard domain generalization benchmarks, as well as viability to further tasks such as multi-source domain adaptation and domain generalization in the presence of label noise

Using Etomidate and Midazolam for Screening Colonoscopies Results in More Stable Hemodynamic Responses in Patients of All Ages

Background/Aims Recent studies have demonstrated that etomidate is a safe sedative drug with noninferior sedative effects. In our recent study, we revealed that etomidate/midazolam was more hemodynamically stable than propofol/midazolam in elderly patients undergoing colonoscopies. We aimed to investigate whether compared with propofol/midazolam, etomidate/midazolam causes fewer cardiopulmonary adverse events with noninferior efficacy for screening colonoscopies in patients of all ages. Methods : In this single-center, randomized, double-blind study, we prospectively enrolled 200 patients. The patients were divided into etomidate and propofol groups. The primary outcome was the occurrence of cardiopulmonary adverse events. The secondary outcomes were the proportion of patients with fluctuations in vital signs (oxygen desaturation and transient hypotension), adverse events interrupting the procedure, and sedation-related outcomes. Results : Adverse cardiopulmonary events were more common in the propofol group than the etomidate group (65.0% vs 51.0%, respectively; p=0.045). Forty-six patients (46.0%) in the propofol group and 29 (29.0%) in the etomidate group experienced fluctuations in their vital signs (p=0.013). The proportions of patients experiencing adverse events that interrupted the procedure, including myoclonus, were not significantly different between the two groups (etomidate: 20.0% vs propofol: 11.0%; p=0.079). Both groups had similar sedation-related outcomes. Multivariate analysis revealed that compared with the propofol groups, the etomidate group had a significantly lower risk of fluctuations in vital signs (odds ratio, 0.427; 95% confidence interval, 0.230 to 0.792; p=0.007). Conclusion : s Compared with using propofol/midazolam, using etomidate/midazolam for screening colonoscopies results in more stable hemodynamic responses in patients of all ages; therefore, we recommend using etomidate/midazolam for colonoscopies in patients with cardiovascular risk factors