20 research outputs found
Why Batch Normalization Damage Federated Learning on Non-IID Data?
As a promising distributed learning paradigm, federated learning (FL)
involves training deep neural network (DNN) models at the network edge while
protecting the privacy of the edge clients. To train a large-scale DNN model,
batch normalization (BN) has been regarded as a simple and effective means to
accelerate the training and improve the generalization capability. However,
recent findings indicate that BN can significantly impair the performance of FL
in the presence of non-i.i.d. data. While several FL algorithms have been
proposed to address this issue, their performance still falls significantly
when compared to the centralized scheme. Furthermore, none of them have
provided a theoretical explanation of how the BN damages the FL convergence. In
this paper, we present the first convergence analysis to show that under the
non-i.i.d. data, the mismatch between the local and global statistical
parameters in BN causes the gradient deviation between the local and global
models, which, as a result, slows down and biases the FL convergence. In view
of this, we develop a new FL algorithm that is tailored to BN, called FedTAN,
which is capable of achieving robust FL performance under a variety of data
distributions via iterative layer-wise parameter aggregation. Comprehensive
experimental results demonstrate the superiority of the proposed FedTAN over
existing baselines for training BN-based DNN models
Recommended from our members
CRISPR-Cas9 Gene Editing of Hematopoietic Stem Cells from Patients with Friedreich's Ataxia.
Friedreich's ataxia (FRDA) is an autosomal recessive neurodegenerative disorder caused by expansion of GAA repeats in intron 1 of the frataxin (FXN) gene, leading to significant decreased expression of frataxin, a mitochondrial iron-binding protein. We previously reported that syngeneic hematopoietic stem and progenitor cell (HSPC) transplantation prevented neurodegeneration in the FRDA mouse model YG8R. We showed that the mechanism of rescue was mediated by the transfer of the functional frataxin from HSPC-derived microglia/macrophage cells to neurons/myocytes. In this study, we report the first step toward an autologous HSPC transplantation using the CRISPR-Cas9 system for FRDA. We first identified a pair of CRISPR RNAs (crRNAs) that efficiently removes the GAA expansions in human FRDA lymphoblasts, restoring the non-pathologic level of frataxin expression and normalizing mitochondrial activity. We also optimized the gene-editing approach in HSPCs isolated from healthy and FRDA patients' peripheral blood and demonstrated normal hematopoiesis of gene-edited cells in vitro and in vivo. The procedure did not induce cellular toxic effect or major off-target events, but a p53-mediated cell proliferation delay was observed in the gene-edited cells. This study provides the foundation for the clinical translation of autologous transplantation of gene-corrected HSPCs for FRDA
Helical Edge States and Quantum Phase Transitions in Tetralayer Graphene
Helical conductors with spin-momentum locking are promising platforms for
Majorana fermions. Here we report observation of two topologically distinct
phases supporting helical edge states in charge neutral Bernal-stacked
tetralayer graphene in Hall bar and Corbino geometries. As the magnetic field B
and out-of-plane displacement field D are varied, we observe a phase diagram
consisting of an insulating phase and two metallic phases, with 0, 1 and 2
helical edge states, respectively. These phases are accounted for by a
theoretical model that relates their conductance to spin-polarization plateaus.
Transitions between them arise from a competition among inter-layer hopping,
electrostatic and exchange interaction energies. Our work highlights the
complex competing symmetries and the rich quantum phases in few-layer graphene.Comment: Accepted by PR