Federated Generalization via Information-Theoretic Distribution Diversification

Federated Learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the non-Independent Identically Distributed (non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the divergence between the training distributions of participating clients and the testing distributions of non-participating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations aim to bolster FL's generalization prowess by encompassing a more varied set of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct

Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networks

With the proliferation of intelligent mobile devices in wireless device-to-device (D2D) networks, decentralized federated learning (DFL) has attracted significant interest. Compared to centralized federated learning (CFL), DFL mitigates the risk of central server failures due to communication bottlenecks. However, DFL faces several challenges, such as the severe heterogeneity of data distributions in diverse environments, and the transmission outages and package errors caused by the adoption of the User Datagram Protocol (UDP) in D2D networks. These challenges often degrade the convergence of training DFL models. To address these challenges, we conduct a thorough theoretical convergence analysis for DFL and derive a convergence bound. By defining a novel quantity named unreliable links-aware neighborhood discrepancy in this convergence bound, we formulate a tractable optimization objective, and develop a novel Topology Learning method considering the Representation Discrepancy and Unreliable Links in DFL, named ToLRDUL. Intensive experiments under both feature skew and label skew settings have validated the effectiveness of our proposed method, demonstrating improved convergence speed and test accuracy, consistent with our theoretical findings.Comment: To appear in IEEE Transactions on Vehicular Technolog

Federated Knowledge Graph Completion via Latent Embedding Sharing and Tensor Factorization

Knowledge graphs (KGs), which consist of triples, are inherently incomplete and always require completion procedure to predict missing triples. In real-world scenarios, KGs are distributed across clients, complicating completion tasks due to privacy restrictions. Many frameworks have been proposed to address the issue of federated knowledge graph completion. However, the existing frameworks, including FedE, FedR, and FEKG, have certain limitations. = FedE poses a risk of information leakage, FedR's optimization efficacy diminishes when there is minimal overlap among relations, and FKGE suffers from computational costs and mode collapse issues. To address these issues, we propose a novel method, i.e., Federated Latent Embedding Sharing Tensor factorization (FLEST), which is a novel approach using federated tensor factorization for KG completion. FLEST decompose the embedding matrix and enables sharing of latent dictionary embeddings to lower privacy risks. Empirical results demonstrate FLEST's effectiveness and efficiency, offering a balanced solution between performance and privacy. FLEST expands the application of federated tensor factorization in KG completion tasks.Comment: Accepted by ICDM 202

Enhanced Federated Optimization: Adaptive Unbiased Sampling with Reduced Variance

Federated Learning (FL) is a distributed learning paradigm to train a global model across multiple devices without collecting local data. In FL, a server typically selects a subset of clients for each training round to optimize resource usage. Central to this process is the technique of unbiased client sampling, which ensures a representative selection of clients. Current methods primarily utilize a random sampling procedure which, despite its effectiveness, achieves suboptimal efficiency owing to the loose upper bound caused by the sampling variance. In this work, by adopting an independent sampling procedure, we propose a federated optimization framework focused on adaptive unbiased client sampling, improving the convergence rate via an online variance reduction strategy. In particular, we present the first adaptive client sampler, K-Vib, employing an independent sampling procedure. K-Vib achieves a linear speed-up on the regret bound $\tilde{\mathcal{O}}\big(N^{\frac{1}{3}}T^{\frac{2}{3}}/K^{\frac{4}{3}}\big)$ within a set communication budget $K$. Empirical studies indicate that K-Vib doubles the speed compared to baseline algorithms, demonstrating significant potential in federated optimization.Comment: Under revie

Mapping of Cu and Pb Contaminations in Soil Using Combined Geochemistry, Topography, and Remote Sensing: A Case Study in the Le’an River Floodplain, China

Heavy metal pollution in soil is becoming a widely concerning environmental problem in China. The aim of this study is to integrate multiple sources of data, namely total Cu and Pb contents, digital elevation model (DEM) data, remote sensing image and interpreted land-use data, for mapping the spatial distribution of total Cu and Pb contamination in top soil along the Le’an River and its branches. Combined with geographical analyses and watershed delineation, the source and transportation route of pollutants are identified. Regions at high risk of Cu or Pb pollution are suggested. Results reveal that topography is the major factor that controls the spatial distribution of Cu and Pb. Watershed delineation shows evidence that the streamflow resulting from rainfall is the major carrier of metal pollutants

RAGE limits regeneration after massive liver injury by coordinated suppression of TNF-α and NF-κB

The exquisite ability of the liver to regenerate is finite. Identification of mechanisms that limit regeneration after massive injury holds the key to expanding the limits of liver transplantation and salvaging livers and hosts overwhelmed by carcinoma and toxic insults. Receptor for advanced glycation endproducts (RAGE) is up-regulated in liver remnants selectively after massive (85%) versus partial (70%) hepatectomy, principally in mononuclear phagocyte-derived dendritic cells (MPDDCs). Blockade of RAGE, using pharmacological antagonists or transgenic mice in which a signaling-deficient RAGE mutant is expressed in cells of mononuclear phagocyte lineage, significantly increases survival after massive liver resection. In the first hours after massive resection, remnants retrieved from RAGE-blocked mice displayed increased activated NF-κB, principally in hepatocytes, and enhanced expression of regeneration-promoting cytokines, TNF-α and IL-6, and the antiinflammatory cytokine, IL-10. Hepatocyte proliferation was increased by RAGE blockade, in parallel with significantly reduced apoptosis. These data highlight central roles for RAGE and MPDDCs in modulation of cell death–promoting mechanisms in massive hepatectomy and suggest that RAGE blockade is a novel strategy to promote regeneration in the massively injured liver