Concept-aware clustering for decentralized deep learning under temporal shift

Decentralized deep learning requires dealing with non-iid data across clients, which may also change over time due to temporal shifts. While non-iid data has been extensively studied in distributed settings, temporal shifts have received no attention. To the best of our knowledge, we are first with tackling the novel and challenging problem of decentralized learning with non-iid and dynamic data. We propose a novel algorithm that can automatically discover and adapt to the evolving concepts in the network, without any prior knowledge or estimation of the number of concepts. We evaluate our algorithm on standard benchmark datasets and demonstrate that it outperforms previous methods for decentralized learning.Comment: 4 pages, 2 figure

Using GANs for Sharing Networked Time Series Data: Challenges, Initial Promise, and Open Questions

Limited data access is a longstanding barrier to data-driven research and development in the networked systems community. In this work, we explore if and how generative adversarial networks (GANs) can be used to incentivize data sharing by enabling a generic framework for sharing synthetic datasets with minimal expert knowledge. As a specific target, our focus in this paper is on time series datasets with metadata (e.g., packet loss rate measurements with corresponding ISPs). We identify key challenges of existing GAN approaches for such workloads with respect to fidelity (e.g., long-term dependencies, complex multidimensional relationships, mode collapse) and privacy (i.e., existing guarantees are poorly understood and can sacrifice fidelity). To improve fidelity, we design a custom workflow called DoppelGANger (DG) and demonstrate that across diverse real-world datasets (e.g., bandwidth measurements, cluster requests, web sessions) and use cases (e.g., structural characterization, predictive modeling, algorithm comparison), DG achieves up to 43% better fidelity than baseline models. Although we do not resolve the privacy problem in this work, we identify fundamental challenges with both classical notions of privacy and recent advances to improve the privacy properties of GANs, and suggest a potential roadmap for addressing these challenges. By shedding light on the promise and challenges, we hope our work can rekindle the conversation on workflows for data sharing.Comment: Published in IMC 2020. 20 pages, 26 figure

Adaptive Expert Models for Personalization in Federated Learning

Federated Learning (FL) is a promising framework for distributed learning whendata is private and sensitive. However, the state-of-the-art solutions in thisframework are not optimal when data is heterogeneous and non-Independent andIdentically Distributed (non-IID). We propose a practical and robust approachto personalization in FL that adjusts to heterogeneous and non-IID data bybalancing exploration and exploitation of several global models. To achieve ouraim of personalization, we use a Mixture of Experts (MoE) that learns to groupclients that are similar to each other, while using the global models moreefficiently. We show that our approach achieves an accuracy up to 29.78 % andup to 4.38 % better compared to a local model in a pathological non-IIDsetting, even though we tune our approach in the IID setting.QC 20220628</p

Adaptive Expert Models for\ua0Federated Learning

Federated Learning (FL) is a promising framework for distributed learning when data is private and sensitive. However, the state-of-the-art solutions in this framework are not optimal when data is heterogeneous and non-IID. We propose a practical and robust approach to personalization in FL that adjusts to heterogeneous and non-IID data by balancing exploration and exploitation of several global models. To achieve our aim of personalization, we use a Mixture of Experts (MoE) that learns to group clients that are similar to each other, while using the global models more efficiently. We show that our approach achieves an accuracy up to 29.78% better than the state-of-the-art and up to 4.38% better compared to a local model in a pathological non-IID setting, even though we tune our approach in the IID setting

Komplexanalytiska metoder inom talteori

Sammandrag I det här kandidatarbetet redogör vi för bevis av tre klassiska satser från talteorin. Vi kommer att bevisa primtalssatsen, två- och fyrkvadratssatsen och Dirichlets sats om primtal i aritmetiska följder. Till vår hjälp tar vi begrepp ifrån komplexanalys och Fourieranalys, och arbetet innehåller därför också en grundlig teorigenomgång innan själva satserna kan bevisas. Abstract In this bachelor thesis we outline proofs for three classic theorems from number theory. We will prove the prime number theorem, Jacobi’s two- and four-squares theorems and Dirichlet’s theorem on primes in arithmetic sequences. In proving these theorems, methods from complex analysis and Fourier analys will be needed. Thus, this thesis includes a thorough review of the necessary theory