Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning

Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In particular, the overhead of the state-of-the-art protocols for secure model aggregation grows quadratically with the number of users. In this paper, we propose the first secure aggregation framework, named Turbo-Aggregate, that in a network with $N$ users achieves a secure aggregation overhead of $O(N\log{N})$, as opposed to $O(N^2)$, while tolerating up to a user dropout rate of $50\%$. Turbo-Aggregate employs a multi-group circular strategy for efficient model aggregation, and leverages additive secret sharing and novel coding techniques for injecting aggregation redundancy in order to handle user dropouts while guaranteeing user privacy. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to $40\times$ speedup over the state-of-the-art protocols with up to $N=200$ users. Our experiments also demonstrate the impact of model size and bandwidth on the performance of Turbo-Aggregate

CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning

How to train a machine learning model while keeping the data private and secure? We present CodedPrivateML, a fast and scalable approach to this critical problem. CodedPrivateML keeps both the data and the model information-theoretically private, while allowing efficient parallelization of training across distributed workers. We characterize CodedPrivateML\u27s privacy threshold and prove its convergence for logistic (and linear) regression. Furthermore, via experiments over Amazon EC2, we demonstrate that CodedPrivateML can provide an order of magnitude speedup (up to $\sim 34\times$) over the state-of-the-art cryptographic approaches

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

Secure aggregation is a critical component in federated learning, which enables the server to learn the aggregate model of the users without observing their local models. Conventionally, secure aggregation algorithms focus only on ensuring the privacy of individual users in a single training round. We contend that such designs can lead to significant privacy leakages over multiple training rounds, due to partial user selection/participation at each round of federated learning. In fact, we empirically show that the conventional random user selection strategies for federated learning lead to leaking users\u27 individual models within number of rounds linear in the number of users. To address this challenge, we introduce a secure aggregation framework with multi-round privacy guarantees. In particular, we introduce a new metric to quantify the privacy guarantees of federated learning over multiple training rounds, and develop a structured user selection strategy that guarantees the long-term privacy of each user (over any number of training rounds). Our framework also carefully accounts for the fairness and the average number of participating users at each round. We perform several experiments on MNIST and CIFAR-10 datasets in the IID and the non-IID settings to demonstrate the performance improvement over the baseline algorithms, both in terms of privacy protection and test accuracy

Measuring patient acuity and nursing care needs in South Korea: application of a new patient classification system

Background An accurate and reliable patient classification system (PCS) can help inform decisions regarding adequate assignments for nurse staffing. This study aimed to evaluate the criterion validity of the Asan Patient Classification System (APCS), a new tertiary hospital-specific PCS, by comparing its rating and total scores with those of KPCS-1 and KPCS-GW for measuring patient activity and nursing needs. Methods We performed a retrospective analysis of the medical records of 50,314 inpatients admitted to the general wards of a tertiary teaching hospital in Seoul, South Korea in March, June, September, and December 2019. Spearman’s correlation and Kappa statistics according to quartiles were calculated to examine the criterion validity of the APCS compared with the KPCS-1 and KPCS-GW. Results The average patient classification score was 28.3 points for APCS, 25.7 points for KPCS-1, and 21.6 points for KPCS-GW. The kappa value between APCS and KPCS-1 was 0.91 (95% CI:0.9072, 0.9119) and that between APCS and KPCS-GW was 0.88 (95% CI:0.8757, 0.8810). Additionally, Spearman's correlation coefficients among APCS, KPCS-1, and KPCS-GW showed a very strong correlation. However, 10.8% of the participants’ results were inconsistent, and KPCS-1 tended to classify patients into groups with lower nursing needs compared to APCS. Conclusion This study showed that electronic health record-generated APCS can provide useful information on patients’ severity and nursing activities to measure workload estimation. Additional research is needed to develop and implement a real-world EHR-based PCS system to accommodate for direct and indirect nursing care while considering diverse population and dynamic healthcare system

Secure Single-Server Aggregation with (Poly)Logarithmic Overhead

Secure aggregation is a cryptographic primitive that enables a server to learn the sum of the vector inputs of many clients. Bonawitz et al. (CCS 2017) presented a construction that incurs computation and communication for each client linear in the number of parties. While this functionality enables a broad range of privacy preserving computational tasks, scaling concerns limit its scope of use. We present the first constructions for secure aggregation that achieve polylogarithmic communication and computation per client. Our constructions provide security in the semi-honest and the semi-malicious setting where the adversary controls the server and a $\gamma$-fraction of the clients, and correctness with up to $\delta$-fraction dropouts among the clients. Our constructions show how to replace the complete communication graph of Bonawitz et al., which entails the linear overheads, with a $k$-regular graph of logarithmic degree while maintaining the security guarantees. Beyond improving the known asymptotics for secure aggregation, our constructions also achieve very efficient concrete parameters. The semi-honest secure aggregation can handle a billion clients at the per client cost of the protocol of Bonawitz et al. for a thousand clients. In the semi-malicious setting with $10^4$ clients, each client needs to communicate only with $3\%$ of the clients to have a guarantee that its input has been added together with the inputs of at least $5000$ other clients, while withstanding up to $5\%$ corrupt clients and $5\%$ dropouts. We also show an application of secure aggregation to the task of secure shuffling which enables the first cryptographically secure instantiation of the shuffle model of differential privacy