20,581 research outputs found
Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs
Distributed online learning has been proven extremely effective in solving
large-scale machine learning problems over streaming data. However, information
sharing between learners in distributed learning also raises concerns about the
potential leakage of individual learners' sensitive data. To mitigate this
risk, differential privacy, which is widely regarded as the "gold standard" for
privacy protection, has been widely employed in many existing results on
distributed online learning. However, these results often face a fundamental
tradeoff between learning accuracy and privacy. In this paper, we propose a
locally differentially private gradient tracking based distributed online
learning algorithm that successfully circumvents this tradeoff. We prove that
the proposed algorithm converges in mean square to the exact optimal solution
while ensuring rigorous local differential privacy, with the cumulative privacy
budget guaranteed to be finite even when the number of iterations tends to
infinity. The algorithm is applicable even when the communication graph among
learners is directed. To the best of our knowledge, this is the first result
that simultaneously ensures learning accuracy and rigorous local differential
privacy in distributed online learning over directed graphs. We evaluate our
algorithm's performance by using multiple benchmark machine-learning
applications, including logistic regression of the "Mushrooms" dataset and
CNN-based image classification of the "MNIST" and "CIFAR-10" datasets,
respectively. The experimental results confirm that the proposed algorithm
outperforms existing counterparts in both training and testing accuracies.Comment: 21 pages, 4 figure
Distributed Private Online Learning for Social Big Data Computing over Data Center Networks
With the rapid growth of Internet technologies, cloud computing and social
networks have become ubiquitous. An increasing number of people participate in
social networks and massive online social data are obtained. In order to
exploit knowledge from copious amounts of data obtained and predict social
behavior of users, we urge to realize data mining in social networks. Almost
all online websites use cloud services to effectively process the large scale
of social data, which are gathered from distributed data centers. These data
are so large-scale, high-dimension and widely distributed that we propose a
distributed sparse online algorithm to handle them. Additionally,
privacy-protection is an important point in social networks. We should not
compromise the privacy of individuals in networks, while these social data are
being learned for data mining. Thus we also consider the privacy problem in
this article. Our simulations shows that the appropriate sparsity of data would
enhance the performance of our algorithm and the privacy-preserving method does
not significantly hurt the performance of the proposed algorithm.Comment: ICC201
Crowd-ML: A Privacy-Preserving Learning Framework for a Crowd of Smart Devices
Smart devices with built-in sensors, computational capabilities, and network
connectivity have become increasingly pervasive. The crowds of smart devices
offer opportunities to collectively sense and perform computing tasks in an
unprecedented scale. This paper presents Crowd-ML, a privacy-preserving machine
learning framework for a crowd of smart devices, which can solve a wide range
of learning problems for crowdsensing data with differential privacy
guarantees. Crowd-ML endows a crowdsensing system with an ability to learn
classifiers or predictors online from crowdsensing data privately with minimal
computational overheads on devices and servers, suitable for a practical and
large-scale employment of the framework. We analyze the performance and the
scalability of Crowd-ML, and implement the system with off-the-shelf
smartphones as a proof of concept. We demonstrate the advantages of Crowd-ML
with real and simulated experiments under various conditions
Decentralized Differentially Private Without-Replacement Stochastic Gradient Descent
While machine learning has achieved remarkable results in a wide variety of
domains, the training of models often requires large datasets that may need to
be collected from different individuals. As sensitive information may be
contained in the individual's dataset, sharing training data may lead to severe
privacy concerns. Therefore, there is a compelling need to develop
privacy-aware machine learning methods, for which one effective approach is to
leverage the generic framework of differential privacy. Considering that
stochastic gradient descent (SGD) is one of the mostly adopted methods for
large-scale machine learning problems, two decentralized differentially private
SGD algorithms are proposed in this work. Particularly, we focus on SGD without
replacement due to its favorable structure for practical implementation. In
addition, both privacy and convergence analysis are provided for the proposed
algorithms. Finally, extensive experiments are performed to verify the
theoretical results and demonstrate the effectiveness of the proposed
algorithms
- …