59,410 research outputs found
Private Model Compression via Knowledge Distillation
The soaring demand for intelligent mobile applications calls for deploying
powerful deep neural networks (DNNs) on mobile devices. However, the
outstanding performance of DNNs notoriously relies on increasingly complex
models, which in turn is associated with an increase in computational expense
far surpassing mobile devices' capacity. What is worse, app service providers
need to collect and utilize a large volume of users' data, which contain
sensitive information, to build the sophisticated DNN models. Directly
deploying these models on public mobile devices presents prohibitive privacy
risk. To benefit from the on-device deep learning without the capacity and
privacy concerns, we design a private model compression framework RONA.
Following the knowledge distillation paradigm, we jointly use hint learning,
distillation learning, and self learning to train a compact and fast neural
network. The knowledge distilled from the cumbersome model is adaptively
bounded and carefully perturbed to enforce differential privacy. We further
propose an elegant query sample selection method to reduce the number of
queries and control the privacy loss. A series of empirical evaluations as well
as the implementation on an Android mobile device show that RONA can not only
compress cumbersome models efficiently but also provide a strong privacy
guarantee. For example, on SVHN, when a meaningful
-differential privacy is guaranteed, the compact model trained
by RONA can obtain 20 compression ratio and 19 speed-up with
merely 0.97% accuracy loss.Comment: Conference version accepted by AAAI'1
On the Interaction Between Differential Privacy and Gradient Compression in Deep Learning
While differential privacy and gradient compression are separately
well-researched topics in machine learning, the study of interaction between
these two topics is still relatively new. We perform a detailed empirical study
on how the Gaussian mechanism for differential privacy and gradient compression
jointly impact test accuracy in deep learning. The existing literature in
gradient compression mostly evaluates compression in the absence of
differential privacy guarantees, and demonstrate that sufficiently high
compression rates reduce accuracy. Similarly, existing literature in
differential privacy evaluates privacy mechanisms in the absence of
compression, and demonstrates that sufficiently strong privacy guarantees
reduce accuracy. In this work, we observe while gradient compression generally
has a negative impact on test accuracy in non-private training, it can
sometimes improve test accuracy in differentially private training.
Specifically, we observe that when employing aggressive sparsification or rank
reduction to the gradients, test accuracy is less affected by the Gaussian
noise added for differential privacy. These observations are explained through
an analysis how differential privacy and compression effects the bias and
variance in estimating the average gradient. We follow this study with a
recommendation on how to improve test accuracy under the context of
differentially private deep learning and gradient compression. We evaluate this
proposal and find that it can reduce the negative impact of noise added by
differential privacy mechanisms on test accuracy by up to 24.6%, and reduce the
negative impact of gradient sparsification on test accuracy by up to 15.1%
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Deep Learning has recently become hugely popular in machine learning,
providing significant improvements in classification accuracy in the presence
of highly-structured and large databases.
Researchers have also considered privacy implications of deep learning.
Models are typically trained in a centralized manner with all the data being
processed by the same training algorithm. If the data is a collection of users'
private data, including habits, personal pictures, geographical positions,
interests, and more, the centralized server will have access to sensitive
information that could potentially be mishandled. To tackle this problem,
collaborative deep learning models have recently been proposed where parties
locally train their deep learning structures and only share a subset of the
parameters in the attempt to keep their respective training sets private.
Parameters can also be obfuscated via differential privacy (DP) to make
information extraction even more challenging, as proposed by Shokri and
Shmatikov at CCS'15.
Unfortunately, we show that any privacy-preserving collaborative deep
learning is susceptible to a powerful attack that we devise in this paper. In
particular, we show that a distributed, federated, or decentralized deep
learning approach is fundamentally broken and does not protect the training
sets of honest participants. The attack we developed exploits the real-time
nature of the learning process that allows the adversary to train a Generative
Adversarial Network (GAN) that generates prototypical samples of the targeted
training set that was meant to be private (the samples generated by the GAN are
intended to come from the same distribution as the training data).
Interestingly, we show that record-level DP applied to the shared parameters of
the model, as suggested in previous work, is ineffective (i.e., record-level DP
is not designed to address our attack).Comment: ACM CCS'17, 16 pages, 18 figure
- …