3,213 research outputs found
XONN: XNOR-based Oblivious Deep Neural Network Inference
Advancements in deep learning enable cloud servers to provide
inference-as-a-service for clients. In this scenario, clients send their raw
data to the server to run the deep learning model and send back the results.
One standing challenge in this setting is to ensure the privacy of the clients'
sensitive data. Oblivious inference is the task of running the neural network
on the client's input without disclosing the input or the result to the server.
This paper introduces XONN, a novel end-to-end framework based on Yao's Garbled
Circuits (GC) protocol, that provides a paradigm shift in the conceptual and
practical realization of oblivious inference. In XONN, the costly
matrix-multiplication operations of the deep learning model are replaced with
XNOR operations that are essentially free in GC. We further provide a novel
algorithm that customizes the neural network such that the runtime of the GC
protocol is minimized without sacrificing the inference accuracy.
We design a user-friendly high-level API for XONN, allowing expression of the
deep learning model architecture in an unprecedented level of abstraction.
Extensive proof-of-concept evaluation on various neural network architectures
demonstrates that XONN outperforms prior art such as Gazelle (USENIX
Security'18) by up to 7x, MiniONN (ACM CCS'17) by 93x, and SecureML (IEEE
S&P'17) by 37x. State-of-the-art frameworks require one round of interaction
between the client and the server for each layer of the neural network,
whereas, XONN requires a constant round of interactions for any number of
layers in the model. XONN is first to perform oblivious inference on Fitnet
architectures with up to 21 layers, suggesting a new level of scalability
compared with state-of-the-art. Moreover, we evaluate XONN on four datasets to
perform privacy-preserving medical diagnosis.Comment: To appear in USENIX Security 201
Private Model Compression via Knowledge Distillation
The soaring demand for intelligent mobile applications calls for deploying
powerful deep neural networks (DNNs) on mobile devices. However, the
outstanding performance of DNNs notoriously relies on increasingly complex
models, which in turn is associated with an increase in computational expense
far surpassing mobile devices' capacity. What is worse, app service providers
need to collect and utilize a large volume of users' data, which contain
sensitive information, to build the sophisticated DNN models. Directly
deploying these models on public mobile devices presents prohibitive privacy
risk. To benefit from the on-device deep learning without the capacity and
privacy concerns, we design a private model compression framework RONA.
Following the knowledge distillation paradigm, we jointly use hint learning,
distillation learning, and self learning to train a compact and fast neural
network. The knowledge distilled from the cumbersome model is adaptively
bounded and carefully perturbed to enforce differential privacy. We further
propose an elegant query sample selection method to reduce the number of
queries and control the privacy loss. A series of empirical evaluations as well
as the implementation on an Android mobile device show that RONA can not only
compress cumbersome models efficiently but also provide a strong privacy
guarantee. For example, on SVHN, when a meaningful
-differential privacy is guaranteed, the compact model trained
by RONA can obtain 20 compression ratio and 19 speed-up with
merely 0.97% accuracy loss.Comment: Conference version accepted by AAAI'1
- …