17,723 research outputs found
MPCViT: Searching for MPC-friendly Vision Transformer with Heterogeneous Attention
Secure multi-party computation (MPC) enables computation directly on
encrypted data on non-colluding untrusted servers and protects both data and
model privacy in deep learning inference. However, existing neural network (NN)
architectures, including Vision Transformers (ViTs), are not designed or
optimized for MPC protocols and incur significant latency overhead due to the
Softmax function in the multi-head attention (MHA). In this paper, we propose
an MPC-friendly ViT, dubbed MPCViT, to enable accurate yet efficient ViT
inference in MPC. We systematically compare different attention variants in MPC
and propose a heterogeneous attention search space, which combines the
high-accuracy and MPC-efficient attentions with diverse structure
granularities. We further propose a simple yet effective differentiable neural
architecture search (NAS) algorithm for fast ViT optimization. MPCViT
significantly outperforms prior-art ViT variants in MPC. With the proposed NAS
algorithm, our extensive experiments demonstrate that MPCViT achieves 7.9x and
2.8x latency reduction with better accuracy compared to Linformer and MPCFormer
on the Tiny-ImageNet dataset, respectively. Further, with proper knowledge
distillation (KD), MPCViT even achieves 1.9% better accuracy compared to the
baseline ViT with 9.9x latency reduction on the Tiny-ImageNet dataset.Comment: 6 pages, 6 figure
DEMO: integrating MPC in big data workflows
Secure multi-party computation (MPC) allows multiple parties to perform a joint computation without disclosing their private inputs. Many real-world joint computation use cases, however, involve data analyses on very large data sets, and are implemented by software engineers who lack MPC knowledge. Moreover, the collaborating parties -- e.g., several companies -- often deploy different data analytics stacks internally. These restrictions hamper the real-world usability of MPC. To address these challenges, we combine existing MPC frameworks with data-parallel analytics frameworks by extending the Musketeer big data workflow manager [4]. Musketeer automatically generates code for both the sensitive parts of a workflow, which are executed in MPC, and the remainder of the computation, which runs on scalable, widely-deployed analytics systems. In a prototype use case, we compute the Herfindahl-Hirschman Index (HHI), an index of market concentration used in antitrust regulation, on an aggregate 156GB of taxi trip data over five transportation companies. Our implementation computes the HHI in about 20 minutes using a combination of Hadoop and VIFF [1], while even "mixed mode" MPC with VIFF alone would have taken many hours. Finally, we discuss future research questions that we seek to address using our approach
- …