210 research outputs found
Secure Split Learning against Property Inference, Data Reconstruction, and Feature Space Hijacking Attacks
Split learning of deep neural networks (SplitNN) has provided a promising
solution to learning jointly for the mutual interest of a guest and a host,
which may come from different backgrounds, holding features partitioned
vertically. However, SplitNN creates a new attack surface for the adversarial
participant, holding back its practical use in the real world. By investigating
the adversarial effects of highly threatening attacks, including property
inference, data reconstruction, and feature hijacking attacks, we identify the
underlying vulnerability of SplitNN and propose a countermeasure. To prevent
potential threats and ensure the learning guarantees of SplitNN, we design a
privacy-preserving tunnel for information exchange between the guest and the
host. The intuition is to perturb the propagation of knowledge in each
direction with a controllable unified solution. To this end, we propose a new
activation function named R3eLU, transferring private smashed data and partial
loss into randomized responses in forward and backward propagations,
respectively. We give the first attempt to secure split learning against three
threatening attacks and present a fine-grained privacy budget allocation
scheme. The analysis proves that our privacy-preserving SplitNN solution
provides a tight privacy budget, while the experimental results show that our
solution performs better than existing solutions in most cases and achieves a
good tradeoff between defense and model usability.Comment: 23 page
Markov modeling of moving target defense games
We introduce a Markov-model-based framework for Moving Target Defense (MTD) analysis. The framework allows modeling of broad range of MTD strategies, provides general theorems about how the probability of a successful adversary defeating an MTD strategy is related to the amount of time/cost spent by the adversary, and shows how a multi-level composition of MTD strategies can be analyzed by a straightforward combination of the analysis for each one of these strategies. Within the proposed framework we define the concept of security capacity which measures the strength or effectiveness of an MTD strategy: the security capacity depends on MTD specific parameters and more general system parameters. We apply our framework to two concrete MTD strategies
Group-based Robustness: A General Framework for Customized Robustness in the Real World
Machine-learning models are known to be vulnerable to evasion attacks that
perturb model inputs to induce misclassifications. In this work, we identify
real-world scenarios where the true threat cannot be assessed accurately by
existing attacks. Specifically, we find that conventional metrics measuring
targeted and untargeted robustness do not appropriately reflect a model's
ability to withstand attacks from one set of source classes to another set of
target classes. To address the shortcomings of existing methods, we formally
define a new metric, termed group-based robustness, that complements existing
metrics and is better-suited for evaluating model performance in certain attack
scenarios. We show empirically that group-based robustness allows us to
distinguish between models' vulnerability against specific threat models in
situations where traditional robustness metrics do not apply. Moreover, to
measure group-based robustness efficiently and accurately, we 1) propose two
loss functions and 2) identify three new attack strategies. We show empirically
that with comparable success rates, finding evasive samples using our new loss
functions saves computation by a factor as large as the number of targeted
classes, and finding evasive samples using our new attack strategies saves time
by up to 99\% compared to brute-force search methods. Finally, we propose a
defense method that increases group-based robustness by up to 3.52
- …