42 research outputs found
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
Deep neural networks are susceptible to various inference attacks as they
remember information about their training data. We design white-box inference
attacks to perform a comprehensive privacy analysis of deep learning models. We
measure the privacy leakage through parameters of fully trained models as well
as the parameter updates of models during training. We design inference
algorithms for both centralized and federated learning, with respect to passive
and active inference attackers, and assuming different adversary prior
knowledge.
We evaluate our novel white-box membership inference attacks against deep
learning algorithms to trace their training data records. We show that a
straightforward extension of the known black-box attacks to the white-box
setting (through analyzing the outputs of activation functions) is ineffective.
We therefore design new algorithms tailored to the white-box setting by
exploiting the privacy vulnerabilities of the stochastic gradient descent
algorithm, which is the algorithm used to train deep neural networks. We
investigate the reasons why deep learning models may leak information about
their training data. We then show that even well-generalized models are
significantly susceptible to white-box membership inference attacks, by
analyzing state-of-the-art pre-trained and publicly available models for the
CIFAR dataset. We also show how adversarial participants, in the federated
learning setting, can successfully run active membership inference attacks
against other participants, even when the global model achieves high prediction
accuracies.Comment: 2019 IEEE Symposium on Security and Privacy (SP
How Does Data Augmentation Affect Privacy in Machine Learning?
It is observed in the literature that data augmentation can significantly
mitigate membership inference (MI) attack. However, in this work, we challenge
this observation by proposing new MI attacks to utilize the information of
augmented data. MI attack is widely used to measure the model's information
leakage of the training set. We establish the optimal membership inference when
the model is trained with augmented data, which inspires us to formulate the MI
attack as a set classification problem, i.e., classifying a set of augmented
instances instead of a single data point, and design input permutation
invariant features. Empirically, we demonstrate that the proposed approach
universally outperforms original methods when the model is trained with data
augmentation. Even further, we show that the proposed approach can achieve
higher MI attack success rates on models trained with some data augmentation
than the existing methods on models trained without data augmentation. Notably,
we achieve a 70.1% MI attack success rate on CIFAR10 against a wide residual
network while the previous best approach only attains 61.9%. This suggests the
privacy risk of models trained with data augmentation could be largely
underestimated.Comment: AAAI Conference on Artificial Intelligence (AAAI-21). Source code
available at: https://github.com/dayu11/MI_with_D