15,063 research outputs found
Privacy Risks of Securing Machine Learning Models against Adversarial Examples
The arms race between attacks and defenses for machine learning models has
come to a forefront in recent years, in both the security community and the
privacy community. However, one big limitation of previous research is that the
security domain and the privacy domain have typically been considered
separately. It is thus unclear whether the defense methods in one domain will
have any unexpected impact on the other domain.
In this paper, we take a step towards resolving this limitation by combining
the two domains. In particular, we measure the success of membership inference
attacks against six state-of-the-art defense methods that mitigate the risk of
adversarial examples (i.e., evasion attacks). Membership inference attacks
determine whether or not an individual data record has been part of a model's
training set. The accuracy of such attacks reflects the information leakage of
training algorithms about individual members of the training set. Adversarial
defense methods against adversarial examples influence the model's decision
boundaries such that model predictions remain unchanged for a small area around
each input. However, this objective is optimized on training data. Thus,
individual data records in the training set have a significant influence on
robust models. This makes the models more vulnerable to inference attacks.
To perform the membership inference attacks, we leverage the existing
inference methods that exploit model predictions. We also propose two new
inference methods that exploit structural properties of robust models on
adversarially perturbed data. Our experimental evaluation demonstrates that
compared with the natural training (undefended) approach, adversarial defense
methods can indeed increase the target model's risk against membership
inference attacks.Comment: ACM CCS 2019, code is available at
https://github.com/inspire-group/privacy-vs-robustnes
Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation
Today's success of state of the art methods for semantic segmentation is
driven by large datasets. Data is considered an important asset that needs to
be protected, as the collection and annotation of such datasets comes at
significant efforts and associated costs. In addition, visual data might
contain private or sensitive information, that makes it equally unsuited for
public release. Unfortunately, recent work on membership inference in the
broader area of adversarial machine learning and inference attacks on machine
learning models has shown that even black box classifiers leak information on
the dataset that they were trained on. We show that such membership inference
attacks can be successfully carried out on complex, state of the art models for
semantic segmentation. In order to mitigate the associated risks, we also study
a series of defenses against such membership inference attacks and find
effective counter measures against the existing risks with little effect on the
utility of the segmentation method. Finally, we extensively evaluate our
attacks and defenses on a range of relevant real-world datasets: Cityscapes,
BDD100K, and Mapillary Vistas.Comment: Accepted to ECCV 2020. Code at:
https://github.com/SSAW14/segmentation_membership_inferenc
- …