367 research outputs found
Proving Differential Privacy with Shadow Execution
Recent work on formal verification of differential privacy shows a trend
toward usability and expressiveness -- generating a correctness proof of
sophisticated algorithm while minimizing the annotation burden on programmers.
Sometimes, combining those two requires substantial changes to program logics:
one recent paper is able to verify Report Noisy Max automatically, but it
involves a complex verification system using customized program logics and
verifiers.
In this paper, we propose a new proof technique, called shadow execution, and
embed it into a language called ShadowDP. ShadowDP uses shadow execution to
generate proofs of differential privacy with very few programmer annotations
and without relying on customized logics and verifiers. In addition to
verifying Report Noisy Max, we show that it can verify a new variant of Sparse
Vector that reports the gap between some noisy query answers and the noisy
threshold. Moreover, ShadowDP reduces the complexity of verification: for all
of the algorithms we have evaluated, type checking and verification in total
takes at most 3 seconds, while prior work takes minutes on the same algorithms.Comment: 23 pages, 12 figures, PLDI'1
Relational Symbolic Execution
Symbolic execution is a classical program analysis technique used to show
that programs satisfy or violate given specifications. In this work we
generalize symbolic execution to support program analysis for relational
specifications in the form of relational properties - these are properties
about two runs of two programs on related inputs, or about two executions of a
single program on related inputs. Relational properties are useful to formalize
notions in security and privacy, and to reason about program optimizations. We
design a relational symbolic execution engine, named RelSym which supports
interactive refutation, as well as proving of relational properties for
programs written in a language with arrays and for-like loops
Probabilistic Couplings For Probabilistic Reasoning
This thesis explores proofs by coupling from the perspective of formal verification. Long employed in probability theory and theoretical computer science, these proofs construct couplings between the output distributions of two probabilistic processes. Couplings can imply various probabilistic relational properties, guarantees that compare two runs of a probabilistic computation.
To give a formal account of this clean proof technique, we first show that proofs in the program logic pRHL (probabilistic Relational Hoare Logic) describe couplings. We formalize couplings that establish various probabilistic properties, including distribution equivalence, convergence, and stochastic domination. Then we deepen the connection between couplings and pRHL by giving a proofs-as-programs interpretation: a coupling proof encodes a probabilistic product program, whose properties imply relational properties of the original two programs. We design the logic xpRHL (product pRHL) to build the product program, with extensions to model more advanced constructions including shift coupling and path coupling.
We then develop an approximate version of probabilistic coupling, based on approximate liftings. It is known that the existence of an approximate lifting implies differential privacy, a relational notion of statistical privacy. We propose a corresponding proof technique---proof by approximate coupling---inspired by the logic apRHL, a version of pRHL for building approximate liftings. Drawing on ideas from existing privacy proofs, we extend apRHL with novel proof rules for constructing new approximate couplings. We give approximate coupling proofs of privacy for the Report-noisy-max and Sparse Vector mechanisms, well-known algorithms from the privacy literature with notoriously subtle privacy proofs, and produce the first formalized proof of privacy for these algorithms in apRHL.
Finally, we enrich the theory of approximate couplings with several more sophisticated constructions: a principle for showing accuracy-dependent privacy, a generalization of the advanced composition theorem from differential privacy, and an optimal approximate coupling relating two subsets of samples. We also show equivalences between approximate couplings and other existing definitions. These ingredients support the first formalized proof of privacy for the Between Thresholds mechanism, an extension of the Sparse Vector mechanism
Local Obfuscation Mechanisms for Hiding Probability Distributions
We introduce a formal model for the information leakage of probability
distributions and define a notion called distribution privacy as the local
differential privacy for probability distributions. Roughly, the distribution
privacy of a local obfuscation mechanism means that the attacker cannot
significantly gain any information on the distribution of the mechanism's input
by observing its output. Then we show that existing local mechanisms can hide
input distributions in terms of distribution privacy, while deteriorating the
utility by adding too much noise. For example, we prove that the Laplace
mechanism needs to add a large amount of noise proportionally to the infinite
Wasserstein distance between the two distributions we want to make
indistinguishable. To improve the tradeoff between distribution privacy and
utility, we introduce a local obfuscation mechanism, called a tupling
mechanism, that adds random dummy data to the output. Then we apply this
mechanism to the protection of user attributes in location based services. By
experiments, we demonstrate that the tupling mechanism outperforms popular
local mechanisms in terms of attribute obfuscation and service quality.Comment: Full version of Proc. ESORICS 2019 (with a longer appendix
Lower Bounds for R\'enyi Differential Privacy in a Black-Box Setting
We present new methods for assessing the privacy guarantees of an algorithm
with regard to R\'enyi Differential Privacy. To the best of our knowledge, this
work is the first to address this problem in a black-box scenario, where only
algorithmic outputs are available. To quantify privacy leakage, we devise a new
estimator for the R\'enyi divergence of a pair of output distributions. This
estimator is transformed into a statistical lower bound that is proven to hold
for large samples with high probability. Our method is applicable for a broad
class of algorithms, including many well-known examples from the privacy
literature. We demonstrate the effectiveness of our approach by experiments
encompassing algorithms and privacy enhancing methods that have not been
considered in related works
- …