25,981 research outputs found
Information Leakage Games
We consider a game-theoretic setting to model the interplay between attacker
and defender in the context of information flow, and to reason about their
optimal strategies. In contrast with standard game theory, in our games the
utility of a mixed strategy is a convex function of the distribution on the
defender's pure actions, rather than the expected value of their utilities.
Nevertheless, the important properties of game theory, notably the existence of
a Nash equilibrium, still hold for our (zero-sum) leakage games, and we provide
algorithms to compute the corresponding optimal strategies. As typical in
(simultaneous) game theory, the optimal strategy is usually mixed, i.e.,
probabilistic, for both the attacker and the defender. From the point of view
of information flow, this was to be expected in the case of the defender, since
it is well known that randomization at the level of the system design may help
to reduce information leaks. Regarding the attacker, however, this seems the
first work (w.r.t. the literature in information flow) proving formally that in
certain cases the optimal attack strategy is necessarily probabilistic
Formal Verification of Differential Privacy for Interactive Systems
Differential privacy is a promising approach to privacy preserving data
analysis with a well-developed theory for functions. Despite recent work on
implementing systems that aim to provide differential privacy, the problem of
formally verifying that these systems have differential privacy has not been
adequately addressed. This paper presents the first results towards automated
verification of source code for differentially private interactive systems. We
develop a formal probabilistic automaton model of differential privacy for
systems by adapting prior work on differential privacy for functions. The main
technical result of the paper is a sound proof technique based on a form of
probabilistic bisimulation relation for proving that a system modeled as a
probabilistic automaton satisfies differential privacy. The novelty lies in the
way we track quantitative privacy leakage bounds using a relation family
instead of a single relation. We illustrate the proof technique on a
representative automaton motivated by PINQ, an implemented system that is
intended to provide differential privacy. To make our proof technique easier to
apply to realistic systems, we prove a form of refinement theorem and apply it
to show that a refinement of the abstract PINQ automaton also satisfies our
differential privacy definition. Finally, we begin the process of automating
our proof technique by providing an algorithm for mechanically checking a
restricted class of relations from the proof technique.Comment: 65 pages with 1 figur
Abstract Hidden Markov Models: a monadic account of quantitative information flow
Hidden Markov Models, HMM's, are mathematical models of Markov processes with
state that is hidden, but from which information can leak. They are typically
represented as 3-way joint-probability distributions.
We use HMM's as denotations of probabilistic hidden-state sequential
programs: for that, we recast them as `abstract' HMM's, computations in the
Giry monad , and we equip them with a partial order of increasing
security. However to encode the monadic type with hiding over some state
we use rather
than the conventional that suffices for
Markov models whose state is not hidden. We illustrate the
construction with a small
Haskell prototype.
We then present uncertainty measures as a generalisation of the extant
diversity of probabilistic entropies, with characteristic analytic properties
for them, and show how the new entropies interact with the order of increasing
security. Furthermore, we give a `backwards' uncertainty-transformer semantics
for HMM's that is dual to the `forwards' abstract HMM's - it is an analogue of
the duality between forwards, relational semantics and backwards,
predicate-transformer semantics for imperative programs with demonic choice.
Finally, we argue that, from this new denotational-semantic viewpoint, one
can see that the Dalenius desideratum for statistical databases is actually an
issue in compositionality. We propose a means for taking it into account
Quantifying Differential Privacy under Temporal Correlations
Differential Privacy (DP) has received increased attention as a rigorous
privacy framework. Existing studies employ traditional DP mechanisms (e.g., the
Laplace mechanism) as primitives, which assume that the data are independent,
or that adversaries do not have knowledge of the data correlations. However,
continuously generated data in the real world tend to be temporally correlated,
and such correlations can be acquired by adversaries. In this paper, we
investigate the potential privacy loss of a traditional DP mechanism under
temporal correlations in the context of continuous data release. First, we
model the temporal correlations using Markov model and analyze the privacy
leakage of a DP mechanism when adversaries have knowledge of such temporal
correlations. Our analysis reveals that the privacy leakage of a DP mechanism
may accumulate and increase over time. We call it temporal privacy leakage.
Second, to measure such privacy leakage, we design an efficient algorithm for
calculating it in polynomial time. Although the temporal privacy leakage may
increase over time, we also show that its supremum may exist in some cases.
Third, to bound the privacy loss, we propose mechanisms that convert any
existing DP mechanism into one against temporal privacy leakage. Experiments
with synthetic data confirm that our approach is efficient and effective.Comment: appears at ICDE 201
Quantitative information flow under generic leakage functions and adaptive adversaries
We put forward a model of action-based randomization mechanisms to analyse
quantitative information flow (QIF) under generic leakage functions, and under
possibly adaptive adversaries. This model subsumes many of the QIF models
proposed so far. Our main contributions include the following: (1) we identify
mild general conditions on the leakage function under which it is possible to
derive general and significant results on adaptive QIF; (2) we contrast the
efficiency of adaptive and non-adaptive strategies, showing that the latter are
as efficient as the former in terms of length up to an expansion factor bounded
by the number of available actions; (3) we show that the maximum information
leakage over strategies, given a finite time horizon, can be expressed in terms
of a Bellman equation. This can be used to compute an optimal finite strategy
recursively, by resorting to standard methods like backward induction.Comment: Revised and extended version of conference paper with the same title
appeared in Proc. of FORTE 2014, LNC
- …