577 research outputs found
What Can We Learn Privately?
Learning problems form an important category of computational tasks that
generalizes many of the computations researchers apply to large real-life data
sets. We ask: what concept classes can be learned privately, namely, by an
algorithm whose output does not depend too heavily on any one input or specific
training example? More precisely, we investigate learning algorithms that
satisfy differential privacy, a notion that provides strong confidentiality
guarantees in contexts where aggregate information is released about a database
containing sensitive information about individuals. We demonstrate that,
ignoring computational constraints, it is possible to privately agnostically
learn any concept class using a sample size approximately logarithmic in the
cardinality of the concept class. Therefore, almost anything learnable is
learnable privately: specifically, if a concept class is learnable by a
(non-private) algorithm with polynomial sample complexity and output size, then
it can be learned privately using a polynomial number of samples. We also
present a computationally efficient private PAC learner for the class of parity
functions. Local (or randomized response) algorithms are a practical class of
private algorithms that have received extensive investigation. We provide a
precise characterization of local private learning algorithms. We show that a
concept class is learnable by a local algorithm if and only if it is learnable
in the statistical query (SQ) model. Finally, we present a separation between
the power of interactive and noninteractive local learning algorithms.Comment: 35 pages, 2 figure
Heavy Hitters and the Structure of Local Privacy
We present a new locally differentially private algorithm for the heavy
hitters problem which achieves optimal worst-case error as a function of all
standardly considered parameters. Prior work obtained error rates which depend
optimally on the number of users, the size of the domain, and the privacy
parameter, but depend sub-optimally on the failure probability.
We strengthen existing lower bounds on the error to incorporate the failure
probability, and show that our new upper bound is tight with respect to this
parameter as well. Our lower bound is based on a new understanding of the
structure of locally private protocols. We further develop these ideas to
obtain the following general results beyond heavy hitters.
Advanced Grouposition: In the local model, group privacy for
users degrades proportionally to , instead of linearly in
as in the central model. Stronger group privacy yields improved max-information
guarantees, as well as stronger lower bounds (via "packing arguments"), over
the central model.
Building on a transformation of Bassily and Smith (STOC 2015), we
give a generic transformation from any non-interactive approximate-private
local protocol into a pure-private local protocol. Again in contrast with the
central model, this shows that we cannot obtain more accurate algorithms by
moving from pure to approximate local privacy
The Role of Interactivity in Local Differential Privacy
We study the power of interactivity in local differential privacy. First, we
focus on the difference between fully interactive and sequentially interactive
protocols. Sequentially interactive protocols may query users adaptively in
sequence, but they cannot return to previously queried users. The vast majority
of existing lower bounds for local differential privacy apply only to
sequentially interactive protocols, and before this paper it was not known
whether fully interactive protocols were more powerful. We resolve this
question. First, we classify locally private protocols by their
compositionality, the multiplicative factor by which the sum of a
protocol's single-round privacy parameters exceeds its overall privacy
guarantee. We then show how to efficiently transform any fully interactive
-compositional protocol into an equivalent sequentially interactive protocol
with an blowup in sample complexity. Next, we show that our reduction is
tight by exhibiting a family of problems such that for any , there is a
fully interactive -compositional protocol which solves the problem, while no
sequentially interactive protocol can solve the problem without at least an
factor more examples. We then turn our attention to
hypothesis testing problems. We show that for a large class of compound
hypothesis testing problems --- which include all simple hypothesis testing
problems as a special case --- a simple noninteractive test is optimal among
the class of all (possibly fully interactive) tests
How NOT to Build an Infinite Lottery Machine
The sustained failure of efforts to design an infinite lottery machine using ordinary probabilistic randomizers is traced back to a problem familiar to set theorists: we have no constructive prescriptions for probabilistically non- measurable sets. Yet construction of such sets is required if we are to be able to read the result of an infinite lottery machine that is built from ordinary probabilistic randomizers. All such designs face a dilemma: they can provide an accessible (readable) result with probability zero; or an inaccessible result with probability greater than zero
Fast and Efficient Compressive Sensing using Structurally Random Matrices
This paper introduces a new framework of fast and efficient sensing matrices
for practical compressive sensing, called Structurally Random Matrix (SRM). In
the proposed framework, we pre-randomize a sensing signal by scrambling its
samples or flipping its sample signs and then fast-transform the randomized
samples and finally, subsample the transform coefficients as the final sensing
measurements. SRM is highly relevant for large-scale, real-time compressive
sensing applications as it has fast computation and supports block-based
processing. In addition, we can show that SRM has theoretical sensing
performance comparable with that of completely random sensing matrices.
Numerical simulation results verify the validity of the theory as well as
illustrate the promising potentials of the proposed sensing framework
- …