17 research outputs found
MCMC Learning
The theory of learning under the uniform distribution is rich and deep, with
connections to cryptography, computational complexity, and the analysis of
boolean functions to name a few areas. This theory however is very limited due
to the fact that the uniform distribution and the corresponding Fourier basis
are rarely encountered as a statistical model.
A family of distributions that vastly generalizes the uniform distribution on
the Boolean cube is that of distributions represented by Markov Random Fields
(MRF). Markov Random Fields are one of the main tools for modeling high
dimensional data in many areas of statistics and machine learning.
In this paper we initiate the investigation of extending central ideas,
methods and algorithms from the theory of learning under the uniform
distribution to the setup of learning concepts given examples from MRF
distributions. In particular, our results establish a novel connection between
properties of MCMC sampling of MRFs and learning under the MRF distribution.Comment: 28 pages, 1 figur
From average case complexity to improper learning complexity
The basic problem in the PAC model of computational learning theory is to
determine which hypothesis classes are efficiently learnable. There is
presently a dearth of results showing hardness of learning problems. Moreover,
the existing lower bounds fall short of the best known algorithms.
The biggest challenge in proving complexity results is to establish hardness
of {\em improper learning} (a.k.a. representation independent learning).The
difficulty in proving lower bounds for improper learning is that the standard
reductions from -hard problems do not seem to apply in this
context. There is essentially only one known approach to proving lower bounds
on improper learning. It was initiated in (Kearns and Valiant 89) and relies on
cryptographic assumptions.
We introduce a new technique for proving hardness of improper learning, based
on reductions from problems that are hard on average. We put forward a (fairly
strong) generalization of Feige's assumption (Feige 02) about the complexity of
refuting random constraint satisfaction problems. Combining this assumption
with our new technique yields far reaching implications. In particular,
1. Learning 's is hard.
2. Agnostically learning halfspaces with a constant approximation ratio is
hard.
3. Learning an intersection of halfspaces is hard.Comment: 34 page
Homomorphic evaluation requires depth
We show that homomorphic evaluation of any non-trivial functionality of sufficiently many inputs with respect to any CPA secure homomorphic encryption scheme cannot be implemented by circuits of polynomial size and constant depth, i.e., in the class AC0. In contrast, we observe that there exist ordinary public-key encryption schemes of quasipolynomial security in AC0 assuming noisy parities are exponentially hard to learn. We view this as evidence that homomorphic evaluation is inherently more complex than basic operations in encryption schemes
Low-degree learning and the metric entropy of polynomials
Let be the class of all functions
on the -dimensional discrete hypercube of degree at most . In the first
part of this paper, we prove that any (deterministic or randomized) algorithm
which learns with -accuracy requires at
least queries for large enough ,
thus establishing the sharpness as of a recent upper bound of
Eskenazis and Ivanisvili (2021). To do this, we show that the -packing
numbers of the
concept class satisfy the two-sided estimate
for large enough , where are universal
constants. In the second part of the paper, we present a logarithmic upper
bound for the randomized query complexity of classes of bounded approximate
polynomials whose Fourier spectra are concentrated on few subsets. As an
application, we prove new estimates for the number of random queries required
to learn approximate juntas of a given degree, functions with rapidly decaying
Fourier tails and constant depth circuits of given size. Finally, we obtain
bounds for the number of queries required to learn the polynomial class
without error in the query and random example models
Cryptographic Sensing
Is it possible to measure a physical object in a way that makes the measurement signals unintelligible to an external observer? Alternatively, can one learn a natural concept by using a contrived training set that makes the labeled examples useless without the line of thought that has led to their choice?
We initiate a study of ``cryptographic sensing\u27\u27 problems of this type, presenting definitions, positive and negative results, and directions for further research