25,214 research outputs found
Weighted Polynomial Approximations: Limits for Learning and Pseudorandomness
Polynomial approximations to boolean functions have led to many positive
results in computer science. In particular, polynomial approximations to the
sign function underly algorithms for agnostically learning halfspaces, as well
as pseudorandom generators for halfspaces. In this work, we investigate the
limits of these techniques by proving inapproximability results for the sign
function.
Firstly, the polynomial regression algorithm of Kalai et al. (SIAM J. Comput.
2008) shows that halfspaces can be learned with respect to log-concave
distributions on in the challenging agnostic learning model. The
power of this algorithm relies on the fact that under log-concave
distributions, halfspaces can be approximated arbitrarily well by low-degree
polynomials. We ask whether this technique can be extended beyond log-concave
distributions, and establish a negative result. We show that polynomials of any
degree cannot approximate the sign function to within arbitrarily low error for
a large class of non-log-concave distributions on the real line, including
those with densities proportional to .
Secondly, we investigate the derandomization of Chernoff-type concentration
inequalities. Chernoff-type tail bounds on sums of independent random variables
have pervasive applications in theoretical computer science. Schmidt et al.
(SIAM J. Discrete Math. 1995) showed that these inequalities can be established
for sums of random variables with only -wise independence,
for a tail probability of . We show that their results are tight up to
constant factors.
These results rely on techniques from weighted approximation theory, which
studies how well functions on the real line can be approximated by polynomials
under various distributions. We believe that these techniques will have further
applications in other areas of computer science.Comment: 22 page
Discovering the roots: Uniform closure results for algebraic classes under factoring
Newton iteration (NI) is an almost 350 years old recursive formula that
approximates a simple root of a polynomial quite rapidly. We generalize it to a
matrix recurrence (allRootsNI) that approximates all the roots simultaneously.
In this form, the process yields a better circuit complexity in the case when
the number of roots is small but the multiplicities are exponentially
large. Our method sets up a linear system in unknowns and iteratively
builds the roots as formal power series. For an algebraic circuit
of size we prove that each factor has size at most a
polynomial in: and the degree of the squarefree part of . Consequently,
if is a -hard polynomial then any nonzero multiple
is equally hard for arbitrary positive 's, assuming
that is at most .
It is an old open question whether the class of poly()-sized formulas
(resp. algebraic branching programs) is closed under factoring. We show that
given a polynomial of degree and formula (resp. ABP) size
we can find a similar size formula (resp. ABP) factor in
randomized poly()-time. Consequently, if determinant requires
size formula, then the same can be said about any of its
nonzero multiples.
As part of our proofs, we identify a new property of multivariate polynomial
factorization. We show that under a random linear transformation ,
completely factors via power series roots. Moreover, the
factorization adapts well to circuit complexity analysis. This with allRootsNI
are the techniques that help us make progress towards the old open problems,
supplementing the large body of classical results and concepts in algebraic
circuit factorization (eg. Zassenhaus, J.NT 1969, Kaltofen, STOC 1985-7 \&
Burgisser, FOCS 2001).Comment: 33 Pages, No figure
Computational Hardness of Certifying Bounds on Constrained PCA Problems
Given a random n×n symmetric matrix W drawn from the Gaussian orthogonal ensemble (GOE), we consider the problem of certifying an upper bound on the maximum value of the quadratic form x⊤Wx over all vectors x in a constraint set S⊂Rn. For a certain class of normalized constraint sets S we show that, conditional on certain complexity-theoretic assumptions, there is no polynomial-time algorithm certifying a better upper bound than the largest eigenvalue of W. A notable special case included in our results is the hypercube S={±1/n−−√}n, which corresponds to the problem of certifying bounds on the Hamiltonian of the Sherrington-Kirkpatrick spin glass model from statistical physics.
Our proof proceeds in two steps. First, we give a reduction from the detection problem in the negatively-spiked Wishart model to the above certification problem. We then give evidence that this Wishart detection problem is computationally hard below the classical spectral threshold, by showing that no low-degree polynomial can (in expectation) distinguish the spiked and unspiked models. This method for identifying computational thresholds was proposed in a sequence of recent works on the sum-of-squares hierarchy, and is believed to be correct for a large class of problems. Our proof can be seen as constructing a distribution over symmetric matrices that appears computationally indistinguishable from the GOE, yet is supported on matrices whose maximum quadratic form over x∈S is much larger than that of a GOE matrix.ISSN:1868-896
Approximate resilience, monotonicity, and the complexity of agnostic learning
A function is -resilient if all its Fourier coefficients of degree at
most are zero, i.e., is uncorrelated with all low-degree parities. We
study the notion of of Boolean
functions, where we say that is -approximately -resilient if
is -close to a -valued -resilient function in
distance. We show that approximate resilience essentially characterizes the
complexity of agnostic learning of a concept class over the uniform
distribution. Roughly speaking, if all functions in a class are far from
being -resilient then can be learned agnostically in time and
conversely, if contains a function close to being -resilient then
agnostic learning of in the statistical query (SQ) framework of Kearns has
complexity of at least . This characterization is based on the
duality between approximation by degree- polynomials and
approximate -resilience that we establish. In particular, it implies that
approximation by low-degree polynomials, known to be sufficient for
agnostic learning over product distributions, is in fact necessary.
Focusing on monotone Boolean functions, we exhibit the existence of
near-optimal -approximately
-resilient monotone functions for all
. Prior to our work, it was conceivable even that every monotone
function is -far from any -resilient function. Furthermore, we
construct simple, explicit monotone functions based on and that are close to highly resilient functions. Our constructions are
based on a fairly general resilience analysis and amplification. These
structural results, together with the characterization, imply nearly optimal
lower bounds for agnostic learning of monotone juntas
Limitations of semidefinite programs for separable states and entangled games
Semidefinite programs (SDPs) are a framework for exact or approximate
optimization that have widespread application in quantum information theory. We
introduce a new method for using reductions to construct integrality gaps for
SDPs. These are based on new limitations on the sum-of-squares (SoS) hierarchy
in approximating two particularly important sets in quantum information theory,
where previously no -round integrality gaps were known: the set of
separable (i.e. unentangled) states, or equivalently, the
norm of a matrix, and the set of quantum correlations; i.e. conditional
probability distributions achievable with local measurements on a shared
entangled state. In both cases no-go theorems were previously known based on
computational assumptions such as the Exponential Time Hypothesis (ETH) which
asserts that 3-SAT requires exponential time to solve. Our unconditional
results achieve the same parameters as all of these previous results (for
separable states) or as some of the previous results (for quantum
correlations). In some cases we can make use of the framework of
Lee-Raghavendra-Steurer (LRS) to establish integrality gaps for any SDP, not
only the SoS hierarchy. Our hardness result on separable states also yields a
dimension lower bound of approximate disentanglers, answering a question of
Watrous and Aaronson et al. These results can be viewed as limitations on the
monogamy principle, the PPT test, the ability of Tsirelson-type bounds to
restrict quantum correlations, as well as the SDP hierarchies of
Doherty-Parrilo-Spedalieri, Navascues-Pironio-Acin and Berta-Fawzi-Scholz.Comment: 47 pages. v2. small changes, fixes and clarifications. published
versio
A Polylogarithmic PRG for Degree Threshold Functions in the Gaussian Setting
We devise a new pseudorandom generator against degree 2 polynomial threshold
functions in the Gaussian setting. We manage to achieve error with
seed length polylogarithmic in and the dimension, and exponential
improvement over previous constructions
- …