14 research outputs found
An average-case depth hierarchy theorem for Boolean circuits
We prove an average-case depth hierarchy theorem for Boolean circuits over
the standard basis of , , and gates.
Our hierarchy theorem says that for every , there is an explicit
-variable Boolean function , computed by a linear-size depth- formula,
which is such that any depth- circuit that agrees with on fraction of all inputs must have size This
answers an open question posed by H{\aa}stad in his Ph.D. thesis.
Our average-case depth hierarchy theorem implies that the polynomial
hierarchy is infinite relative to a random oracle with probability 1,
confirming a conjecture of H{\aa}stad, Cai, and Babai. We also use our result
to show that there is no "approximate converse" to the results of Linial,
Mansour, Nisan and Boppana on the total influence of small-depth circuits, thus
answering a question posed by O'Donnell, Kalai, and Hatami.
A key ingredient in our proof is a notion of \emph{random projections} which
generalize random restrictions
A Near-Optimal Depth-Hierarchy Theorem for Small-Depth Multilinear Circuits
We study the size blow-up that is necessary to convert an algebraic circuit
of product-depth to one of product-depth in the multilinear
setting.
We show that for every positive
there is an explicit multilinear polynomial on variables
that can be computed by a multilinear formula of product-depth and
size , but not by any multilinear circuit of product-depth and
size less than . This result is tight up to the
constant implicit in the double exponent for all
This strengthens a result of Raz and Yehudayoff (Computational Complexity
2009) who prove a quasipolynomial separation for constant-depth multilinear
circuits, and a result of Kayal, Nair and Saha (STACS 2016) who give an
exponential separation in the case
Our separating examples may be viewed as algebraic analogues of variants of
the Graph Reachability problem studied by Chen, Oliveira, Servedio and Tan
(STOC 2016), who used them to prove lower bounds for constant-depth Boolean
circuits
Super-Linear Gate and Super-Quadratic Wire Lower Bounds for Depth-Two and Depth-Three Threshold Circuits
In order to formally understand the power of neural computing, we first need
to crack the frontier of threshold circuits with two and three layers, a regime
that has been surprisingly intractable to analyze. We prove the first
super-linear gate lower bounds and the first super-quadratic wire lower bounds
for depth-two linear threshold circuits with arbitrary weights, and depth-three
majority circuits computing an explicit function.
We prove that for all , the
linear-time computable Andreev's function cannot be computed on a
-fraction of -bit inputs by depth-two linear threshold
circuits of gates, nor can it be computed with
wires. This establishes an average-case
``size hierarchy'' for threshold circuits, as Andreev's function is computable
by uniform depth-two circuits of linear threshold gates, and by
uniform depth-three circuits of majority gates.
We present a new function in based on small-biased sets, which
we prove cannot be computed by a majority vote of depth-two linear threshold
circuits with gates, nor with
wires.
We give tight average-case (gate and wire) complexity results for
computing PARITY with depth-two threshold circuits; the answer turns out to be
the same as for depth-two majority circuits.
The key is a new random restriction lemma for linear threshold functions. Our
main analytical tool is the Littlewood-Offord Lemma from additive
combinatorics
Affine Extractors and AC0-Parity
We study a simple and general template for constructing affine extractors by composing a linear transformation with resilient functions. Using this we show that good affine extractors can be computed by non-explicit circuits of various types, including AC0-Xor circuits: AC0 circuits with a layer of parity gates at the input. We also show that one-sided extractors can be computed by small DNF-Xor circuits, and separate these circuits from other well-studied classes. As a further motivation for studying DNF-Xor circuits we show that if they can approximate inner product then small AC0-Xor circuits can compute it exactly - a long-standing open problem
PAC-learning gains of Turing machines over circuits and neural networks
A caveat to many applications of the current Deep Learning approach is the
need for large-scale data. One improvement suggested by Kolmogorov Complexity
results is to apply the minimum description length principle with
computationally universal models. We study the potential gains in sample
efficiency that this approach can bring in principle. We use polynomial-time
Turing machines to represent computationally universal models and Boolean
circuits to represent Artificial Neural Networks (ANNs) acting on
finite-precision digits.
Our analysis unravels direct links between our question and Computational
Complexity results. We provide lower and upper bounds on the potential gains in
sample efficiency between the MDL applied with Turing machines instead of ANNs.
Our bounds depend on the bit-size of the input of the Boolean function to be
learned. Furthermore, we highlight close relationships between classical open
problems in Circuit Complexity and the tightness of these