Bounds on an exponential sum arising in Boolean circuit complexity

We study exponential sums of the form S = 2-n ∑x∈{0,1}n em (h(x))eq (p(x)), where m, q ∈ Z+ are relatively prime, p is a polynomial with coefficients in Zq, and h(x) = a(x1 +⋯+ xn) for some 1 ≤ a \u3c m. We prove an upper bound of the form 2-Ω(n) on S . This generalizes a result of J. Bourgain, who establishes this bound in the case where q is odd. This bound has consequences in Boolean circuit complexity. © Académie des sciences. Published by Elsevier SAS. All rights reserved

Incomplete Quadratic Exponential Sums in Several Variables

We consider incomplete exponential sums in several variables of the form S(f,n,m) = \frac{1}{2^n} \sum_{x_1 \in \{-1,1\}} ... \sum_{x_n \in \{-1,1\}} x_1 ... x_n e^{2\pi i f(x)/p}, where m>1 is odd and f is a polynomial of degree d with coefficients in Z/mZ. We investigate the conjecture, originating in a problem in computational complexity, that for each fixed d and m the maximum norm of S(f,n,m) converges exponentially fast to 0 as n grows to infinity. The conjecture is known to hold in the case when m=3 and d=2, but existing methods for studying incomplete exponential sums appear to be insufficient to resolve the question for an arbitrary odd modulus m, even when d=2. In the present paper we develop three separate techniques for studying the problem in the case of quadratic f, each of which establishes a different special case of the conjecture. We show that a bound of the required sort holds for almost all quadratic polynomials, a stronger form of the conjecture holds for all quadratic polynomials with no more than 10 variables, and for arbitrarily many variables the conjecture is true for a class of quadratic polynomials having a special form.Comment: 31 pages (minor corrections from original draft, references to new results in the subject, publication information

Uniqueness of Optimal Mod 3 Circuits for Parity

We prove that the quadratic polynomials modulo $3$ with the largest correlation with parity are unique up to permutation of variables and constant factors. As a consequence of our result, we completely characterize the smallest MAJ~$circ mbox{MOD}_3 circ { m AND}_2$ circuits that compute parity, where a MAJ~$circ mbox{MOD}_3 circ { m AND}_2$ circuit is one that has a majority gate as output, a middle layer of MOD$_3$ gates and a bottom layer of AND gates of fan-in $2$. We also prove that the sub-optimal circuits exhibit a stepped behavior: any sub-optimal circuits of this class that compute parity must have size at least a factor of $frac{2}{sqrt{3}}$ times the optimal size. This verifies, for the special case of $m=3$, two conjectures made by Due~{n}ez, Miller, Roy and Straubing (Journal of Number Theory, 2006) for general MAJ~$circ mathrm{MOD}_m circ { m AND}_2$ circuits for any odd $m$. The correlation and circuit bounds are obtained by studying the associated exponential sums, based on some of the techniques developed by Green (JCSS, 2004). We regard this as a step towards obtaining tighter bounds both for the $m ot = 3$ quadratic case as well as for higher degrees

Limits on Representing Boolean Functions by Linear Combinations of Simple Functions: Thresholds, ReLUs, and Low-Degree Polynomials

We consider the problem of representing Boolean functions exactly by "sparse" linear combinations (over $\mathbb{R}$) of functions from some "simple" class ${\cal C}$. In particular, given ${\cal C}$ we are interested in finding low-complexity functions lacking sparse representations. When ${\cal C}$ is the set of PARITY functions or the set of conjunctions, this sort of problem has a well-understood answer, the problem becomes interesting when ${\cal C}$ is "overcomplete" and the set of functions is not linearly independent. We focus on the cases where ${\cal C}$ is the set of linear threshold functions, the set of rectified linear units (ReLUs), and the set of low-degree polynomials over a finite field, all of which are well-studied in different contexts. We provide generic tools for proving lower bounds on representations of this kind. Applying these, we give several new lower bounds for "semi-explicit" Boolean functions. For example, we show there are functions in nondeterministic quasi-polynomial time that require super-polynomial size: $\bullet$ Depth-two neural networks with sign activation function, a special case of depth-two threshold circuit lower bounds. $\bullet$ Depth-two neural networks with ReLU activation function. $\bullet$ $\mathbb{R}$-linear combinations of $O(1)$-degree $\mathbb{F}_p$-polynomials, for every prime $p$ (related to problems regarding Higher-Order "Uncertainty Principles"). We also obtain a function in $E^{NP}$ requiring $2^{\Omega(n)}$ linear combinations. $\bullet$ $\mathbb{R}$-linear combinations of $ACC \circ THR$ circuits of polynomial size (further generalizing the recent lower bounds of Murray and the author). (The above is a shortened abstract. For the full abstract, see the paper.

The correlation between parity and quadratic polynomials mod 3

We prove exponentially small upper bounds on the correlation between parity and quadratic polynomials mod3. One corollary of this is that in order to compute parity, circuits consisting of a threshold gate at the top, mod3 gates in the middle, and AND gates of fan-in two at the inputs must be of size 2 Ω(n). This is the first result of this type for general mod3 subcircuits with ANDs of fan-in greater than 1. This yields an exponential improvement over a long-standing result of Smolensky, answering a question recently posed by Alon and Beigel. The proof uses a novel inductive estimate of the relevant exponential sums introduced by Cai, Green and Thierauf. The exponential sum and correlation bounds presented here are tight. © 2004 Elsevier Inc. All rights reserved

The Correlation Between Parity and Quadratic Polynomials

We prove exponentially small upper bounds on the correlation between parity and quadratic polynomials mod 3. One corollary of this is that in order to compute parity, circuits consisting of a threshold gate at the top, mod 3 gates in the middle, and AND gates of fan-in two at the inputs must be of size 2 . This is the first result of this type for general mod 3 subcircuits with ANDs of fan-in greater than 1. This yields an exponential improvement over a long-standing result of Smolensky, answering a question recently posed by Alon and Beigel. The proof uses a novel inductive estimate of the relevant exponential sums introduced by Cai, Green and Thierauf. The exponential sum and correlation bounds presented here are tight