Algorithms and Lower Bounds in Circuit Complexity

Computational complexity theory aims to understand what problems can be efficiently solved by computation. This thesis studies computational complexity in the model of Boolean circuits. Boolean circuits provide a basic mathematical model for computation and play a central role in complexity theory, with important applications in separations of complexity classes, algorithm design, and pseudorandom constructions. In this thesis, we investigate various types of circuit models such as threshold circuits, Boolean formulas, and their extensions, focusing on obtaining complexity-theoretic lower bounds and algorithmic upper bounds for these circuits. (1) Algorithms and lower bounds for generalized threshold circuits: We extend the study of linear threshold circuits, circuits with gates computing linear threshold functions, to the more powerful model of polynomial threshold circuits where the gates can compute polynomial threshold functions. We obtain hardness and meta-algorithmic results for this circuit model, including strong average-case lower bounds, satisfiability algorithms, and derandomization algorithms for constant-depth polynomial threshold circuits with super-linear wire complexity. (2) Algorithms and lower bounds for enhanced formulas: We investigate the model of Boolean formulas whose leaf gates can compute complex functions. In particular, we study De Morgan formulas whose leaf gates are functions with "low communication complexity". Such gates can capture a broad class of functions including symmetric functions and polynomial threshold functions. We obtain new and improved results in terms of lower bounds and meta-algorithms (satisfiability, derandomization, and learning) for such enhanced formulas. (3) Circuit lower bounds for MCSP: We study circuit lower bounds for the Minimum Circuit Size Problem (MCSP), the fundamental problem of deciding whether a given function (in the form of a truth table) can be computed by small circuits. We get new and improved lower bounds for MCSP that nearly match the best-known lower bounds against several well-studied circuit models such as Boolean formulas and constant-depth circuits

Approximate Degree and the Complexity of Depth Three Circuits

Threshold weight, margin complexity, and Majority-of-Threshold circuit size are basic complexity measures of Boolean functions that arise in learning theory, communication complexity, and circuit complexity. Each of these measures might exhibit a chasm at depth three: namely, all polynomial size Boolean circuits of depth two have polynomial complexity under the measure, but there may exist Boolean circuits of depth three that have essentially maximal complexity exp(Theta(n)). However, existing techniques are far from showing this: for all three measures, the best lower bound for depth three circuits is exp(Omega(n^{2/5})). Moreover, prior methods exclusively study block-composed functions. Such methods appear intrinsically unable to prove lower bounds better than exp(Omega(sqrt{n})) even for depth four circuits, and have yet to prove lower bounds better than exp(Omega(sqrt{n})) for circuits of any constant depth. We take a step toward showing that all of these complexity measures indeed exhibit a chasm at depth three. Specifically, for any arbitrarily small constant delta > 0, we exhibit a depth three circuit of polynomial size (in fact, an O(log n)-decision list) of complexity exp(Omega(n^{1/2-delta})) under each of these measures. Our methods go beyond the block-composed functions studied in prior work, and hence may not be subject to the same barriers. Accordingly, we suggest natural candidate functions that may exhibit stronger bounds

Multiple Product Modulo Arbitrary Numbers

AbstractLetnbinary numbers of lengthnbe given. The Boolean function “Multiple Product”MPnasks for (some binary representation of ) the value of their product. It has been shown (K.-Y. Siu and V. Roychowdhury, On optimal depth threshold circuits for multiplication and related problems,SIAM J. Discrete Math.7, 285–292 (1994)) that this function can be computed in polynomial-size threshold circuits of depth 4. For many other arithmetic functions, circuits of depth 3 are known. They are mostly based on the fact that the value of the considered function modulo some prime numbers p can be computed easily in threshold circuits of depth 2. In this paper, we investigate the complexity of computingMPnmodulomby depth-2 threshold circuits. It turns out that for all but a few integersm, exponential size is required. In particular, it is shown that form∈{2, 4, 8}, polynomial-size circuits exist, form∈{3, 6, 12, 24}, the question remains open and in all other cases, exponential-size circuits are required. The result still holds if we allowmto grow withn

On the Power of Democratic Networks

Linear Threshold Boolean units (LTU) are the basic processing components of artificial neural networks of Boolean activations. Quantization of their parameters is a central question in hardware implementation, when numerical technologies are used to store the configuration of the circuit. In the previous studies on the circuit complexity of feedforward neural networks, no differences had been made between a network with ``small'' integer weights and one composed of majority units (LTU with weights in {-1,0, 1}), since any connection of weight w (w integer) can be simulated by |w| connections of value Sgn(w). This paper will focus on the circuit complexity of democratic networks, i.e. circuits of majority units with at most one connection between each pair of units. The main results presented are the following: any Boolean function can be computed by a depth-3 non-degenerate democratic network and can be expressed as a linear threshold function of majorities; AT-LEAST-k and AT-MOST-k are computable by a depth-2, polynomial size democratic network; the smallest sizes of depth-2 circuits computing PARITY are identical for a democratic network and for a usual network; the VC of the class of the majority functions is n 1, i.e. equal to that of the class of any linear threshold functions

Polynomial Threshold Functions, AC^0 Functions and Spectral Norms

The class of polynomial-threshold functions is studied using harmonic analysis, and the results are used to derive lower bounds related to AC^0 functions. A Boolean function is polynomial threshold if it can be represented as a sign function of a sparse polynomial (one that consists of a polynomial number of terms). The main result is that polynomial-threshold functions can be characterized by means of their spectral representation. In particular, it is proved that a Boolean function whose L_1 spectral norm is bounded by a polynomial in n is a polynomial-threshold function, and that a Boolean function whose L_∞^(-1) spectral norm is not bounded by a polynomial in n is not a polynomial-threshold function. Some results for AC^0 functions are derived

Average-Case Lower Bounds and Satisfiability Algorithms for Small Threshold Circuits

We show average-case lower bounds for explicit Boolean functions against bounded-depth threshold circuits with a superlinear number of wires. We show that for each integer d > 1, there is epsilon_d > 0 such that Parity has correlation at most 1/n^{Omega(1)} with depth-d threshold circuits which have at most n^{1+epsilon_d} wires, and the Generalized Andreev Function has correlation at most 1/2^{n^{Omega(1)}} with depth-d threshold circuits which have at most n^{1+epsilon_d} wires. Previously, only worst-case lower bounds in this setting were known [Impagliazzo/Paturi/Saks, SIAM J. Comp., 1997]. We use our ideas to make progress on several related questions. We give satisfiability algorithms beating brute force search for depth-$d$ threshold circuits with a superlinear number of wires. These are the first such algorithms for depth greater than 2. We also show that Parity cannot be computed by polynomial-size AC^0 circuits with n^{o(1)} general threshold gates. Previously no lower bound for Parity in this setting could handle more than log(n) gates. This result also implies subexponential-time learning algorithms for AC^0 with n^{o(1)} threshold gates under the uniform distribution. In addition, we give almost optimal bounds for the number of gates in a depth-d threshold circuit computing Parity on average, and show average-case lower bounds for threshold formulas ofany depth. Our techniques include adaptive random restrictions, anti-concentration and the structural theory of linear threshold functions, and bounded-read Chernoff bounds