Formulas vs. Circuits for Small Distance Connectivity

We give the first super-polynomial separation in the power of bounded-depth boolean formulas vs. circuits. Specifically, we consider the problem Distance $k(n)$ Connectivity, which asks whether two specified nodes in a graph of size $n$ are connected by a path of length at most $k(n)$. This problem is solvable (by the recursive doubling technique) on {\bf circuits} of depth $O(\log k)$ and size $O(kn^3)$. In contrast, we show that solving this problem on {\bf formulas} of depth $\log n/(\log\log n)^{O(1)}$ requires size $n^{\Omega(\log k)}$ for all $k(n) \leq \log\log n$. As corollaries: (i) It follows that polynomial-size circuits for Distance $k(n)$ Connectivity require depth $\Omega(\log k)$ for all $k(n) \leq \log\log n$. This matches the upper bound from recursive doubling and improves a previous $\Omega(\log\log k)$ lower bound of Beame, Pitassi and Impagliazzo [BIP98]. (ii) We get a tight lower bound of $s^{\Omega(d)}$ on the size required to simulate size-$s$ depth-$d$ circuits by depth-$d$ formulas for all $s(n) = n^{O(1)}$ and $d(n) \leq \log\log\log n$. No lower bound better than $s^{\Omega(1)}$ was previously known for any $d(n) \nleq O(1)$. Our proof technique is centered on a new notion of pathset complexity, which roughly speaking measures the minimum cost of constructing a set of (partial) paths in a universe of size $n$ via the operations of union and relational join, subject to certain density constraints. Half of our proof shows that bounded-depth formulas solving Distance $k(n)$ Connectivity imply upper bounds on pathset complexity. The other half is a combinatorial lower bound on pathset complexity

Near-optimal small-depth lower bounds for small distance connectivity

We show that any depth-$d$ circuit for determining whether an $n$-node graph has an $s$-to-$t$ path of length at most $k$ must have size $n^{\Omega(k^{1/d}/d)}$. The previous best circuit size lower bounds for this problem were $n^{k^{\exp(-O(d))}}$ (due to Beame, Impagliazzo, and Pitassi [BIP98]) and $n^{\Omega((\log k)/d)}$ (following from a recent formula size lower bound of Rossman [Ros14]). Our lower bound is quite close to optimal, since a simple construction gives depth-$d$ circuits of size $n^{O(k^{2/d})}$ for this problem (and strengthening our bound even to $n^{k^{\Omega(1/d)}}$ would require proving that undirected connectivity is not in $\mathsf{NC^1}.$) Our proof is by reduction to a new lower bound on the size of small-depth circuits computing a skewed variant of the "Sipser functions" that have played an important role in classical circuit lower bounds [Sip83, Yao85, H{\aa}s86]. A key ingredient in our proof of the required lower bound for these Sipser-like functions is the use of \emph{random projections}, an extension of random restrictions which were recently employed in [RST15]. Random projections allow us to obtain sharper quantitative bounds while employing simpler arguments, both conceptually and technically, than in the previous works [Ajt89, BPU92, BIP98, Ros14]

An Atypical Survey of Typical-Case Heuristic Algorithms

Heuristic approaches often do so well that they seem to pretty much always give the right answer. How close can heuristic algorithms get to always giving the right answer, without inducing seismic complexity-theoretic consequences? This article first discusses how a series of results by Berman, Buhrman, Hartmanis, Homer, Longpr\'{e}, Ogiwara, Sch\"{o}ening, and Watanabe, from the early 1970s through the early 1990s, explicitly or implicitly limited how well heuristic algorithms can do on NP-hard problems. In particular, many desirable levels of heuristic success cannot be obtained unless severe, highly unlikely complexity class collapses occur. Second, we survey work initiated by Goldreich and Wigderson, who showed how under plausible assumptions deterministic heuristics for randomized computation can achieve a very high frequency of correctness. Finally, we consider formal ways in which theory can help explain the effectiveness of heuristics that solve NP-hard problems in practice.Comment: This article is currently scheduled to appear in the December 2012 issue of SIGACT New

Improved Pseudorandom Generators from Pseudorandom Multi-Switching Lemmas

We give the best known pseudorandom generators for two touchstone classes in unconditional derandomization: an $\varepsilon$-PRG for the class of size-$M$ depth-$d$ $\mathsf{AC}^0$ circuits with seed length $\log(M)^{d+O(1)}\cdot \log(1/\varepsilon)$, and an $\varepsilon$-PRG for the class of $S$-sparse $\mathbb{F}_2$ polynomials with seed length $2^{O(\sqrt{\log S})}\cdot \log(1/\varepsilon)$. These results bring the state of the art for unconditional derandomization of these classes into sharp alignment with the state of the art for computational hardness for all parameter settings: improving on the seed lengths of either PRG would require breakthrough progress on longstanding and notorious circuit lower bounds. The key enabling ingredient in our approach is a new \emph{pseudorandom multi-switching lemma}. We derandomize recently-developed \emph{multi}-switching lemmas, which are powerful generalizations of H{\aa}stad's switching lemma that deal with \emph{families} of depth-two circuits. Our pseudorandom multi-switching lemma---a randomness-efficient algorithm for sampling restrictions that simultaneously simplify all circuits in a family---achieves the parameters obtained by the (full randomness) multi-switching lemmas of Impagliazzo, Matthews, and Paturi [IMP12] and H{\aa}stad [H{\aa}s14]. This optimality of our derandomization translates into the optimality (given current circuit lower bounds) of our PRGs for $\mathsf{AC}^0$ and sparse $\mathbb{F}_2$ polynomials

An Improved Homomorphism Preservation Theorem From Lower Bounds in Circuit Complexity

Previous work of the author [Rossmann\u2708] showed that the Homomorphism Preservation Theorem of classical model theory remains valid when its statement is restricted to finite structures. In this paper, we give a new proof of this result via a reduction to lower bounds in circuit complexity, specifically on the AC0 formula size of the colored subgraph isomorphism problem. Formally, we show the following: if a first-order sentence of quantifier-rank k is preserved under homomorphisms on finite structures, then it is equivalent on finite structures to an existential-positive sentence of quantifier-rank poly(k). Quantitatively, this improves the result of [Rossmann\u2708], where the upper bound on quantifier-rank is a non-elementary function of k

Near-optimal small-depth lower bounds for small distance connectivity

We show that any depth-d circuit for determining whether an n-node graph has an s-to-t path of length at most k must have size nΩ(k1/d/d). The previous best circuit size lower bounds for this problem were nkexp(−O(d)) (due to Beame, Impagliazzo, and Pitassi [BIP98]) and nΩ((log k)/d) (following from a recent formula size lower bound of Rossman [Ros14]). Our lower bound is quite close to optimal, since a simple construction gives depth-d circuits of size nO(k2/d) for this problem (and strengthening our bound even to nkΩ(1/d) would require proving that undirected connectivity is not in NC1.) Our proof is by reduction to a new lower bound on the size of small-depth circuits computing a skewed variant of the “Sipser functions” that have played an important role in classical circuit lower bounds [Sip83, Yao85, H˚as86]. A key ingredient in our proof of the required lower bound for these Sipser-like functions is the use of random projections, an extension of random restrictions which were recently employed in [RST15]. Random projections allow us to obtain sharper quantitative bounds while employing simpler arguments, both conceptually and technically, than in the previous works [Ajt89, BPU92, BIP98, Ros14]

A Bayesian approach for inferring neuronal connectivity from calcium fluorescent imaging data

Deducing the structure of neural circuits is one of the central problems of modern neuroscience. Recently-introduced calcium fluorescent imaging methods permit experimentalists to observe network activity in large populations of neurons, but these techniques provide only indirect observations of neural spike trains, with limited time resolution and signal quality. In this work we present a Bayesian approach for inferring neural circuitry given this type of imaging data. We model the network activity in terms of a collection of coupled hidden Markov chains, with each chain corresponding to a single neuron in the network and the coupling between the chains reflecting the network's connectivity matrix. We derive a Monte Carlo Expectation--Maximization algorithm for fitting the model parameters; to obtain the sufficient statistics in a computationally-efficient manner, we introduce a specialized blockwise-Gibbs algorithm for sampling from the joint activity of all observed neurons given the observed fluorescence data. We perform large-scale simulations of randomly connected neuronal networks with biophysically realistic parameters and find that the proposed methods can accurately infer the connectivity in these networks given reasonable experimental and computational constraints. In addition, the estimation accuracy may be improved significantly by incorporating prior knowledge about the sparseness of connectivity in the network, via standard L$_1$ penalization methods.Comment: Published in at http://dx.doi.org/10.1214/09-AOAS303 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org