Average-case complexity of detecting cliques

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2010.Cataloged from PDF version of thesis.Includes bibliographical references (p. 79-83).The computational problem of testing whether a graph contains a complete subgraph of size k is among the most fundamental problems studied in theoretical computer science. This thesis is concerned with proving lower bounds for k-CLIQUE, as this problem is known. Our results show that, in certain models of computation, solving k-CLIQUE in the average case requires Q(nk/4) resources (moreover, k/4 is tight). Here the models of computation are bounded-depth Boolean circuits and unbounded-depth monotone circuits, the complexity measure is the number of gates, and the input distributions are random graphs with an appropriate density of edges. Such random graphs (the well-studied Erdos-Renyi random graphs) are widely believed to be a source of computationally hard instances for clique problems (as Karp suggested in 1976). Our results are the first unconditional lower bounds supporting this hypothesis. For bounded-depth Boolean circuits, our average-case hardness result significantly improves the previous worst-case lower bounds of Q(nk/Poly(d)) for depth-d circuits. In particular, our lower bound of Q(nk/ 4 ) has no noticeable dependence on d for circuits of depth d ; k- log n/log log n, thus bypassing the previous "size-depth tradeoffs". As a consequence, we obtain a novel Size Hierarchy Theorem for uniform AC0 . A related application answers a longstanding open question in finite model theory (raised by Immerman in 1982): we show that the hierarchy of bounded-variable fragments of first-order logic is strict on finite ordered graphs. Additional results of this thesis characterize the average-case descriptive complexity of k-CLIQUE through the lens of first-order logic.by Benjamin Rossman.Ph.D

A proof that Reed-Muller codes achieve Shannon capacity on symmetric channels

Reed-Muller codes were introduced in 1954, with a simple explicit construction based on polynomial evaluations, and have long been conjectured to achieve Shannon capacity on symmetric channels. Major progress was made towards a proof over the last decades; using combinatorial weight enumerator bounds, a breakthrough on the erasure channel from sharp thresholds, hypercontractivity arguments, and polarization theory. Another major progress recently established that the bit error probability vanishes slowly below capacity. However, when channels allow for errors, the results of Bourgain-Kalai do not apply for converting a vanishing bit to a vanishing block error probability, neither do the known weight enumerator bounds. The conjecture that RM codes achieve Shannon capacity on symmetric channels, with high probability of recovering the codewords, has thus remained open. This paper closes the conjecture's proof. It uses a new recursive boosting framework, which aggregates the decoding of codeword restrictions on `subspace-sunflowers', handling their dependencies via an $L_p$ Boolean Fourier analysis, and using a list-decoding argument with a weight enumerator bound from Sberlo-Shpilka. The proof does not require a vanishing bit error probability for the base case, but only a non-trivial probability, obtained here for general symmetric codes. This gives in particular a shortened and tightened argument for the vanishing bit error probability result of Reeves-Pfister, and with prior works, it implies the strong wire-tap secrecy of RM codes on pure-state classical-quantum channels

Combinatorial geometry of neural codes, neural data analysis, and neural networks

This dissertation explores applications of discrete geometry in mathematical neuroscience. We begin with convex neural codes, which model the activity of hippocampal place cells and other neurons with convex receptive fields. In Chapter 4, we introduce order-forcing, a tool for constraining convex realizations of codes, and use it to construct new examples of non-convex codes with no local obstructions. In Chapter 5, we relate oriented matroids to convex neural codes, showing that a code has a realization with convex polytopes iff it is the image of a representable oriented matroid under a neural code morphism. We also show that determining whether a code is convex is at least as difficult as determining whether an oriented matroid is representable, implying that the problem of determining whether a code is convex is NP-hard. Next, we turn to the problem of the underlying rank of a matrix. This problem is motivated by the problem of determining the dimensionality of (neural) data which has been corrupted by an unknown monotone transformation. In Chapter 6, we introduce two tools for computing underlying rank, the minimal nodes and the Radon rank. We apply these to analyze calcium imaging data from a larval zebrafish. In Chapter 7, we explore the underlying rank in more detail, establish connections to oriented matroid theory, and show that computing underlying rank is also NP-hard. Finally, we study the dynamics of threshold-linear networks (TLNs), a simple model of the activity of neural circuits. In Chapter 9, we describe the nullcline arrangement of a threshold linear network, and show that a subset of its chambers are an attracting set. In Chapter 10, we focus on combinatorial threshold linear networks (CTLNs), which are TLNs defined from a directed graph. We prove that if the graph of a CTLN is a directed acyclic graph, then all trajectories of the CTLN approach a fixed point.Comment: 193 pages, 69 figure

Numerical methods for optimal transport and optimal information transport on the sphere

The primary contribution of this dissertation is in developing and analyzing efficient, provably convergent numerical schemes for solving fully nonlinear elliptic partial differential equation arising from Optimal Transport on the sphere, and then applying and adapting the methods to two specific engineering applications: the reflector antenna problem and the moving mesh methods problem. For these types of nonlinear partial differential equations, many numerical studies have been done in recent years, the vast majority in subsets of Euclidean space. In this dissertation, the first major goal is to develop convergent schemes for the sphere. However, another goal of this dissertation is application-centered, that is evaluating whether the partial differential equation techniques using Optimal Transport are actually the best methods for solving such problems. The reflector antenna is an optics inverse problem where one finds the shape of a reflector surface in order to refocus light into a prescribed far-field output intensity. This problem can be solved using Optimal Transport. The moving mesh methods problem is an adaptive mesh technique where one redistributes the density of the vertices of a mesh without tangling the edges connecting the vertices. Both Optimal Transport and Optimal Information Transport approaches can be used in solving this problem. The Monge Problem of Optimal Transport is concerned with computing the “optimal” mapping between two probability distributions. This actually can define a Riemannian distance between probability measures in a probability space. An-other choice of Riemannian metric on this space, the infinite-dimensional Fisher-Rao metric, gives an “information geometric” structure to the space of probability measures. It turns out that a simple partial differential equation can be solved for a mapping that relates to the underlying information geometry given by the Fisher-Rao metric. Solving for such an “information geometric” mapping is known as Optimal Information Transport. In this dissertation, a convergence framework is first established for com-puting the solution to the partial differential equation formulation of Optimal Transport on the sphere. This convergence framework uses geodesic normal coor-dinates to perform computations in local tangent planes. The numerical scheme also has a control on the Lipschitz constant of the discrete solution, which allows a convergence theorem for consistent and monotone discretizations to be proved in the absence of a comparison principle for the partial differential equation. Then, a finite-difference scheme for the partial differential equation formulation of Opti-mal Transport on the sphere is constructed which satisfies the hypotheses of the convergence theorem. An explicit formula for the mixed Hessian term is derived for two different cost functions. In order to construct a monotone discretization, discrete Laplacian terms are carefully added into the scheme. Current work has established convergence rates for solutions of monotone discretizations of linear elliptic partial differential equations on compact 2D manifolds without boundary. The goal is to then generalize these linearized arguments for the Optimal Transport case on the sphere. Computations are performed for the reflector antenna problem. Other ad hoc schemes exist for computing the reflector antenna problem, but the proposed scheme is the most efficient provably convergent scheme. Further adaptations are made that allow for the scheme to deal with non-smooth cases more explicitly. For the moving mesh methods problem, a comparison of computations via Optimal Transport and Optimal Information Transport is performed for the sphere using provably convergent monotone schemes for both computations. These comparisons show the merits of using Optimal Information Transport for some challenging computations. Optimal Information Transport also seems like a natural generalization to other compact 2D surfaces beyond the sphere