A Birthday Repetition Theorem and Complexity of Approximating Dense CSPs

A $(k \times l)$-birthday repetition $\mathcal{G}^{k \times l}$ of a two-prover game $\mathcal{G}$ is a game in which the two provers are sent random sets of questions from $\mathcal{G}$ of sizes $k$ and $l$ respectively. These two sets are sampled independently uniformly among all sets of questions of those particular sizes. We prove the following birthday repetition theorem: when $\mathcal{G}$ satisfies some mild conditions, $val(\mathcal{G}^{k \times l})$ decreases exponentially in $\Omega(kl/n)$ where $n$ is the total number of questions. Our result positively resolves an open question posted by Aaronson, Impagliazzo and Moshkovitz (CCC 2014). As an application of our birthday repetition theorem, we obtain new fine-grained hardness of approximation results for dense CSPs. Specifically, we establish a tight trade-off between running time and approximation ratio for dense CSPs by showing conditional lower bounds, integrality gaps and approximation algorithms. In particular, for any sufficiently large $i$ and for every $k \geq 2$, we show the following results: - We exhibit an $O(q^{1/i})$-approximation algorithm for dense Max $k$-CSPs with alphabet size $q$ via $O_k(i)$-level of Sherali-Adams relaxation. - Through our birthday repetition theorem, we obtain an integrality gap of $q^{1/i}$ for $\tilde\Omega_k(i)$-level Lasserre relaxation for fully-dense Max $k$-CSP. - Assuming that there is a constant $\epsilon > 0$ such that Max 3SAT cannot be approximated to within $(1-\epsilon)$ of the optimal in sub-exponential time, our birthday repetition theorem implies that any algorithm that approximates fully-dense Max $k$-CSP to within a $q^{1/i}$ factor takes $(nq)^{\tilde \Omega_k(i)}$ time, almost tightly matching the algorithmic result based on Sherali-Adams relaxation.Comment: 45 page

Limitations of Algebraic Approaches to Graph Isomorphism Testing

We investigate the power of graph isomorphism algorithms based on algebraic reasoning techniques like Gr\"obner basis computation. The idea of these algorithms is to encode two graphs into a system of equations that are satisfiable if and only if if the graphs are isomorphic, and then to (try to) decide satisfiability of the system using, for example, the Gr\"obner basis algorithm. In some cases this can be done in polynomial time, in particular, if the equations admit a bounded degree refutation in an algebraic proof systems such as Nullstellensatz or polynomial calculus. We prove linear lower bounds on the polynomial calculus degree over all fields of characteristic different from 2 and also linear lower bounds for the degree of Positivstellensatz calculus derivations. We compare this approach to recently studied linear and semidefinite programming approaches to isomorphism testing, which are known to be related to the combinatorial Weisfeiler-Lehman algorithm. We exactly characterise the power of the Weisfeiler-Lehman algorithm in terms of an algebraic proof system that lies between degree-k Nullstellensatz and degree-k polynomial calculus

Subsampling Mathematical Relaxations and Average-case Complexity

We initiate a study of when the value of mathematical relaxations such as linear and semidefinite programs for constraint satisfaction problems (CSPs) is approximately preserved when restricting the instance to a sub-instance induced by a small random subsample of the variables. Let $C$ be a family of CSPs such as 3SAT, Max-Cut, etc., and let $\Pi$ be a relaxation for $C$, in the sense that for every instance $P\in C$, $\Pi(P)$ is an upper bound the maximum fraction of satisfiable constraints of $P$. Loosely speaking, we say that subsampling holds for $C$ and $\Pi$ if for every sufficiently dense instance $P \in C$ and every $\epsilon>0$, if we let $P'$ be the instance obtained by restricting $P$ to a sufficiently large constant number of variables, then $\Pi(P') \in (1\pm \epsilon)\Pi(P)$. We say that weak subsampling holds if the above guarantee is replaced with $\Pi(P')=1-\Theta(\gamma)$ whenever $\Pi(P)=1-\gamma$. We show: 1. Subsampling holds for the BasicLP and BasicSDP programs. BasicSDP is a variant of the relaxation considered by Raghavendra (2008), who showed it gives an optimal approximation factor for every CSP under the unique games conjecture. BasicLP is the linear programming analog of BasicSDP. 2. For tighter versions of BasicSDP obtained by adding additional constraints from the Lasserre hierarchy, weak subsampling holds for CSPs of unique games type. 3. There are non-unique CSPs for which even weak subsampling fails for the above tighter semidefinite programs. Also there are unique CSPs for which subsampling fails for the Sherali-Adams linear programming hierarchy. As a corollary of our weak subsampling for strong semidefinite programs, we obtain a polynomial-time algorithm to certify that random geometric graphs (of the type considered by Feige and Schechtman, 2002) of max-cut value $1-\gamma$ have a cut value at most $1-\gamma/10$.Comment: Includes several more general results that subsume the previous version of the paper

Faster SDP hierarchy solvers for local rounding algorithms

Convex relaxations based on different hierarchies of linear/semi-definite programs have been used recently to devise approximation algorithms for various optimization problems. The approximation guarantee of these algorithms improves with the number of {\em rounds} $r$ in the hierarchy, though the complexity of solving (or even writing down the solution for) the $r$'th level program grows as $n^{\Omega(r)}$ where $n$ is the input size. In this work, we observe that many of these algorithms are based on {\em local} rounding procedures that only use a small part of the SDP solution (of size $n^{O(1)} 2^{O(r)}$ instead of $n^{\Omega(r)}$). We give an algorithm to find the requisite portion in time polynomial in its size. The challenge in achieving this is that the required portion of the solution is not fixed a priori but depends on other parts of the solution, sometimes in a complicated iterative manner. Our solver leads to $n^{O(1)} 2^{O(r)}$ time algorithms to obtain the same guarantees in many cases as the earlier $n^{O(r)}$ time algorithms based on $r$ rounds of the Lasserre hierarchy. In particular, guarantees based on $O(\log n)$ rounds can be realized in polynomial time. We develop and describe our algorithm in a fairly general abstract framework. The main technical tool in our work, which might be of independent interest in convex optimization, is an efficient ellipsoid algorithm based separation oracle for convex programs that can output a {\em certificate of infeasibility with restricted support}. This is used in a recursive manner to find a sequence of consistent points in nested convex bodies that "fools" local rounding algorithms.Comment: 30 pages, 8 figure

Lower bounds on the size of semidefinite programming relaxations

We introduce a method for proving lower bounds on the efficacy of semidefinite programming (SDP) relaxations for combinatorial problems. In particular, we show that the cut, TSP, and stable set polytopes on $n$-vertex graphs are not the linear image of the feasible region of any SDP (i.e., any spectrahedron) of dimension less than $2^{n^c}$, for some constant $c > 0$. This result yields the first super-polynomial lower bounds on the semidefinite extension complexity of any explicit family of polytopes. Our results follow from a general technique for proving lower bounds on the positive semidefinite rank of a matrix. To this end, we establish a close connection between arbitrary SDPs and those arising from the sum-of-squares SDP hierarchy. For approximating maximum constraint satisfaction problems, we prove that SDPs of polynomial-size are equivalent in power to those arising from degree-$O(1)$ sum-of-squares relaxations. This result implies, for instance, that no family of polynomial-size SDP relaxations can achieve better than a 7/8-approximation for MAX-3-SAT