Subsampled Power Iteration: a Unified Algorithm for Block Models and Planted CSP's

We present an algorithm for recovering planted solutions in two well-known models, the stochastic block model and planted constraint satisfaction problems, via a common generalization in terms of random bipartite graphs. Our algorithm matches up to a constant factor the best-known bounds for the number of edges (or constraints) needed for perfect recovery and its running time is linear in the number of edges used. The time complexity is significantly better than both spectral and SDP-based approaches. The main contribution of the algorithm is in the case of unequal sizes in the bipartition (corresponding to odd uniformity in the CSP). Here our algorithm succeeds at a significantly lower density than the spectral approaches, surpassing a barrier based on the spectral norm of a random matrix. Other significant features of the algorithm and analysis include (i) the critical use of power iteration with subsampling, which might be of independent interest; its analysis requires keeping track of multiple norms of an evolving solution (ii) it can be implemented statistically, i.e., with very limited access to the input distribution (iii) the algorithm is extremely simple to implement and runs in linear time, and thus is practical even for very large instances

Improved Parameterized Algorithms for Constraint Satisfaction

For many constraint satisfaction problems, the algorithm which chooses a random assignment achieves the best possible approximation ratio. For instance, a simple random assignment for {\sc Max-E3-Sat} allows 7/8-approximation and for every \eps >0 there is no polynomial-time (7/8+\eps)-approximation unless P=NP. Another example is the {\sc Permutation CSP} of bounded arity. Given the expected fraction $\rho$ of the constraints satisfied by a random assignment (i.e. permutation), there is no (\rho+\eps)-approximation algorithm for every \eps >0, assuming the Unique Games Conjecture (UGC). In this work, we consider the following parameterization of constraint satisfaction problems. Given a set of $m$ constraints of constant arity, can we satisfy at least $\rho m +k$ constraint, where $\rho$ is the expected fraction of constraints satisfied by a random assignment? {\sc Constraint Satisfaction Problems above Average} have been posed in different forms in the literature \cite{Niedermeier2006,MahajanRamanSikdar09}. We present a faster parameterized algorithm for deciding whether $m/2+k/2$ equations can be simultaneously satisfied over ${\mathbb F}_2$. As a consequence, we obtain $O(k)$-variable bikernels for {\sc boolean CSPs} of arity $c$ for every fixed $c$, and for {\sc permutation CSPs} of arity 3. This implies linear bikernels for many problems under the "above average" parameterization, such as {\sc Max-$c$-Sat}, {\sc Set-Splitting}, {\sc Betweenness} and {\sc Max Acyclic Subgraph}. As a result, all the parameterized problems we consider in this paper admit $2^{O(k)}$-time algorithms. We also obtain non-trivial hybrid algorithms for every Max $c$-CSP: for every instance $I$, we can either approximate $I$ beyond the random assignment threshold in polynomial time, or we can find an optimal solution to $I$ in subexponential time.Comment: A preliminary version of this paper has been accepted for IPEC 201

Certifying solution geometry in random CSPs: counts, clusters and balance

An active topic in the study of random constraint satisfaction problems (CSPs) is the geometry of the space of satisfying or almost satisfying assignments as the function of the density, for which a precise landscape of predictions has been made via statistical physics-based heuristics. In parallel, there has been a recent flurry of work on refuting random constraint satisfaction problems, via nailing refutation thresholds for spectral and semidefinite programming-based algorithms, and also on counting solutions to CSPs. Inspired by this, the starting point for our work is the following question: what does the solution space for a random CSP look like to an efficient algorithm? In pursuit of this inquiry, we focus on the following problems about random Boolean CSPs at the densities where they are unsatisfiable but no refutation algorithm is known. 1. Counts. For every Boolean CSP we give algorithms that with high probability certify a subexponential upper bound on the number of solutions. We also give algorithms to certify a bound on the number of large cuts in a Gaussian-weighted graph, and the number of large independent sets in a random $d$-regular graph. 2. Clusters. For Boolean $3$CSPs we give algorithms that with high probability certify an upper bound on the number of clusters of solutions. 3. Balance. We also give algorithms that with high probability certify that there are no "unbalanced" solutions, i.e., solutions where the fraction of $+1$s deviates significantly from $50\%$. Finally, we also provide hardness evidence suggesting that our algorithms for counting are optimal

Satisfiability threshold for random regular NAE-SAT

We consider the random regular $k$-NAE-SAT problem with $n$ variables each appearing in exactly $d$ clauses. For all $k$ exceeding an absolute constant $k_0$, we establish explicitly the satisfiability threshold $d_*=d_*(k)$. We prove that for $d<d_*$ the problem is satisfiable with high probability while for $d>d_*$ the problem is unsatisfiable with high probability. If the threshold $d_*$ lands exactly on an integer, we show that the problem is satisfiable with probability bounded away from both zero and one. This is the first result to locate the exact satisfiability threshold in a random constraint satisfaction problem exhibiting the condensation phenomenon identified by Krzakala et al. (2007). Our proof verifies the one-step replica symmetry breaking formalism for this model. We expect our methods to be applicable to a broad range of random constraint satisfaction problems and combinatorial problems on random graphs

Random Max-CSPs Inherit Algorithmic Hardness from Spin Glasses

We study random constraint satisfaction problems (CSPs) in the unsatisfiable regime. We relate the structure of near-optimal solutions for any Max-CSP to that for an associated spin glass on the hypercube, using the Guerra-Toninelli interpolation from statistical physics. The noise stability polynomial of the CSP's predicate is, up to a constant, the mixture polynomial of the associated spin glass. We prove two main consequences: 1) We relate the maximum fraction of constraints that can be satisfied in a random Max-CSP to the ground state energy density of the corresponding spin glass. Since the latter value can be computed with the Parisi formula, we provide numerical values for some popular CSPs. 2) We prove that a Max-CSP possesses generalized versions of the overlap gap property if and only if the same holds for the corresponding spin glass. We transfer results from Huang et al. [arXiv:2110.07847, 2021] to obstruct algorithms with overlap concentration on a large class of Max-CSPs. This immediately includes local classical and local quantum algorithms.Comment: 41 pages, 1 tabl

On the Complexity of Random Satisfiability Problems with Planted Solutions

The problem of identifying a planted assignment given a random $k$-SAT formula consistent with the assignment exhibits a large algorithmic gap: while the planted solution becomes unique and can be identified given a formula with $O(n\log n)$ clauses, there are distributions over clauses for which the best known efficient algorithms require $n^{k/2}$ clauses. We propose and study a unified model for planted $k$-SAT, which captures well-known special cases. An instance is described by a planted assignment $\sigma$ and a distribution on clauses with $k$ literals. We define its distribution complexity as the largest $r$ for which the distribution is not $r$-wise independent ($1 \le r \le k$ for any distribution with a planted assignment). Our main result is an unconditional lower bound, tight up to logarithmic factors, for statistical (query) algorithms [Kearns 1998, Feldman et. al 2012], matching known upper bounds, which, as we show, can be implemented using a statistical algorithm. Since known approaches for problems over distributions have statistical analogues (spectral, MCMC, gradient-based, convex optimization etc.), this lower bound provides a rigorous explanation of the observed algorithmic gap. The proof introduces a new general technique for the analysis of statistical query algorithms. It also points to a geometric paring phenomenon in the space of all planted assignments. We describe consequences of our lower bounds to Feige's refutation hypothesis [Feige 2002] and to lower bounds on general convex programs that solve planted $k$-SAT. Our bounds also extend to other planted $k$-CSP models, and, in particular, provide concrete evidence for the security of Goldreich's one-way function and the associated pseudorandom generator when used with a sufficiently hard predicate [Goldreich 2000].Comment: Extended abstract appeared in STOC 201