Computational Complexity of Certifying Restricted Isometry Property

Given a matrix $A$ with $n$ rows, a number $k<n$, and $0<\delta < 1$, $A$ is $(k,\delta)$-RIP (Restricted Isometry Property) if, for any vector $x \in \mathbb{R}^n$, with at most $k$ non-zero co-ordinates, $$(1-\delta) \|x\|_2 \leq \|A x\|_2 \leq (1+\delta)\|x\|_2$$ In many applications, such as compressed sensing and sparse recovery, it is desirable to construct RIP matrices with a large $k$ and a small $\delta$. Given the efficacy of random constructions in generating useful RIP matrices, the problem of certifying the RIP parameters of a matrix has become important. In this paper, we prove that it is hard to approximate the RIP parameters of a matrix assuming the Small-Set-Expansion-Hypothesis. Specifically, we prove that for any arbitrarily large constant $C>0$ and any arbitrarily small constant $0<\delta<1$, there exists some $k$ such that given a matrix $M$, it is SSE-Hard to distinguish the following two cases: - (Highly RIP) $M$ is $(k,\delta)$-RIP. - (Far away from RIP) $M$ is not $(k/C, 1-\delta)$-RIP. Most of the previous results on the topic of hardness of RIP certification only hold for certification when $\delta=o(1)$. In practice, it is of interest to understand the complexity of certifying a matrix with $\delta$ being close to $\sqrt{2}-1$, as it suffices for many real applications to have matrices with $\delta = \sqrt{2}-1$. Our hardness result holds for any constant $\delta$. Specifically, our result proves that even if $\delta$ is indeed very small, i.e. the matrix is in fact \emph{strongly RIP}, certifying that the matrix exhibits \emph{weak RIP} itself is SSE-Hard. In order to prove the hardness result, we prove a variant of the Cheeger's Inequality for sparse vectors

Computational barriers in minimax submatrix detection

This paper studies the minimax detection of a small submatrix of elevated mean in a large matrix contaminated by additive Gaussian noise. To investigate the tradeoff between statistical performance and computational cost from a complexity-theoretic perspective, we consider a sequence of discretized models which are asymptotically equivalent to the Gaussian model. Under the hypothesis that the planted clique detection problem cannot be solved in randomized polynomial time when the clique size is of smaller order than the square root of the graph size, the following phase transition phenomenon is established: when the size of the large matrix $p\to\infty$, if the submatrix size $k=\Theta(p^{\alpha})$ for any $\alpha\in(0,{2}/{3})$, computational complexity constraints can incur a severe penalty on the statistical performance in the sense that any randomized polynomial-time test is minimax suboptimal by a polynomial factor in $p$; if $k=\Theta(p^{\alpha})$ for any $\alpha\in({2}/{3},1)$, minimax optimal detection can be attained within constant factors in linear time. Using Schatten norm loss as a representative example, we show that the hardness of attaining the minimax estimation rate can crucially depend on the loss function. Implications on the hardness of support recovery are also obtained.Comment: Published at http://dx.doi.org/10.1214/14-AOS1300 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Finding Planted Cliques in Sublinear Time

We study the planted clique problem in which a clique of size $k$ is planted in an Erd\H{o}s-R\'enyi graph of size $n$ and one wants to recover this planted clique. For $k=\Omega(\sqrt{n})$, polynomial time algorithms can find the planted clique. The fastest such algorithms run in time linear $O(n^2)$ (or nearly linear) in the size of the input [FR10,DGGP14,DM15a]. In this work, we initiate the development of sublinear time algorithms that find the planted clique when $k=\omega(\sqrt{n \log \log n})$. Our algorithms can recover the clique in time $\widetilde{O}\left(n+(\frac{n}{k})^{3}\right)=\widetilde{O}\left(n^{\frac{3}{2}}\right)$ when $k=\Omega(\sqrt{n\log n})$, and in time $\widetilde{O}\left(n^2/\exp{\left(\frac{k^2}{24n}\right)}\right)$ for $\omega(\sqrt{n\log \log n})=k=o(\sqrt{n\log{n}})$. An ${\Omega}(n)$ running time lower bound for the planted clique recovery problem follows easily from the results of [RS19] and therefore our recovery algorithms are optimal whenever $k = \Omega(n^{\frac{2}{3}})$. As the lower bound of [RS19] builds on purely information theoretic arguments, it cannot provide a detection lower bound stronger than $\widetilde{\Omega}(\frac{n^2}{k^2})$. Since our algorithms for $k = \Omega(\sqrt{n \log n})$ run in time $\widetilde{O}\left(\frac{n^3}{k^3} + n\right)$, we show stronger lower bounds based on computational hardness assumptions. With a slightly different notion of the planted clique problem we show that the Planted Clique Conjecture implies the following. A natural family of non-adaptive algorithms---which includes our algorithms for clique detection---cannot reliably solve the planted clique detection problem in time $O\left( \frac{n^{3-\delta}}{k^3}\right)$ for any constant $\delta>0$. Thus we provide evidence that if detecting small cliques is hard, it is also likely that detecting large cliques is not \textit{too} easy