943 research outputs found
A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem
We prove that with high probability over the choice of a random graph
from the Erd\H{o}s-R\'enyi distribution , the -time degree
Sum-of-Squares semidefinite programming relaxation for the clique problem
will give a value of at least for some constant
. This yields a nearly tight bound on the value of this
program for any degree . Moreover we introduce a new framework
that we call \emph{pseudo-calibration} to construct Sum of Squares lower
bounds. This framework is inspired by taking a computational analog of Bayesian
probability theory. It yields a general recipe for constructing good
pseudo-distributions (i.e., dual certificates for the Sum-of-Squares
semidefinite program), and sheds further light on the ways in which this
hierarchy differs from others.Comment: 55 page
Fast, Sample-Efficient, Affine-Invariant Private Mean and Covariance Estimation for Subgaussian Distributions
We present a fast, differentially private algorithm for high-dimensional
covariance-aware mean estimation with nearly optimal sample complexity. Only
exponential-time estimators were previously known to achieve this guarantee.
Given samples from a (sub-)Gaussian distribution with unknown mean
and covariance , our -differentially private
estimator produces such that as long as . The
Mahalanobis error metric measures the distance
between and relative to ; it characterizes the error
of the sample mean. Our algorithm runs in time , where is the matrix multiplication exponent.
We adapt an exponential-time approach of Brown, Gaboardi, Smith, Ullman, and
Zakynthinou (2021), giving efficient variants of stable mean and covariance
estimation subroutines that also improve the sample complexity to the nearly
optimal bound above.
Our stable covariance estimator can be turned to private covariance
estimation for unrestricted subgaussian distributions. With
samples, our estimate is accurate in spectral norm. This is the first such
algorithm using samples, answering an open question posed by Alabi
et al. (2022). With samples, our estimate is accurate in
Frobenius norm. This leads to a fast, nearly optimal algorithm for private
learning of unrestricted Gaussian distributions in TV distance.
Duchi, Haque, and Kuditipudi (2023) obtained similar results independently
and concurrently.Comment: 44 pages. New version fixes typos and includes additional exposition
and discussion of related wor
The power of sum-of-squares for detecting hidden structures
We study planted problems---finding hidden structures in random noisy
inputs---through the lens of the sum-of-squares semidefinite programming
hierarchy (SoS). This family of powerful semidefinite programs has recently
yielded many new algorithms for planted problems, often achieving the best
known polynomial-time guarantees in terms of accuracy of recovered solutions
and robustness to noise. One theme in recent work is the design of spectral
algorithms which match the guarantees of SoS algorithms for planted problems.
Classical spectral algorithms are often unable to accomplish this: the twist in
these new spectral algorithms is the use of spectral structure of matrices
whose entries are low-degree polynomials of the input variables. We prove that
for a wide class of planted problems, including refuting random constraint
satisfaction problems, tensor and sparse PCA, densest-k-subgraph, community
detection in stochastic block models, planted clique, and others, eigenvalues
of degree-d matrix polynomials are as powerful as SoS semidefinite programs of
roughly degree d. For such problems it is therefore always possible to match
the guarantees of SoS without solving a large semidefinite program. Using
related ideas on SoS algorithms and low-degree matrix polynomials (and inspired
by recent work on SoS and the planted clique problem by Barak et al.), we prove
new nearly-tight SoS lower bounds for the tensor and sparse principal component
analysis problems. Our lower bounds for sparse principal component analysis are
the first to suggest that going beyond existing algorithms for this problem may
require sub-exponential time
- …