371 research outputs found
Randomized Sketches of Convex Programs with Sharp Guarantees
Random projection (RP) is a classical technique for reducing storage and
computational costs. We analyze RP-based approximations of convex programs, in
which the original optimization problem is approximated by the solution of a
lower-dimensional problem. Such dimensionality reduction is essential in
computation-limited settings, since the complexity of general convex
programming can be quite high (e.g., cubic for quadratic programs, and
substantially higher for semidefinite programs). In addition to computational
savings, random projection is also useful for reducing memory usage, and has
useful properties for privacy-sensitive optimization. We prove that the
approximation ratio of this procedure can be bounded in terms of the geometry
of constraint set. For a broad class of random projections, including those
based on various sub-Gaussian distributions as well as randomized Hadamard and
Fourier transforms, the data matrix defining the cost function can be projected
down to the statistical dimension of the tangent cone of the constraints at the
original solution, which is often substantially smaller than the original
dimension. We illustrate consequences of our theory for various cases,
including unconstrained and -constrained least squares, support vector
machines, low-rank matrix estimation, and discuss implications on
privacy-sensitive optimization and some connections with de-noising and
compressed sensing
Scalable and Robust Community Detection with Randomized Sketching
This paper explores and analyzes the unsupervised clustering of large
partially observed graphs. We propose a scalable and provable randomized
framework for clustering graphs generated from the stochastic block model. The
clustering is first applied to a sub-matrix of the graph's adjacency matrix
associated with a reduced graph sketch constructed using random sampling. Then,
the clusters of the full graph are inferred based on the clusters extracted
from the sketch using a correlation-based retrieval step. Uniform random node
sampling is shown to improve the computational complexity over clustering of
the full graph when the cluster sizes are balanced. A new random degree-based
node sampling algorithm is presented which significantly improves upon the
performance of the clustering algorithm even when clusters are unbalanced. This
algorithm improves the phase transitions for matrix-decomposition-based
clustering with regard to computational complexity and minimum cluster size,
which are shown to be nearly dimension-free in the low inter-cluster
connectivity regime. A third sampling technique is shown to improve balance by
randomly sampling nodes based on spatial distribution. We provide analysis and
numerical results using a convex clustering algorithm based on matrix
completion
Iterative Hessian sketch: Fast and accurate solution approximation for constrained least-squares
We study randomized sketching methods for approximately solving least-squares
problem with a general convex constraint. The quality of a least-squares
approximation can be assessed in different ways: either in terms of the value
of the quadratic objective function (cost approximation), or in terms of some
distance measure between the approximate minimizer and the true minimizer
(solution approximation). Focusing on the latter criterion, our first main
result provides a general lower bound on any randomized method that sketches
both the data matrix and vector in a least-squares problem; as a surprising
consequence, the most widely used least-squares sketch is sub-optimal for
solution approximation. We then present a new method known as the iterative
Hessian sketch, and show that it can be used to obtain approximations to the
original least-squares problem using a projection dimension proportional to the
statistical complexity of the least-squares minimizer, and a logarithmic number
of iterations. We illustrate our general theory with simulations for both
unconstrained and constrained versions of least-squares, including
-regularization and nuclear norm constraints. We also numerically
demonstrate the practicality of our approach in a real face expression
classification experiment
- β¦