370 research outputs found
Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls
We propose a rank- variant of the classical Frank-Wolfe algorithm to solve
convex optimization over a trace-norm ball. Our algorithm replaces the top
singular-vector computation (-SVD) in Frank-Wolfe with a top-
singular-vector computation (-SVD), which can be done by repeatedly applying
-SVD times. Alternatively, our algorithm can be viewed as a rank-
restricted version of projected gradient descent. We show that our algorithm
has a linear convergence rate when the objective function is smooth and
strongly convex, and the optimal solution has rank at most . This improves
the convergence rate and the total time complexity of the Frank-Wolfe method
and its variants.Comment: In NIPS 201
Much Faster Algorithms for Matrix Scaling
We develop several efficient algorithms for the classical \emph{Matrix
Scaling} problem, which is used in many diverse areas, from preconditioning
linear systems to approximation of the permanent. On an input
matrix , this problem asks to find diagonal (scaling) matrices and
(if they exist), so that -approximates a doubly
stochastic, or more generally a matrix with prescribed row and column sums.
We address the general scaling problem as well as some important special
cases. In particular, if has nonzero entries, and if there exist
and with polynomially large entries such that is doubly stochastic,
then we can solve the problem in total complexity .
This greatly improves on the best known previous results, which were either
or .
Our algorithms are based on tailor-made first and second order techniques,
combined with other recent advances in continuous optimization, which may be of
independent interest for solving similar problems
- …