30,148 research outputs found
The Bit Complexity of Efficient Continuous Optimization
We analyze the bit complexity of efficient algorithms for fundamental
optimization problems, such as linear regression, -norm regression, and
linear programming (LP). State-of-the-art algorithms are iterative, and in
terms of the number of arithmetic operations, they match the current time
complexity of multiplying two -by- matrices (up to polylogarithmic
factors). However, previous work has typically assumed infinite precision
arithmetic, and due to complicated inverse maintenance techniques, the actual
running times of these algorithms are unknown. To settle the running time and
bit complexity of these algorithms, we demonstrate that a core common
subroutine, known as \emph{inverse maintenance}, is backward-stable.
Additionally, we show that iterative approaches for solving constrained
weighted regression problems can be accomplished with bounded-error
pre-conditioners. Specifically, we prove that linear programs can be solved
approximately in matrix multiplication time multiplied by polylog factors that
depend on the condition number of the matrix and the inner and outer
radius of the LP problem. -norm regression can be solved approximately in
matrix multiplication time multiplied by polylog factors in . Lastly,
linear regression can be solved approximately in input-sparsity time multiplied
by polylog factors in . Furthermore, we present results for achieving
lower than matrix multiplication time for -norm regression by utilizing
faster solvers for sparse linear systems.Comment: 71 page
Structured Semidefinite Programming for Recovering Structured Preconditioners
We develop a general framework for finding approximately-optimal
preconditioners for solving linear systems. Leveraging this framework we obtain
improved runtimes for fundamental preconditioning and linear system solving
problems including the following. We give an algorithm which, given positive
definite with
nonzero entries, computes an -optimal
diagonal preconditioner in time , where is the
optimal condition number of the rescaled matrix. We give an algorithm which,
given that is either the pseudoinverse
of a graph Laplacian matrix or a constant spectral approximation of one, solves
linear systems in in time. Our diagonal
preconditioning results improve state-of-the-art runtimes of
attained by general-purpose semidefinite programming, and our solvers improve
state-of-the-art runtimes of where is the
current matrix multiplication constant. We attain our results via new
algorithms for a class of semidefinite programs (SDPs) we call
matrix-dictionary approximation SDPs, which we leverage to solve an associated
problem we call matrix-dictionary recovery.Comment: Merge of arXiv:1812.06295 and arXiv:2008.0172
- …