5,033 research outputs found
A randomized Kaczmarz algorithm with exponential convergence
The Kaczmarz method for solving linear systems of equations is an iterative
algorithm that has found many applications ranging from computer tomography to
digital signal processing. Despite the popularity of this method, useful
theoretical estimates for its rate of convergence are still scarce. We
introduce a randomized version of the Kaczmarz method for consistent,
overdetermined linear systems and we prove that it converges with expected
exponential rate. Furthermore, this is the first solver whose rate does not
depend on the number of equations in the system. The solver does not even need
to know the whole system, but only a small random part of it. It thus
outperforms all previously known methods on general extremely overdetermined
systems. Even for moderately overdetermined systems, numerical simulations as
well as theoretical analysis reveal that our algorithm can converge faster than
the celebrated conjugate gradient algorithm. Furthermore, our theory and
numerical simulations confirm a prediction of Feichtinger et al. in the context
of reconstructing bandlimited functions from nonuniform sampling
Two-subspace Projection Method for Coherent Overdetermined Systems
We present a Projection onto Convex Sets (POCS) type algorithm for solving
systems of linear equations. POCS methods have found many applications ranging
from computer tomography to digital signal and image processing. The Kaczmarz
method is one of the most popular solvers for overdetermined systems of linear
equations due to its speed and simplicity. Here we introduce and analyze an
extension of the Kaczmarz method that iteratively projects the estimate onto a
solution space given by two randomly selected rows. We show that this
projection algorithm provides exponential convergence to the solution in
expectation. The convergence rate improves upon that of the standard randomized
Kaczmarz method when the system has correlated rows. Experimental results
confirm that in this case our method significantly outperforms the randomized
Kaczmarz method.Comment: arXiv admin note: substantial text overlap with arXiv:1204.027
Acceleration of Randomized Kaczmarz Method via the Johnson-Lindenstrauss Lemma
The Kaczmarz method is an algorithm for finding the solution to an
overdetermined consistent system of linear equations Ax=b by iteratively
projecting onto the solution spaces. The randomized version put forth by
Strohmer and Vershynin yields provably exponential convergence in expectation,
which for highly overdetermined systems even outperforms the conjugate gradient
method. In this article we present a modified version of the randomized
Kaczmarz method which at each iteration selects the optimal projection from a
randomly chosen set, which in most cases significantly improves the convergence
rate. We utilize a Johnson-Lindenstrauss dimension reduction technique to keep
the runtime on the same order as the original randomized version, adding only
extra preprocessing time. We present a series of empirical studies which
demonstrate the remarkable acceleration in convergence to the solution using
this modified approach
Randomized Extended Kaczmarz for Solving Least-Squares
We present a randomized iterative algorithm that exponentially converges in
expectation to the minimum Euclidean norm least squares solution of a given
linear system of equations. The expected number of arithmetic operations
required to obtain an estimate of given accuracy is proportional to the square
condition number of the system multiplied by the number of non-zeros entries of
the input matrix. The proposed algorithm is an extension of the randomized
Kaczmarz method that was analyzed by Strohmer and Vershynin.Comment: 19 Pages, 5 figures; code is available at
https://github.com/zouzias/RE
- …