57 research outputs found
Acceleration of Randomized Kaczmarz Method via the Johnson-Lindenstrauss Lemma
The Kaczmarz method is an algorithm for finding the solution to an
overdetermined consistent system of linear equations Ax=b by iteratively
projecting onto the solution spaces. The randomized version put forth by
Strohmer and Vershynin yields provably exponential convergence in expectation,
which for highly overdetermined systems even outperforms the conjugate gradient
method. In this article we present a modified version of the randomized
Kaczmarz method which at each iteration selects the optimal projection from a
randomly chosen set, which in most cases significantly improves the convergence
rate. We utilize a Johnson-Lindenstrauss dimension reduction technique to keep
the runtime on the same order as the original randomized version, adding only
extra preprocessing time. We present a series of empirical studies which
demonstrate the remarkable acceleration in convergence to the solution using
this modified approach
Accelerating Random Kaczmarz Algorithm Based on Clustering Information
Kaczmarz algorithm is an efficient iterative algorithm to solve
overdetermined consistent system of linear equations. During each updating
step, Kaczmarz chooses a hyperplane based on an individual equation and
projects the current estimate for the exact solution onto that space to get a
new estimate. Many vairants of Kaczmarz algorithms are proposed on how to
choose better hyperplanes. Using the property of randomly sampled data in
high-dimensional space, we propose an accelerated algorithm based on clustering
information to improve block Kaczmarz and Kaczmarz via Johnson-Lindenstrauss
lemma. Additionally, we theoretically demonstrate convergence improvement on
block Kaczmarz algorithm
Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method
The block Kaczmarz method is an iterative scheme for solving overdetermined
least-squares problems. At each step, the algorithm projects the current
iterate onto the solution space of a subset of the constraints. This paper
describes a block Kaczmarz algorithm that uses a randomized control scheme to
choose the subset at each step. This algorithm is the first block Kaczmarz
method with an (expected) linear rate of convergence that can be expressed in
terms of the geometric properties of the matrix and its submatrices. The
analysis reveals that the algorithm is most effective when it is given a good
row paving of the matrix, a partition of the rows into well-conditioned blocks.
The operator theory literature provides detailed information about the
existence and construction of good row pavings. Together, these results yield
an efficient block Kaczmarz scheme that applies to many overdetermined
least-squares problem
On the exponential convergence of the Kaczmarz algorithm
The Kaczmarz algorithm (KA) is a popular method for solving a system of
linear equations. In this note we derive a new exponential convergence result
for the KA. The key allowing us to establish the new result is to rewrite the
KA in such a way that its solution path can be interpreted as the output from a
particular dynamical system. The asymptotic stability results of the
corresponding dynamical system can then be leveraged to prove exponential
convergence of the KA. The new bound is also compared to existing bounds
- …