495 research outputs found
Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method
The block Kaczmarz method is an iterative scheme for solving overdetermined
least-squares problems. At each step, the algorithm projects the current
iterate onto the solution space of a subset of the constraints. This paper
describes a block Kaczmarz algorithm that uses a randomized control scheme to
choose the subset at each step. This algorithm is the first block Kaczmarz
method with an (expected) linear rate of convergence that can be expressed in
terms of the geometric properties of the matrix and its submatrices. The
analysis reveals that the algorithm is most effective when it is given a good
row paving of the matrix, a partition of the rows into well-conditioned blocks.
The operator theory literature provides detailed information about the
existence and construction of good row pavings. Together, these results yield
an efficient block Kaczmarz scheme that applies to many overdetermined
least-squares problem
Preasymptotic Convergence of Randomized Kaczmarz Method
Kaczmarz method is one popular iterative method for solving inverse problems,
especially in computed tomography. Recently, it was established that a
randomized version of the method enjoys an exponential convergence for
well-posed problems, and the convergence rate is determined by a variant of the
condition number. In this work, we analyze the preasymptotic convergence
behavior of the randomized Kaczmarz method, and show that the low-frequency
error (with respect to the right singular vectors) decays faster during first
iterations than the high-frequency error. Under the assumption that the inverse
solution is smooth (e.g., sourcewise representation), the result explains the
fast empirical convergence behavior, thereby shedding new insights into the
excellent performance of the randomized Kaczmarz method in practice. Further,
we propose a simple strategy to stabilize the asymptotic convergence of the
iteration by means of variance reduction. We provide extensive numerical
experiments to confirm the analysis and to elucidate the behavior of the
algorithms.Comment: 20 page
An Asynchronous Parallel Randomized Kaczmarz Algorithm
We describe an asynchronous parallel variant of the randomized Kaczmarz (RK)
algorithm for solving the linear system . The analysis shows linear
convergence and indicates that nearly linear speedup can be expected if the
number of processors is bounded by a multiple of the number of rows in
- …