24,276 research outputs found
Deterministic Versus Randomized Kaczmarz Iterative Projection
Kaczmarz's alternating projection method has been widely used for solving a
consistent (mostly over-determined) linear system of equations Ax=b. Because of
its simple iterative nature with light computation, this method was
successfully applied in computerized tomography. Since tomography generates a
matrix A with highly coherent rows, randomized Kaczmarz algorithm is expected
to provide faster convergence as it picks a row for each iteration at random,
based on a certain probability distribution. It was recently shown that picking
a row at random, proportional with its norm, makes the iteration converge
exponentially in expectation with a decay constant that depends on the scaled
condition number of A and not the number of equations. Since Kaczmarz's method
is a subspace projection method, the convergence rate for simple Kaczmarz
algorithm was developed in terms of subspace angles. This paper provides
analyses of simple and randomized Kaczmarz algorithms and explain the link
between them. It also propose new versions of randomization that may speed up
convergence
Fast linear algebra is stable
In an earlier paper, we showed that a large class of fast recursive matrix
multiplication algorithms is stable in a normwise sense, and that in fact if
multiplication of -by- matrices can be done by any algorithm in
operations for any , then it can be done
stably in operations for any . Here we extend
this result to show that essentially all standard linear algebra operations,
including LU decomposition, QR decomposition, linear equation solving, matrix
inversion, solving least squares problems, (generalized) eigenvalue problems
and the singular value decomposition can also be done stably (in a normwise
sense) in operations.Comment: 26 pages; final version; to appear in Numerische Mathemati
- …