Dynamical Systems Method for solving ill-conditioned linear algebraic systems

A new method, the Dynamical Systems Method (DSM), justified recently, is applied to solving ill-conditioned linear algebraic system (ICLAS). The DSM gives a new approach to solving a wide class of ill-posed problems. In this paper a new iterative scheme for solving ICLAS is proposed. This iterative scheme is based on the DSM solution. An a posteriori stopping rules for the proposed method is justified. This paper also gives an a posteriori stopping rule for a modified iterative scheme developed in A.G.Ramm, JMAA,330 (2007),1338-1346, and proves convergence of the solution obtained by the iterative scheme.Comment: 26 page

Probabilistic Linear Solvers: A Unifying View

Several recent works have developed a new, probabilistic interpretation for numerical algorithms solving linear systems in which the solution is inferred in a Bayesian framework, either directly or by inferring the unknown action of the matrix inverse. These approaches have typically focused on replicating the behavior of the conjugate gradient method as a prototypical iterative method. In this work surprisingly general conditions for equivalence of these disparate methods are presented. We also describe connections between probabilistic linear solvers and projection methods for linear systems, providing a probabilistic interpretation of a far more general class of iterative methods. In particular, this provides such an interpretation of the generalised minimum residual method. A probabilistic view of preconditioning is also introduced. These developments unify the literature on probabilistic linear solvers, and provide foundational connections to the literature on iterative solvers for linear systems

On a new iterative method for solving linear systems and comparison results

AbstractIn Ujević [A new iterative method for solving linear systems, Appl. Math. Comput. 179 (2006) 725–730], the author obtained a new iterative method for solving linear systems, which can be considered as a modification of the Gauss–Seidel method. In this paper, we show that this is a special case from a point of view of projection techniques. And a different approach is established, which is both theoretically and numerically proven to be better than (at least the same as) Ujević's. As the presented numerical examples show, in most cases, the convergence rate is more than one and a half that of Ujević

LSMR: An iterative algorithm for sparse least-squares problems

An iterative method LSMR is presented for solving linear systems $Ax=b$ and least-squares problem \min \norm{Ax-b}_2, with $A$ being sparse or a fast linear operator. LSMR is based on the Golub-Kahan bidiagonalization process. It is analytically equivalent to the MINRES method applied to the normal equation A\T Ax = A\T b, so that the quantities \norm{A\T r_k} are monotonically decreasing (where $r_k = b - Ax_k$ is the residual for the current iterate $x_k$). In practice we observe that \norm{r_k} also decreases monotonically. Compared to LSQR, for which only \norm{r_k} is monotonic, it is safer to terminate LSMR early. Improvements for the new iterative method in the presence of extra available memory are also explored.Comment: 21 page

Preconditioned WR–LMF-based method for ODE systems

AbstractThe waveform relaxation (WR) method was developed as an iterative method for solving large systems of ordinary differential equations (ODEs). In each WR iteration, we are required to solve a system of ODEs. We then introduce the boundary value method (BVM) which is a relatively new method based on the linear multistep formulae to solve ODEs. In particular, we apply the generalized minimal residual method with the Strang-type block-circulant preconditioner for solving linear systems arising from the application of BVMs to each WR iteration. It is demonstrated that these techniques are very effective in speeding up the convergence rate of the resulting iterative processes. Numerical experiments are presented to illustrate the effectiveness of our methods