Preconditioners for Krylov subspace methods: An overview

When simulating a mechanism from science or engineering, or an industrial process, one is frequently required to construct a mathematical model, and then resolve this model numerically. If accurate numerical solutions are necessary or desirable, this can involve solving large-scale systems of equations. One major class of solution methods is that of preconditioned iterative methods, involving preconditioners which are computationally cheap to apply while also capturing information contained in the linear system. In this article, we give a short survey of the field of preconditioning. We introduce a range of preconditioners for partial differential equations, followed by optimization problems, before discussing preconditioners constructed with less standard objectives in mind

Riemannian optimization of isometric tensor networks

Several tensor networks are built of isometric tensors, i.e. tensors satisfying $W^\dagger W = \mathrm{I}$. Prominent examples include matrix product states (MPS) in canonical form, the multiscale entanglement renormalization ansatz (MERA), and quantum circuits in general, such as those needed in state preparation and quantum variational eigensolvers. We show how gradient-based optimization methods on Riemannian manifolds can be used to optimize tensor networks of isometries to represent e.g. ground states of 1D quantum Hamiltonians. We discuss the geometry of Grassmann and Stiefel manifolds, the Riemannian manifolds of isometric tensors, and review how state-of-the-art optimization methods like nonlinear conjugate gradient and quasi-Newton algorithms can be implemented in this context. We apply these methods in the context of infinite MPS and MERA, and show benchmark results in which they outperform the best previously-known optimization methods, which are tailor-made for those specific variational classes. We also provide open-source implementations of our algorithms.Comment: 18 pages + appendices, 3 figures; v3 submission to SciPost; v4 expand preconditioning discussion and add polish, resubmit to SciPos

Additive block diagonal preconditioning for block two-by-two linear systems of skew-Hamiltonian coefficient matrices

For a class of block two-by-two systems of linear equations with certain skew-Hamiltonian coefficient matrices, we construct additive block diagonal preconditioning matrices and discuss the eigen-properties of the corresponding preconditioned matrices. The additive block diagonal preconditioners can be employed to accelerate the convergence rates of Krylov subspace iteration methods such as MINRES and GMRES. Numerical experiments show that MINRES preconditioned by the exact and the inexact additive block diagonal preconditioners are effective, robust and scalable solvers for the block two-by-two linear systems arising from the Galerkin finite-element discretizations of a class of distributed control problems

Additive block diagonal preconditioning for block two-by-two linear systems of skew-Hamiltonian coefficient matrices

For a class of block two-by-two systems of linear equations with certain skew-Hamiltonian coefficient matrices, we construct additive block diagonal preconditioning matrices and discuss the eigen-properties of the corresponding preconditioned matrices. The additive block diagonal preconditioners can be employed to accelerate the convergence rates of Krylov subspace iteration methods such as MINRES and GMRES. Numerical experiments show that MINRES preconditioned by the exact and the inexact additive block diagonal preconditioners are effective, robust and scalable solvers for the block two-by-two linear systems arising from the Galerkin finite-element discretizations of a class of distributed control problems

A numerical comparison of solvers for large-scale, continuous-time algebraic Riccati equations and LQR problems

In this paper, we discuss numerical methods for solving large-scale continuous-time algebraic Riccati equations. These methods have been the focus of intensive research in recent years, and significant progress has been made in both the theoretical understanding and efficient implementation of various competing algorithms. There are several goals of this manuscript: first, to gather in one place an overview of different approaches for solving large-scale Riccati equations, and to point to the recent advances in each of them. Second, to analyze and compare the main computational ingredients of these algorithms, to detect their strong points and their potential bottlenecks. And finally, to compare the effective implementations of all methods on a set of relevant benchmark examples, giving an indication of their relative performance

Multilevel Schwarz methods for multigroup radiation transport problems

The development of advanced discretization methods for the radiation transport equation is of fundamental importance, since the numerical effort of modeling increasingly complex multidimensional problems with increasing accuracy is extremely challenging. Different expressions of this equation arise in several science fields, from nuclear fission and fusion to astrophysics, climatology and combustion. Mathematically, the radiation intensity is usually a rapidly changing function, causing a considerable loss in accuracy for many discretization methods. Depending on the coefficient ranges, the equation behaves like totally different equation types, making it very difficult to find a discretization method that is efficient in all regimes. Computationally, the huge amount of unknowns involved demands not only extremely powerful computers, but also efficient numerical methods and optimized implementations. Today, solvers covering all the coefficient ranges and still being robust in the diffusion dominated case are very scarce. In the last 20 years, Discontinous Galerkin (DG) methods have been studied for the monoenergetic problem, unsuccessfully, due to lack of stability for diffusion-dominated cases. Recently, new mathematical developments have fully explained the instability and provided a remedy by using a numerical flux depending on the scattering cross section and the mesh size. The new formulation has proven to be stable and allows the application of multigrid, matrix-free methods, reducing the memory needed for such an amount of unknowns. We use these numerical methods to address the solution of a energy dependent problem with a multigroup approach. We study the diffusion approximation to the transport problem, obtaining convergence proofs for the symmetric scattering case and advances in the nonsymmetric case, using field of values analysis. For the full transport case, we discretize by means of an asymptotic preserving, weakly penalized discontinuous Galerkin method that we solve with a multigrid preconditioned GMRES solver, using nonoverlapping Schwarz smoothers for the energy and direction dependent radiative transfer problem. To address the local thermodynamic equilibrium (LTE) constraint, we use a nonlinear additive Schwarz method to precondition the Newton solver. By solving full local radiative transfer problems for each grid cell, performed in parallel on a matrix-free implementation, we achieve a method capable to address large scale calculations arising from applications such as astrophysics, atmospheric radiation calculations and nuclear applications. To the best of our knowledge, this is the first time this preconditioner combination has been used in LTE radiation transport and in several tests we show the robustness of the approach for different mesh sizes, cross sections, energy distributions and anisotropic regimes, both in the linear and nonlinear cases