Krylov Methods for Large Sparse Systems: A Comprehensive Overview

In this paper are analyzed behavior and properties for different Krylov methods applied in different categories of problems. These categories often include PDEs, econometrics and network models, which are represented by large sparse systems. For our empirical analysis are taken into consideration size, the density of non-zero elements, symmetry/un-symmetry, eigenvalue distribution, also well/ill-conditioned and random systems. Convergence, approximation error and residuals are compared for the full version of methods, some restarted methods and preconditioned methods. Two preconditioners are considered respectively, ILU(0) and IC(0) by using at least five preconditioning techniques. In each case, empirical results show which technique is best to use based on properties of the system and are backed up by general theoretical information already found on Krylov space methods

Some transpose-free CG-like solvers for nonsymmetric ill-posed problems

2siThis paper introduces and analyzes an original class of Krylov subspace methods that provide an efficient alternative to many well-known conjugate-gradient-like (CG-like) Krylov solvers for square nonsymmetric linear systems arising from discretizations of inverse ill-posed problems. The main idea underlying the new methods is to consider some rank-deficient approximations of the transpose of the system matrix, obtained by running the (transpose-free) Arnoldi algorithm, and then apply some Krylov solvers to a formally right-preconditioned system of equations. Theoretical insight is given, and many numerical tests show that the new solvers outperform classical Arnoldi-based or CG-like methods in a variety of situations.openembargoed_20210328Gazzola S.; Novati P.Gazzola, S.; Novati, P

Reduced order modeling of convection-dominated flows, dimensionality reduction and stabilization

We present methodologies for reduced order modeling of convection dominated flows. Accordingly, three main problems are addressed. Firstly, an optimal manifold is realized to enhance reducibility of convection dominated flows. We design a low-rank auto-encoder to specifically reduce the dimensionality of solution arising from convection-dominated nonlinear physical systems. Although existing nonlinear manifold learning methods seem to be compelling tools to reduce the dimensionality of data characterized by large Kolmogorov n-width, they typically lack a straightforward mapping from the latent space to the high-dimensional physical space. Also, considering that the latent variables are often hard to interpret, many of these methods are dismissed in the reduced order modeling of dynamical systems governed by partial differential equations (PDEs). This deficiency is of importance to the extent that linear methods, such as principle component analysis (PCA) and Koopman operators, are still prevalent. Accordingly, we propose an interpretable nonlinear dimensionality reduction algorithm. An unsupervised learning problem is constructed that learns a diffeomorphic spatio-temporal grid which registers the output sequence of the PDEs on a non-uniform time-varying grid. The Kolmogorov n-width of the mapped data on the learned grid is minimized. Secondly, the reduced order models are constructed on the realized manifolds. We project the high fidelity models on the learned manifold, leading to a time-varying system of equations. Moreover, as a data-driven model free architecture, recurrent neural networks on the learned manifold are trained, showing versatility of the proposed framework. Finally, a stabilization method is developed to maintain stability and accuracy of the projection based ROMs on the learned manifold a posteriori. We extend the eigenvalue reassignment method of stabilization of linear time-invariant ROMs, to the more general case of linear time-varying systems. Through a post-processing step, the ROMs are controlled using a constrained nonlinear lease-square minimization problem. The controller and the input signals are defined at the algebraic level, using left and right singular vectors of the reduced system matrices. The proposed stabilization method is general and applicable to a large variety of linear time-varying ROMs

Krylov subspace methods and their generalizations for solving singular linear operator equations with applications to continuous time Markov chains

Viele Resultate über MR- und OR-Verfahren zur Lösung linearer Gleichungssysteme bleiben (in leicht modifizierter Form) gültig, wenn der betrachtete Operator nicht invertierbar ist. Neben dem für reguläre Probleme charakteristischen Abbruchverhalten, kann bei einem singulären Gleichungssystem auch ein so genannter singulärer Zusammenbruch auftreten. Für beide Fälle werden verschiedene Charakterisierungen angegeben. Die Unterrauminverse, eine spezielle verallgemeinerte Inverse, beschreibt die Näherungen eines MR-Unterraumkorrektur-Verfahrens. Für Krylov-Unterräume spielt die Drazin-Inverse eine Schlüsselrolle. Bei Krylov-Unterraum-Verfahren kann a-priori entschieden werden, ob ein regulärer oder ein singulärer Abbruch auftritt. Wir können zeigen, dass ein Krylov-Verfahren genau dann für beliebige Startwerte eine Lösung des linearen Gleichungssystems liefert, wenn der Index der Matrix nicht größer als eins und das Gleichungssystem konsistent ist. Die Berechnung stationärer Zustandsverteilungen zeitstetiger Markov-Ketten mit endlichem Zustandsraum stellt eine praktische Aufgabe dar, welche die Lösung eines singulären linearen Gleichungssystems erfordert. Die Eigenschaften der Übergangs-Halbgruppe folgen aus einfachen Annahmen auf rein analytischem und matrixalgebrischen Wege. Insbesondere ist die erzeugende Matrix eine singuläre M-Matrix mit Index 1. Ist die Markov-Kette irreduzibel, so ist die stationäre Zustandsverteilung eindeutig bestimmt

Local preconditioning for parallel iterative solvers

This thesis aims at improving the convergence of iterative solvers, used for algebraic systems coming from the discretization of partial differential equations (PDE), in the context of large scale simulations and high performance computing (HPC). The methodology followed consists in adapting some existing preconditiong techniques to the physics and numerics of convection-dominated transport and boundary layer problems in flows. For convection-dominated flows, a physics-based permutation algorithm is presented, which consists in renumbering the mesh in the direction of convection. This renumbering is then used together with a Gauss-Seidel preconditioner to propagate the result of the matrix-vector products along the convection. The robutsness and effectiveness of this preconditioner is proved in several test cases solving the heat equation as well as the Navier-Stokes equations in both sequential and in parallel using the Message Passing Interface library MPI. Additionally, the composition of preconditioners is proposed to solve cases where different local physical behaviors co-exist in the same flow. In particular, we focus on such problems where of a highly convective flow encounters an obstacle. Such problems involve a zone with high convection far from the obstacle and the development of a boundary layer in the vicinity of the obstacle. In numerical terms, these local behaviors translate into specific matrix structures that we will take advantage of to adapt the preconditioner locally. On the one hand, the linelet preconditioner is a well-known efficient preconditioner for boundary layers where the mesh is highly anisotropic, in particular to solve the Poisson equation. On the other hand, the streamline linelet that we propose in this thesis (Gauss-Seidel together with a mesh renumbering in the convection direction) is well adapted for locally hyperbolic flows. Both preconditioners will be composed (combined) in different ways to investigate their robustness in terms of convergence as well as their costs to solve the proposed transport problems. We will study as well their performances in terms of parallelization.Esta tesis tiene como objetivo mejorar la convergencia de los métodos iterativos utilizados para resolver sistemas de ecuaciones algebraicas provenientes de la discretización de ecuaciones diferenciales en derivadas parciales (EDP), en el contexto de las simulaciones a gran escala y computación de altas prestaciones (HPC). La metodología seguida consiste en adaptar algunas técnicas de precondicionamiento existentes, a la física y la numérica en flujos que presentan una alta convección y flujos que presentan una capa límite. Para los flujos dominados por convección, se presenta un algoritmo de permutación basado en la física, que consiste en la renumeración de la malla en la dirección de la convección. Esta renumeración se usa luego junto con el precondicionador Gauss-Seidel para propagar el resultado de los productos matriz-vector a lo largo de la convección. La robustez y eficiencia de este precondicionador se demuestra en varios ejemplos en los que se resuelve la ecuación de calor y las ecuaciones de Navier-Stokes tanto en secuencial como en paralelo utilizando la librería interfaz de paso de mensajes (MPI). Además, se propone la composición de precondicionadores para resolver casos donde diferentes comportamientos físicos locales coexisten en el mismo flujo. En particular, nos enfocamos en los casos donde un flujo altamente convectivo se encuentra un obstáculo. En este tipo de problemas nos encontramos dos zonas: una con alta convección lejos del obstáculo y otra donde se desarrolla una capa límite en los alrededores del obstáculo. En términos numéricos, estos comportamientos locales se traducen en estructuras matriciales específicas que aprovecharemos para adaptar localmente el precondicionador. Por un lado, sabemos que el linelet es un precondicionador eficiente para resolver problemas de capa límite donde la malla es altamente anisótropa. En particular resulta eficiente para resolver la ecuación de Poisson. Por otro lado, sabemos que el linelet aerodinámico, el precondicionador que proponemos en esta tesis (precondicionador Gauss-Seidel junto con una renumeración de malla en la dirección de la convección) está bien adaptado para flujos localmente hiperbólicos. Con todo esto, proponemos también una composición de los dos precondicionadores (combinación de ambos) de distintas formas para investigar su robustez en términos de convergencia, así como sus costes para resolver los problemas de transporte propuestos. Estudiaremos también el rendimiento en cuanto a la paralelización se refiere.Postprint (published version