Scalable iterative methods for sampling from massive Gaussian random vectors

Sampling from Gaussian Markov random fields (GMRFs), that is multivariate Gaussian ran- dom vectors that are parameterised by the inverse of their covariance matrix, is a fundamental problem in computational statistics. In this paper, we show how we can exploit arbitrarily accu- rate approximations to a GMRF to speed up Krylov subspace sampling methods. We also show that these methods can be used when computing the normalising constant of a large multivariate Gaussian distribution, which is needed for both any likelihood-based inference method. The method we derive is also applicable to other structured Gaussian random vectors and, in particu- lar, we show that when the precision matrix is a perturbation of a (block) circulant matrix, it is still possible to derive O(n log n) sampling schemes.Comment: 17 Pages, 4 Figure

Approximate and Incomplete Factorizations

In this chapter, we give a brief overview of a particular class of preconditioners known as incomplete factorizations. They can be thought of as approximating the exact LU factorization of a given matrix A (e.g. computed via Gaussian elimination) by disallowing certain ll-ins. As opposed to other PDE-based preconditioners such as multigrid and domain decomposition, this class of preconditioners are primarily algebraic in nature and can in principle be applied to any sparse matrices. When applied to PDE problems, they are usually not optimal in the sense that the condition number of the preconditioned system will grow as the mesh size h is reduced, although usually at a slower rate than for the unpreconditioned system. On the other hand, they are often quite robust with respect to other more algebraic features of the problem such as rough and anisotropic coecients and strong convection terms. We will describe the basic ILU and (modied) MILU preconditioners. Then we will review brie y several variants: more lls, relaxed ILU, shifted ILU, ILQ, as well as block and multilevel variants. We will also touch on a related class of approximate factorization methods which arise more directly from approximating a partial dierential operator by a product of simpler operators. Finally, we will discuss parallelization aspects, including re-ordering, series expansion and domain decomposition techniques. Generally, this class of preconditioner does not possess a high degree of parallelism in its original form. Re-ordering and approximations by truncating certain series expansion will increase the parallelism, but usually with a deterioration in convergence rate. Domain decomposition oers a compromise

KSPHPDDM and PCHPDDM: Extending PETSc with advanced Krylov methods and robust multilevel overlapping Schwarz preconditioners

[EN] Contemporary applications in computational science and engineering often require the solution of linear systems which may be of different sizes, shapes, and structures. The goal of this paper is to explain how two libraries, PETSc and HPDDM, have been interfaced in order to offer end-users robust overlapping Schwarz preconditioners and advanced Krylov methods featuring recycling and the ability to deal with multiple right-hand sides. The flexibility of the implementation is showcased and explained with minimalist, easy-to-run, and reproducible examples, to ease the integration of these algorithms into more advanced frameworks. The examples provided cover applications from eigenanalysis, elasticity, combustion, and electromagnetism.Jose E. Roman was supported by the Spanish Agencia Estatal de Investigacion (AEI) under project SLEPc-DA (PID2019-107379RB-I00)Jolivet, P.; Roman, JE.; Zampini, S. (2021). KSPHPDDM and PCHPDDM: Extending PETSc with advanced Krylov methods and robust multilevel overlapping Schwarz preconditioners. Computers & Mathematics with Applications. 84:277-295. https://doi.org/10.1016/j.camwa.2021.01.0032772958

Neural incomplete factorization: learning preconditioners for the conjugate gradient method

Finding suitable preconditioners to accelerate iterative solution methods, such as the conjugate gradient method, is an active area of research. In this paper, we develop a computationally efficient data-driven approach to replace the typically hand-engineered algorithms with neural networks. Optimizing the condition number of the linear system directly is computationally infeasible. Instead, our method generates an incomplete factorization of the matrix and is, therefore, referred to as neural incomplete factorization (NeuralIF). For efficient training, we utilize a stochastic approximation of the Frobenius loss which only requires matrix-vector multiplications. At the core of our method is a novel messagepassing block, inspired by sparse matrix theory, that aligns with the objective of finding a sparse factorization of the matrix. By replacing conventional preconditioners used within the conjugate gradient method by data-driven models based on graph neural networks, we accelerate the iterative solving procedure. We evaluate our proposed method on both a synthetic and a real-world problem arising from scientific computing and show its ability to reduce the solving time while remaining computationally efficient.Comment: Under review. 18 pages, 8 figure

Schwarz type preconditioners for the neutron diffusion equation

[EN] Domain decomposition is a mature methodology that has been used to accelerate the convergence of partial differential equations. Even if it was devised as a solver by itself, it is usually employed together with Krylov iterative methods improving its rate of convergence, and providing scalability with respect to the size of the problem. In this work, a high order finite element discretization of the neutron diffusion equation is considered. In this problem the preconditioning of large and sparse linear systems arising from a source driven formulation becomes necessary due to the complexity of the problem. On the other hand, preconditioners based on an incomplete factorization are very expensive from the point of view of memory requirements. The acceleration of the neutron diffusion equation is thus studied here by using alternative preconditioners based on domain decomposition techniques inside Schur complement methodology. The study considers substructuring preconditioners, which do not involve overlapping, and additive Schwarz preconditioners, where some overlapping between the subdomains is taken into account. The performance of the different approaches is studied numerically using two-dimensional and three-dimensional problems. It is shown that some of the proposed methodologies outperform incomplete LU factorization for preconditioning as long as the linear system to be solved is large enough, as it occurs for three-dimensional problems. They also outperform classical diagonal Jacobi preconditioners, as long as the number of systems to be solved is large enough in such a way that the overhead of building the pre-conditioner is less than the improvement in the convergence rate. (C) 2016 Elsevier B.V. All rights reserved.The work has been partially supported by the spanish Ministerio de Economía y Competitividad under projects ENE 2014-59442-P and MTM2014-58159-P, the Generalitat Valenciana under the project PROMETEO II/2014/008 and the Universitat Politècnica de València under the project FPI-2013. The work has also been supported partially by the Swedish Research Council (VR-Vetenskapsrådet) within a framework grant called DREAM4SAFER, research contract C0467701.Vidal-Ferràndiz, A.; González Pintor, S.; Ginestar Peiro, D.; Verdú Martín, GJ.; Demazière, C. (2017). Schwarz type preconditioners for the neutron diffusion equation. Journal of Computational and Applied Mathematics. 309:563-574. https://doi.org/10.1016/j.cam.2016.02.056S56357430

On Efficiency of the OpenFOAM-based Parallel Solver for the Heat Transfer in Electrical Power Cables

Proceedings of: First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014). Porto (Portugal), August 27-28, 2014.In this work, we study the efficiency of the OpenFOAM-based parallel solver for the heat conduction in electrical power cables. The 2D benchmark problem with three cables is used for our numerical tests. We study and compare the efficiency of conjugate gradient solver with diagonal incomplete Cholesky (DIC) preconditioner and generalized geometric algebraic multigrid solver (GAMG), which is available in Open- FOAM. The convergence and parallel scalability of the solvers are presented and analyzed. Parallel numerical tests are performed on the cluster of multicore computers.The work of authors was supported by Eureka project E!6799 POWEROPT "Mathematical modelling and optimization of electrical power cables for an improvement of their design rules". The work presented in this paper has been partially supported by EU under the COST programme Action IC1305, ’Network for Sustainable Ultrascale Computing (NESUS)’

Optimal-complexity and robust multigrid methods for high-order FEM

The numerical solution of elliptic PDEs is often the most computationally intensive task in large-scale continuum mechanics simulations. High-order finite element methods can efficiently exploit modern parallel hardware while offering very rapid convergence properties. As the polynomial degree is increased, the efficient solution of such PDEs becomes difficult. This thesis develops preconditioners for high-order discretizations. We build upon the pioneering work of Pavarino, who proved in 1993 that the additive Schwarz method with vertex patches and a low-order coarse space gives a solver for symmetric and coercive problems that is robust to the polynomial degree. However, for very high polynomial degrees it is not feasible to assemble or factorize the matrices for each vertex patch, as the patch matrices contain dense blocks, which couple together all degrees of freedom within a cell. The central novelty of the preconditioners we develop is that they have optimal time and space complexity on unstructured meshes of tensor-product cells. Our solver relies on new finite elements for the de Rham complex that enable the blocks in the stiffness matrix corresponding to the cell interiors to become diagonal for scalar PDEs or block diagonal for vector-valued PDEs. With these new elements, the patch problems are as sparse as a low-order finite difference discretization, while having a sparser Cholesky factorization. In the non-separable case, the method can be applied as a preconditioner by approximating the problem with a separable surrogate. Through the careful use of incomplete factorizations and choice of space decomposition we achieve optimal fill-in in the patch factors, ultimately allowing for optimal-complexity storage and computational cost across the setup and solution stages. We demonstrate the approach by solving a variety of symmetric and coercive problems, including the Poisson equation, the Riesz maps of H(curl) and H(div), and a H(div)-conforming interior penalty discretization of linear elasticity in three dimensions at p = 15