Tuned preconditioners for the eigensolution of large SPD matrices arising in engineering problems

In this paper, we study a class of tuned preconditioners that will be designed to accelerate both the DACG-Newton method and the implicitly restarted Lanczos method for the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices arising in large-scale scientific computations. These tuning strategies are based on low-rank modifications of a given initial preconditioner. We present some theoretical properties of the preconditioned matrix. We experimentally show how the aforementioned methods benefit from the acceleration provided by these tuned/deflated preconditioners. Comparisons are carried out with the Jacobi-Davidson method onto matrices arising from various large realistic problems arising from finite element discretization of PDEs modeling either groundwater flow in porous media or geomechanical processes in reservoirs. The numerical results show that the Newton-based methods (which includes also the Jacobi-Davidson method) are to be preferred to the - yet efficiently implemented - implicitly restarted Lanczos method whenever a small to moderate number of eigenpairs is required. \ua9 2016 John Wiley & Sons, Ltd

Preconditioning issues in the numerical solution of nonlinear equations and nonlinear least squares

Second order methods for optimization call for the solution of sequences of linear systems. In this survey we will discuss several issues related to the preconditioning of such sequences. Covered topics include both techniques for building updates of factorized preconditioners and quasi-Newton approaches. Sequences of unsymmetric linear systems arising in Newton-Krylov methods will be considered as well as symmetric positive definite sequences arising in the solution of nonlinear least-squares by Truncated Gauss-Newton methods

Parallel RFSAI-BFGS Preconditioners for Large Symmetric Eigenproblems

We propose a parallel preconditioner for the Newton method in the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices. A sequence of preconditioners starting from an enhanced approximate inverse RFSAI (Bergamaschi and Martínez, 2012) and enriched by a BFGS-like update formula is proposed to accelerate the preconditioned conjugate gradient solution of the linearized Newton system to solve Au=q(u)u, q(u) being the Rayleigh quotient. In a previous work (Bergamaschi and Martínez, 2013) the sequence of preconditioned Jacobians is proven to remain close to the identity matrix if the initial preconditioned Jacobian is so. Numerical results onto matrices arising from various realistic problems with size up to 1.5 million unknowns account for the efficiency and the scalability of the proposed low rank update of the RFSAI preconditioner. The overall RFSAI-BFGS preconditioned Newton algorithm has shown comparable efficiencies with a well-established eigenvalue solver on all the test problems

Compact quasi-Newton preconditioners for symmetric positive definite linear systems

[EN] In this paper, preconditioners for the conjugate gradient method are studied to solve the Newton system with symmetric positive definite Jacobian. In particular, we define a sequence of preconditioners built by means of Symmetric Rank one (SR1) and Broyden-Fletcher-Goldfarb-Shanno (BFGS) low-rank updates. We develop conditions under which the SR1 update maintains the preconditioner symmetric positive definite. Spectral analysis of the SR1 preconditioned Jacobians shows an improved eigenvalue distribution as the Newton iteration proceeds. A compact matrix formulation of the preconditioner update is developed which reduces the cost of its application and is more suitable to parallel implementation. Some notes on the implementation of the corresponding Inexact Newton method are given and some numerical results on a number of model problems illustrate the efficiency of the proposed preconditioners.This work was supported by the Spanish Ministerio de Economia y Competitividad under grants MTM2014-58159-P, MTM2017-85669-P, and MTM2017-90682-REDT. The first and third authors have been also partially supported by the INdAM Research group GNCS, 2020 Project: Optimization and advanced linear algebra for problems arising from PDEs.Bergamaschi, L.; Marín Mateos-Aparicio, J.; Martinez, A. (2020). Compact quasi-Newton preconditioners for symmetric positive definite linear systems. Numerical Linear Algebra with Applications. 27(6):1-17. https://doi.org/10.1002/nla.2322S117276Bergamaschi, L., & Putti, M. (1999). Mixed finite elements and Newton-type linearizations for the solution of Richards’ equation. International Journal for Numerical Methods in Engineering, 45(8), 1025-1046. doi:10.1002/(sici)1097-0207(19990720)45:83.0.co;2-gNotay, Y. (2001). Combination of Jacobi-Davidson and conjugate gradients for the partial symmetric eigenproblem. Numerical Linear Algebra with Applications, 9(1), 21-44. doi:10.1002/nla.246Bergamaschi, L., & Martínez, A. (2014). Efficiently preconditioned inexact Newton methods for large symmetric eigenvalue problems. Optimization Methods and Software, 30(2), 301-322. doi:10.1080/10556788.2014.908878Bergamaschi, L., & Martínez, A. (2013). Parallel RFSAI-BFGS Preconditioners for Large Symmetric Eigenproblems. Journal of Applied Mathematics, 2013, 1-10. doi:10.1155/2013/767042Martinez, J. M. (1993). A Theory of Secant Preconditioners. Mathematics of Computation, 60(202), 681. doi:10.2307/2153109Morales, J. L., & Nocedal, J. (2000). Automatic Preconditioning by Limited Memory Quasi-Newton Updating. SIAM Journal on Optimization, 10(4), 1079-1096. doi:10.1137/s1052623497327854Nabben, R., & Vuik, C. (2006). A Comparison of Deflation and the Balancing Preconditioner. SIAM Journal on Scientific Computing, 27(5), 1742-1759. doi:10.1137/040608246Freitag, M. A., & Spence, A. (2007). A tuned preconditioner for inexact inverse iteration applied to Hermitian eigenvalue problems. IMA Journal of Numerical Analysis, 28(3), 522-551. doi:10.1093/imanum/drm036Martínez, Á. (2016). Tuned preconditioners for the eigensolution of large SPD matrices arising in engineering problems. Numerical Linear Algebra with Applications, 23(3), 427-443. doi:10.1002/nla.2032Gratton, S., Sartenaer, A., & Tshimanga, J. (2011). On A Class of Limited Memory Preconditioners For Large Scale Linear Systems With Multiple Right-Hand Sides. SIAM Journal on Optimization, 21(3), 912-935. doi:10.1137/08074008Bergamaschi, L., Bru, R., & Martínez, A. (2011). Low-rank update of preconditioners for the inexact Newton method with SPD Jacobian. Mathematical and Computer Modelling, 54(7-8), 1863-1873. doi:10.1016/j.mcm.2010.11.064DeGuchy, O., Erway, J. B., & Marcia, R. F. (2018). Compact representation of the full Broyden class of quasi‐Newton updates. Numerical Linear Algebra with Applications, 25(5). doi:10.1002/nla.2186Nocedal, J., & Wright, S. J. (Eds.). (1999). Numerical Optimization. Springer Series in Operations Research and Financial Engineering. doi:10.1007/b98874Dembo, R. S., Eisenstat, S. C., & Steihaug, T. (1982). Inexact Newton Methods. SIAM Journal on Numerical Analysis, 19(2), 400-408. doi:10.1137/0719025Kelley, C. T. (1999). Iterative Methods for Optimization. doi:10.1137/1.9781611970920Byrd, R. H., Nocedal, J., & Schnabel, R. B. (1994). Representations of quasi-Newton matrices and their use in limited memory methods. Mathematical Programming, 63(1-3), 129-156. doi:10.1007/bf01582063Bergamaschi, L. (2020). A Survey of Low-Rank Updates of Preconditioners for Sequences of Symmetric Linear Systems. Algorithms, 13(4), 100. doi:10.3390/a13040100Powers, R. T., & Størmer, E. (1970). Free states of the canonical anticommutation relations. Communications in Mathematical Physics, 16(1), 1-33. doi:10.1007/bf01645492Ipsen, I. C. F., & Nadler, B. (2009). Refined Perturbation Bounds for Eigenvalues of Hermitian and Non-Hermitian Matrices. SIAM Journal on Matrix Analysis and Applications, 31(1), 40-53. doi:10.1137/070682745Simoncini, V., & Eldén, L. (2002). Bit Numerical Mathematics, 42(1), 159-182. doi:10.1023/a:1021930421106G. Sleijpen, G. L., & Van der Vorst, H. A. (1996). A Jacobi–Davidson Iteration Method for Linear Eigenvalue Problems. SIAM Journal on Matrix Analysis and Applications, 17(2), 401-425. doi:10.1137/s0895479894270427Tapia, R. A., Dennis, J. E., & Schäfermeyer, J. P. (2018). Inverse, Shifted Inverse, and Rayleigh Quotient Iteration as Newton’s Method. SIAM Review, 60(1), 3-55. doi:10.1137/15m104995