Some Preconditioning Techniques for Saddle Point Problems

Saddle point problems arise frequently in many applications in science and engineering, including constrained optimization, mixed finite element formulations of partial differential equations, circuit analysis, and so forth. Indeed the formulation of most problems with constraints gives rise to saddle point systems. This paper provides a concise overview of iterative approaches for the solution of such systems which are of particular importance in the context of large scale computation. In particular we describe some of the most useful preconditioning techniques for Krylov subspace solvers applied to saddle point problems, including block and constrained preconditioners.\ud \ud The work of Michele Benzi was supported in part by the National Science Foundation grant DMS-0511336

A Hamiltonian Krylov-Schur-type method based on the symplectic Lanczos process

We discuss a Krylov-Schur like restarting technique applied within the symplectic Lanczos algorithm for the Hamiltonian eigenvalue problem. This allows to easily implement a purging and locking strategy in order to improve the convergence properties of the symplectic Lanczos algorithm. The Krylov-Schur-like restarting is based on the SR algorithm. Some ingredients of the latter need to be adapted to the structure of the symplectic Lanczos recursion. We demonstrate the efficiency of the new method for several Hamiltonian eigenproblems

A New Approximation of the Schur Complement in Preconditioners for PDE Constrained Optimization

Saddle point systems arise widely in optimization problems with constraints. The utility of Schur complement approximation is now broadly appreciated in the context of solving such saddle point systems by iteration. In this short manuscript, we present a new Schur complement approximation for PDE constrained optimization, an important class of these problems. Block diagonal and block triangular preconditioners have previously been designed to be used to solve such problems along with MINRES and non-standard Conjugate Gradients respectively; with appropriate approximation blocks these can be optimal in the sense that the time required for solution scales linearly with the problem size, however small the mesh size we use. In this paper, we extend this work to designing such preconditioners for which this optimality property holds independently of both the mesh size and of the Tikhonov regularization parameter \beta that is used. This also leads to an effective symmetric indefinite preconditioner that exhibits mesh and \beta-independence. We motivate the choice of these preconditioners based on observations about approximating the Schur complement obtained from the matrix system, derive eigenvalue bounds which verify the effectiveness of the approximation, and present numerical results which show that these new preconditioners work well in practice

Fast iterative solution of reaction-diffusion control problems arising from chemical processes

PDE-constrained optimization problems, and the development of preconditioned iterative methods for the efficient solution of the arising matrix system, is a field of numerical analysis that has recently been attracting much attention. In this paper, we analyze and develop preconditioners for matrix systems that arise from the optimal control of reaction-diffusion equations, which themselves result from chemical processes. Important aspects in our solvers are saddle point theory, mass matrix representation and effective Schur complement approximation, as well as the outer (Newton) iteration to take account of the nonlinearity of the underlying PDEs

Preconditioners for state constrained optimal control problems with Moreau-Yosida penalty function

Optimal control problems with partial differential equations as constraints play an important role in many applications. The inclusion of bound constraints for the state variable poses a significant challenge for optimization methods. Our focus here is on the incorporation of the constraints via the Moreau-Yosida regularization technique. This method has been studied recently and has proven to be advantageous compared to other approaches. In this paper we develop robust preconditioners for the efficient solution of the Newton steps associated with solving the Moreau-Yosida regularized problem. Numerical results illustrate the efficiency of our approach

Fixing Nonconvergence of Algebraic Iterative Reconstruction with an Unmatched Backprojector

We consider algebraic iterative reconstruction methods with applications in image reconstruction. In particular, we are concerned with methods based on an unmatched projector/backprojector pair; i.e., the backprojector is not the exact adjoint or transpose of the forward projector. Such situations are common in large-scale computed tomography, and we consider the common situation where the method does not converge due to the nonsymmetry of the iteration matrix. We propose a modified algorithm that incorporates a small shift parameter, and we give the conditions that guarantee convergence of this method to a fixed point of a slightly perturbed problem. We also give perturbation bounds for this fixed point. Moreover, we discuss how to use Krylov subspace methods to efficiently estimate the leftmost eigenvalue of a certain matrix to select a proper shift parameter. The modified algorithm is illustrated with test problems from computed tomography

Strategies for spectrum slicing based on restarted Lanczos methods

In the context of symmetric-definite generalized eigenvalue problems, it is often required to compute all eigenvalues contained in a prescribed interval. For large-scale problems, the method of choice is the so-called spectrum slicing technique: a shift-and-invert Lanczos method combined with a dynamic shift selection that sweeps the interval in a smart way. This kind of strategies were proposed initially in the context of unrestarted Lanczos methods, back in the 1990's. We propose variations that try to incorporate recent developments in the field of Krylov methods, including thick restarting in the Lanczos solver and a rational Krylov update when moving from one shift to the next. We discuss a parallel implementation in the SLEPc library and provide performance results. © 2012 Springer Science+Business Media, LLC.This work was supported by the Spanish Ministerio de Ciencia e Innovacion under grant TIN2009-07519.Campos González, MC.; Román Moltó, JE. (2012). Strategies for spectrum slicing based on restarted Lanczos methods. Numerical Algorithms. 60(2):279-295. https://doi.org/10.1007/s11075-012-9564-z279295602Amestoy, P.R, Duff, I.S., L’Excellent, J.Y.: Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput. Methods Appl. Mech. Eng. 184(2–4), 501–520 (2000)Balay, S., Brown, J., Buschelman, K., Eijkhout, V., Gropp, W., Kaushik, D., Knepley, M., McInnes, L.C., Smith, B., Zhang, H.: PETSc users manual. Tech. Rep. ANL-95/11 - Revision 3.2, Argonne National Laboratory (2011)Ericsson, T., Ruhe, A.: The spectral transformation Lanczos method for the numerical solution of large sparse generalized symmetric eigenvalue problems. Math. Comput. 35(152), 1251–1268 (1980)Grimes, R.G., Lewis, J.G., Simon, H.D.: A shifted block Lanczos algorithm for solving sparse symmetric generalized eigenproblems. SIAM J. Matrix Anal. Appl. 15(1), 228–272 (1994)Hernandez, V., Roman, J.E., Vidal, V.: SLEPc: a scalable and flexible toolkit for the solution of eigenvalue problems. ACM Trans. Math. Softw. 31(3), 351–362 (2005)Hernandez, V., Roman, J.E., Tomas, A.: Parallel Arnoldi eigensolvers with enhanced scalability via global communications rearrangement. Parallel Comput. 33(7–8), 521–540 (2007)Marques, O.A.: BLZPACK: description and user’s guide. Tech. Rep. TR/PA/95/30, CERFACS, Toulouse, France (1995)Meerbergen, K.: Changing poles in the rational Lanczos method for the Hermitian eigenvalue problem. Numer. Linear Algebra Appl. 8(1), 33–52 (2001)Meerbergen, K., Scott, J.: The design of a block rational Lanczos code with partial reorthogonalization and implicit restarting. Tech. Rep. RAL-TR-2000-011, Rutherford Appleton Laboratory (2000)Nour-Omid, B., Parlett, B.N., Ericsson, T., Jensen, P.S.: How to implement the spectral transformation. Math. Comput. 48(178), 663–673 (1987)Olsson, K.H.A., Ruhe, A.: Rational Krylov for eigenvalue computation and model order reduction. BIT Numer. Math. 46, 99–111 (2006)Ruhe, A.: Rational Krylov sequence methods for eigenvalue computation. Linear Algebra Appl. 58, 391–405 (1984)Ruhe, A.: Rational Krylov subspace method. In: Bai, Z., Demmel, J., Dongarra, J., Ruhe, A., van der Vorst, H. (eds.) Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, Society for Industrial and Applied Mathematics, pp. 246–249. Philadelphia (2000)Sorensen, D.C.: Implicit application of polynomial filters in a k-step Arnoldi method. SIAM J. Matrix Anal. Appl. 13, 357–385 (1992)Stewart, G.W.: A Krylov–Schur algorithm for large eigenproblems. SIAM J. Matrix Anal. Appl. 23(3), 601–614 (2001)Vidal, AM., Garcia, V.M., Alonso, P., Bernabeu, M.O.: Parallel computation of the eigenvalues of symmetric Toeplitz matrices through iterative methods. J. Parallel Distrib. Comput. 68(8), 1113–1121 (2008)Wu, K., Simon, H.: Thick-restart Lanczos method for large symmetric eigenvalue problems. SIAM J. Matrix Anal. Appl. 22(2), 602–616 (2000)Zhang, H., Smith, B., Sternberg, M., Zapol, P.: SIPs: Shift-and-invert parallel spectral transformations. ACM Trans. Math. Softw. 33(2), 1–19 (2007

The Bramble-Pasciak preconditioner for saddle point problems

The Bramble-Pasciak Conjugate Gradient method is a well known tool to solve linear systems in saddle point form. A drawback of this method in order to ensure applicability of Conjugate Gradients is the need for scaling the preconditioner which typically involves the solution of an eigenvalue problem. Here, we introduce a modified preconditioner and inner product which without scaling enable the use of a MINRES variant and can be used for the simplified Lanczos process. Furthermore, the modified preconditioner and inner product can be combined with the original Bramble-Pasciak setup to give new preconditioners and inner products. We undermine the new methods by showing numerical experiments for Stokes problems