Hessian Matrix-Free Lagrange-Newton-Krylov-Schur-Schwarz Methods for Elliptic Inverse Problems

This study focuses on the solution of inverse problems for elliptic systems. The inverse problem is constructed as a PDE-constrained optimization, where the cost function is the L2 norm of the difference between the measured data and the predicted state variable, and the constraint is an elliptic PDE. Particular examples of the system considered in this stud, are groundwater flow and radiation transport. The inverse problems are typically ill-posed due to error in measurements of the data. Regularization methods are employed to partially alleviate this problem. The PDE-constrained optimization is formulated as the minimization of a Lagrangian functional, formed from the regularized cost function and the discretized PDE, with respect to the parameters, the state variables, and the Lagrange multipliers. Our approach is known as an all at once method. An algorithm is proposed for an inverse problem that is capable of being extended to large scales. To overcome storage limitations, we develop a parallel preconditioned Newton-Krylov method employed in a Hessian-free manner. The preconditioners have an inner-outer structure, taking the form of a Schur complement (block factorization) at the outer level and Schwarz projections at the inner level. However, building an exact Schur complement is prohibitively expensive. Thus, we use Schur complement approximations, including the identity, probing, the Laplacian, the J operator, and a BFGS operator. For exact data the exact Schur complements are superior to the inexact approximations. However, for data with noise the inexact methods are competitive to or even better than the exact in every computational aspect. We also find that nousymmetric forms of the Karush-Kuhn-Tucker matrices and preconditioners are competitive to or better than the symmetric forms that are commonly used in the optimization community. In this study, iterative Tikhonov and Total Variation regularizations are proposed and compared to the standard regularizations and each other. For exact data with jump discontinuities the standard and iterative Total Variation regulations are superior to the standard and iterative Tikhonov regularizations. However, in the case of noisy data the proposed iterative Tikhonov regularizations are superior to the standard and iterative Total Variation methods. We also show that in some cases the iterative regularizations are better than the noniterative. To demonstrate the performance of the algorithm, including the effectiveness of the preconditioners and regularizations, synthetic one- and two-dimensional elliptic inverse problems are solved, and we also compare with other methodologies that are available in the literature. The proposed algorithm performs well with regard to robustness, reconstructs the parameter models effectively, and is easily implemented in the framework of the available parallel PDE software PETSc and the automatic differentiation software ADIC. The algorithm is also extendable to three-dimensional problems

Bringing PDEs to JAX with forward and reverse modes automatic differentiation

Partial differential equations (PDEs) are used to describe a variety of physical phenomena. Often these equations do not have analytical solutions and numerical approximations are used instead. One of the common methods to solve PDEs is the finite element method. Computing derivative information of the solution with respect to the input parameters is important in many tasks in scientific computing. We extend JAX automatic differentiation library with an interface to Firedrake finite element library. High-level symbolic representation of PDEs allows bypassing differentiating through low-level possibly many iterations of the underlying nonlinear solvers. Differentiating through Firedrake solvers is done using tangent-linear and adjoint equations. This enables the efficient composition of finite element solvers with arbitrary differentiable programs. The code is available at github.com/IvanYashchuk/jax-firedrake.Comment: Published as a workshop paper at ICLR 2020 DeepDiffEq worksho

The Idea and Concept of Metos3D: A Marine Ecosystem Toolkit for Optimization and Simulation in 3-D

The simulation and parameter optimization of coupled ocean circulation and ecosystem models in three space dimensions is one of the most challenging tasks in numerical climate research. Here we present a scientific toolkit that aims at supporting researchers by defining clear coupling interfaces, providing state-of-the-art numerical methods for simulation, parallelization and optimization while using only freely available and (to a great extend) platform-independent software. Besides defining a user-friendly coupling interface (API) for marine ecosystem or biogeochemical models, we heavily rely on the Portable, Extensible Toolkit for Scientific computation [PETSc] developed in Argonne Nat. Lab. [PETSc] for a wide variety of parallel linear and non-linear solvers and optimizers. We specifically focus on the usage of matrix-free Newton-Krylov methods for the fast computation of steady periodic solutions, and make use of the Transport Matrix Method (TMM) introduced by Khatiwala et al. in [KhViCa05]