Adaptive Wavelet Methods for Variational Formulations of Nonlinear Elliptic PDES on Tensor-Product Domains

This thesis is concerned with the numerical solution of boundary value problems (BVPs) governed by semilinear elliptic partial differential equations (PDEs). Semilinearity here refers to a special case of nonlinearity, i.e., the case of a linear operator combined with a nonlinear operator acting as a perturbation. In general, such BVPs are solved in an iterative fashion. It is, therefore, of primal importance to develop efficient schemes that guarantee convergence of the numerically approximated PDE solutions towards the exact solution. Unlike the typical finite element method (FEM) theory for the numerical solution of the nonlinear operators, the new adaptive wavelet theory proposed in [Cohen.Dahmen.DeVore:2003:a, Cohen.Dahmen.DeVore:2003:b] guarantees convergence of adaptive schemes with fixed approximation rates. Furthermore, optimal, i.e., linear, complexity estimates of such adaptive solution methods have been established. These achievements are possible since wavelets allow for a completely new perspective to attack BVPs: namely, to represent PDEs in their original infinite dimensional realm. Wavelets are the ideal candidate for this purpose since they allow to represent functions in infinite-dimensional general Banach or Hilbert spaces and operators on these. The purpose of adaptivity in the solution process of nonlinear PDEs is to invest extra degrees of freedom (DOFs) only where necessary, i.e., where the exact solution requires a higher number of function coefficients to represent it accurately. Wavelets in this context represent function bases with special analytical properties, e.g., the wavelets considered herein are piecewise polynomials, have compact support and norm equivalences between certain function spaces and the l_2 sequence spaces of expansion coefficients exist. This new paradigm presents nevertheless some problems in the design of practical algorithms. Imposing a certain structure, a tree structure, remedies these problems completely while restricting the applicability of the theoretical scheme only very slightly. It turns out that the considered approach naturally fits the theoretical background of nonlinear PDEs. The practical realization on a computer, however, requires to reduce the relevant ingredients to finite-dimensional quantities. It is this particular aspect that is the guiding principle of this thesis. This theoretical framework is implemented in the course of this thesis in a truly dimensionally unrestricted adaptive wavelet program code, which allows one to harness the proven theoretical results for the first time when numerically solving the above mentioned BVPs. In the implementation, great emphasis is put on speed, i.e., overall execution speed and convergence speed, while not sacrificing on the freedom to adapt many important numerical details at runtime and not at the compilation stage. This means that the user can test and choose wavelets perfectly suitable for any specific task without having to rebuild the software. The computational overhead of these freedoms is minimized by caching any interim data, e.g., values for the preconditioners and polynomial representations of wavelets in multiple dimensions. Exploiting the structure in the construction of wavelet spaces prevents this step from becoming a burden on the memory requirements while at the same time providing a huge performance boost because necessary computations are only executed as needed and then only once. The essential BVP boundary conditions are enforced using trace operators, which leads to a saddle point problem formulation. This particular treatment of boundary conditions is very flexible, which especially useful if changing boundary conditions have to be accommodated, e.g., when iteratively solving control problems with Dirichlet boundary control based upon the herein considered PDE operators. Another particular feature is that saddle point problems allow for a variety of different geometrical setups, including fictitious domain approaches. Numerical studies of 2D and 3D PDEs and BVPs demonstrate the feasibility and performance of the developed schemes. Local transformations of the wavelet basis are employed to lower the absolute condition number of the already optimally preconditioned operators. The effect of these basis transformations can be seen in the absolute runtimes of solution processes, where the semilinear PDEs are solved as fast as in fractions of a second. This task can be accomplished using simple Richardson-style solvers, e.g., the method of steepest descent, or more involved solvers like the Newton's method. The BVPs are solved using an adaptive Uzawa algorithm, which requires repeated solution of semilinear PDE sub-problems. The efficiency of different numerical methods is compared and the theoretical optimal convergence rates and complexity estimates are verified. In summary, this thesis presents for the first time a numerically competitive implementation of a new theoretical paradigm to solve semilinear elliptic PDEs in arbitrary space dimensions with a complete convergence and complexity theory

Inference, Computation, and Games

In this thesis, we use statistical inference and competitive games to design algorithms for computational mathematics. In the first part, comprising chapters two through six, we use ideas from Gaussian process statistics to obtain fast solvers for differential and integral equations. We begin by observing the equivalence of conditional (near-)independence of Gaussian processes and the (near-)sparsity of the Cholesky factors of its precision and covariance matrices. This implies the existence of a large class of dense matrices with almost sparse Cholesky factors, thereby greatly increasing the scope of application of sparse Cholesky factorization. Using an elimination ordering and sparsity pattern motivated by the screening effect in spatial statistics, we can compute approximate Cholesky factors of the covariance matrices of Gaussian processes admitting a screening effect in near-linear computational complexity. These include many popular smoothness priors such as the Matérn class of covariance functions. In the special case of Green's matrices of elliptic boundary value problems (with possibly unknown elliptic operators of arbitrarily high order, with possibly rough coefficients), we can use tools from numerical homogenization to prove the exponential accuracy of our method. This result improves the state-of-the-art for solving general elliptic integral equations and provides the first proof of an exponential screening effect. We also derive a fast solver for elliptic partial differential equations, with accuracy-vs-complexity guarantees that improve upon the state-of-the-art. Furthermore, the resulting solver is performant in practice, frequently beating established algebraic multigrid libraries such as AMGCL and Trilinos on a series of challenging problems in two and three dimensions. Finally, for any given covariance matrix, we obtain a closed-form expression for its optimal (in terms of Kullback-Leibler divergence) approximate inverse-Cholesky factorization subject to a sparsity constraint, recovering Vecchia approximation and factorized sparse approximate inverses. Our method is highly robust, embarrassingly parallel, and further improves our asymptotic results on the solution of elliptic integral equations. We also provide a way to apply our techniques to sums of independent Gaussian processes, resolving a major limitation of existing methods based on the screening effect. As a result, we obtain fast algorithms for large-scale Gaussian process regression problems with possibly noisy measurements. In the second part of this thesis, comprising chapters seven through nine, we study continuous optimization through the lens of competitive games. In particular, we consider competitive optimization, where multiple agents attempt to minimize conflicting objectives. In the single-agent case, the updates of gradient descent are minimizers of quadratically regularized linearizations of the loss function. We propose to generalize this idea by using the Nash equilibria of quadratically regularized linearizations of the competitive game as updates (linearize the game). We provide fundamental reasons why the natural notion of linearization for competitive optimization problems is given by the multilinear (as opposed to linear) approximation of the agents' loss functions. The resulting algorithm, which we call competitive gradient descent, thus provides a natural generalization of gradient descent to competitive optimization. By using ideas from information geometry, we extend CGD to competitive mirror descent (CMD) that can be applied to a vast range of constrained competitive optimization problems. CGD and CMD resolve the cycling problem of simultaneous gradient descent and show promising results on problems arising in constrained optimization, robust control theory, and generative adversarial networks. Finally, we point out the GAN-dilemma that refutes the common interpretation of GANs as approximate minimizers of a divergence obtained in the limit of a fully trained discriminator. Instead, we argue that GAN performance relies on the implicit competitive regularization (ICR) due to the simultaneous optimization of generator and discriminator and support this hypothesis with results on low-dimensional model problems and GANs on CIFAR10.</p