Proximal Galerkin: A structure-preserving finite element method for pointwise bound constraints

The proximal Galerkin finite element method is a high-order, low iteration complexity, nonlinear numerical method that preserves the geometric and algebraic structure of bound constraints in infinite-dimensional function spaces. This paper introduces the proximal Galerkin method and applies it to solve free boundary problems, enforce discrete maximum principles, and develop scalable, mesh-independent algorithms for optimal design. The paper leads to a derivation of the latent variable proximal point (LVPP) algorithm: an unconditionally stable alternative to the interior point method. LVPP is an infinite-dimensional optimization algorithm that may be viewed as having an adaptive barrier function that is updated with a new informative prior at each (outer loop) optimization iteration. One of the main benefits of this algorithm is witnessed when analyzing the classical obstacle problem. Therein, we find that the original variational inequality can be replaced by a sequence of semilinear partial differential equations (PDEs) that are readily discretized and solved with, e.g., high-order finite elements. Throughout this work, we arrive at several unexpected contributions that may be of independent interest. These include (1) a semilinear PDE we refer to as the entropic Poisson equation; (2) an algebraic/geometric connection between high-order positivity-preserving discretizations and certain infinite-dimensional Lie groups; and (3) a gradient-based, bound-preserving algorithm for two-field density-based topology optimization. The complete latent variable proximal Galerkin methodology combines ideas from nonlinear programming, functional analysis, tropical algebra, and differential geometry and can potentially lead to new synergies among these areas as well as within variational and numerical analysis

Wasserstein Consensus ADMM

We introduce Wasserstein consensus alternating direction method of multipliers (ADMM) and its entropic-regularized version: Sinkhorn consensus ADMM, to solve measure-valued optimization problems with convex additive objectives. Several problems of interest in stochastic prediction and learning can be cast in this form of measure-valued convex additive optimization. The proposed algorithm generalizes a variant of the standard Euclidean ADMM to the space of probability measures but departs significantly from its Euclidean counterpart. In particular, we derive a two layer ADMM algorithm wherein the outer layer is a variant of consensus ADMM on the space of probability measures while the inner layer is a variant of Euclidean ADMM. The resulting computational framework is particularly suitable for solving Wasserstein gradient flows via distributed computation. We demonstrate the proposed framework using illustrative numerical examples

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

SAM as an Optimal Relaxation of Bayes

Sharpness-aware minimization (SAM) and related adversarial deep-learning methods can drastically improve generalization, but their underlying mechanisms are not yet fully understood. Here, we establish SAM as a relaxation of the Bayes objective where the expected negative-loss is replaced by the optimal convex lower bound, obtained by using the so-called Fenchel biconjugate. The connection enables a new Adam-like extension of SAM to automatically obtain reasonable uncertainty estimates, while sometimes also improving its accuracy. By connecting adversarial and Bayesian methods, our work opens a new path to robustness

Pairwise MRF Calibration by Perturbation of the Bethe Reference Point

We investigate different ways of generating approximate solutions to the pairwise Markov random field (MRF) selection problem. We focus mainly on the inverse Ising problem, but discuss also the somewhat related inverse Gaussian problem because both types of MRF are suitable for inference tasks with the belief propagation algorithm (BP) under certain conditions. Our approach consists in to take a Bethe mean-field solution obtained with a maximum spanning tree (MST) of pairwise mutual information, referred to as the \emph{Bethe reference point}, for further perturbation procedures. We consider three different ways following this idea: in the first one, we select and calibrate iteratively the optimal links to be added starting from the Bethe reference point; the second one is based on the observation that the natural gradient can be computed analytically at the Bethe point; in the third one, assuming no local field and using low temperature expansion we develop a dual loop joint model based on a well chosen fundamental cycle basis. We indeed identify a subclass of planar models, which we refer to as \emph{Bethe-dual graph models}, having possibly many loops, but characterized by a singly connected dual factor graph, for which the partition function and the linear response can be computed exactly in respectively O(N) and $O(N^2)$ operations, thanks to a dual weight propagation (DWP) message passing procedure that we set up. When restricted to this subclass of models, the inverse Ising problem being convex, becomes tractable at any temperature. Experimental tests on various datasets with refined $L_0$ or $L_1$ regularization procedures indicate that these approaches may be competitive and useful alternatives to existing ones.Comment: 54 pages, 8 figure. section 5 and refs added in V

Numerical Methods for Optimal Transport and Elastic Shape Optimization

In this thesis, we consider a novel unbalanced optimal transport model incorporating singular sources, we develop a numerical computation scheme for an optimal transport distance on graphs, we propose a simultaneous elastic shape optimization problem for bone tissue engineering, and we investigate optimal material distributions on thin elastic objects. The by now classical theory of optimal transport admits a metric between measures of the same total mass. Various generalizations of this so-called Wasserstein distance have been recently studied in the literature. In particular, these have been motivated by imaging applications, where the mass-preserving condition is too restrictive. Based on the Benamou Brenier formulation we present a novel unbalanced optimal transport model by introducing a source term in the continuity equation, which is incorporated in the path energy by a squared L2-norm in time of a functional with linear growth in space. As a key advantage of our model, this source term functional allows singular sources in space. We demonstrate the existence of constant speed geodesics in the space of Radon measures. Furthermore, for a numerical computation scheme, we apply a proximal splitting algorithm for a finite element discretization. On discrete spaces, Maas introduced a Benamou Brenier formulation, where a kinetic energy is defined via an appropriate (e.g., logarithmic) averaging of mass on nodes and momentum on edges. Concerning a numerical optimization scheme, this, unfortunately, couples all these variables on the graph. We propose a conforming finite element discretization in time and prove convergence of corresponding path energy minimizing curves. To apply a proximal splitting algorithm, we introduce suitable auxiliary variables. Besides similar projections as for the classical optimal transport distance and additional simple operations, this allows us to separate the nonlinearity given by the averaging operator to projections onto three-dimensional convex sets, the associated (e.g., logarithmic) cones. In elastic shape optimization, we are usually concerned with finding a subdomain maximizing the mechanical stability w.r.t. given forces acting onto a larger domain of interest. Motivated by a biomechanical application in bone tissue engineering, where recently biologically degradable polymers have been explored as bone substitutes, we propose a simultaneous elastic shape optimization problem to guarantee stiffness of the polymer implant and of the complementary set where new bone tissue will grow first. Under the assumption that the microstructure of the scaffold is periodic, we optimize a single microcell. We define a novel cost functional depending on specific entries of the homogenized elasticity tensors of polymer and regrown bone. Additionally, the perimeter is penalized for regularizing the interface of the scaffold. For a numerical optimization scheme, we choose a phase-field model, which allows a diffuse approximation of the elastic objects and the perimeter by the Modica Mortola functional. We also incorporate further biomechanically relevant constraints like the diffusivity of the regrown bone. Finally, we investigate shape optimization problems for thin elastic objects. For a numerical discretization, we take into account the discrete Kirchhoff triangle (DKT) element for parametric surfaces and approximate the material distribution by a phase-field. To describe equilibrium deformations for a given force, we study different corresponding state equations. In particular, we consider nonlinear elasticity combining membrane and bending models. Furthermore, a special focus is on pure bending isometries, which can be efficiently approximated by the DKT element. We also analyze a one-dimensional model of nonlinear elastic planar beams, where our numerical simulations confirm and extend a theoretical classification result of the optimal design

On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints

The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that iteratively solve the system of optimality conditions on a grid of discrete points. Both algorithms use a block coordinate descent strategy and terminate once the optimality gap falls below the desired tolerance. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.Comment: 13 pages, 5 figures, 2 tables, published in the IEEE Transactions on Signal Processing. In previous versions, the example in Section VI.B contained some mistakes and inaccuracies, which have been fixed in this versio