Reduced Order Modeling for Nonlinear PDE-constrained Optimization using Neural Networks

Nonlinear model predictive control (NMPC) often requires real-time solution to optimization problems. However, in cases where the mathematical model is of high dimension in the solution space, e.g. for solution of partial differential equations (PDEs), black-box optimizers are rarely sufficient to get the required online computational speed. In such cases one must resort to customized solvers. This paper present a new solver for nonlinear time-dependent PDE-constrained optimization problems. It is composed of a sequential quadratic programming (SQP) scheme to solve the PDE-constrained problem in an offline phase, a proper orthogonal decomposition (POD) approach to identify a lower dimensional solution space, and a neural network (NN) for fast online evaluations. The proposed method is showcased on a regularized least-square optimal control problem for the viscous Burgers' equation. It is concluded that significant online speed-up is achieved, compared to conventional methods using SQP and finite elements, at a cost of a prolonged offline phase and reduced accuracy.Comment: Accepted for publishing at the 58th IEEE Conference on Decision and Control, Nice, France, 11-13 December, https://cdc2019.ieeecss.org

A fast and accurate domain-decomposition nonlinear manifold reduced order model

This paper integrates nonlinear-manifold reduced order models (NM-ROMs) with domain decomposition (DD). NM-ROMs approximate the FOM state in a nonlinear-manifold by training a shallow, sparse autoencoder using FOM snapshot data. These NM-ROMs can be advantageous over linear-subspace ROMs (LS-ROMs) for problems with slowly decaying Kolmogorov $n$-width. However, the number of NM-ROM parameters that need to trained scales with the size of the FOM. Moreover, for "extreme-scale" problems, the storage of high-dimensional FOM snapshots alone can make ROM training expensive. To alleviate the training cost, this paper applies DD to the FOM, computes NM-ROMs on each subdomain, and couples them to obtain a global NM-ROM. This approach has several advantages: Subdomain NM-ROMs can be trained in parallel, each involve fewer parameters to be trained than global NM-ROMs, require smaller subdomain FOM dimensional training data, and training of subdomain NM-ROMs can tailor them to subdomain-specific features of the FOM. The shallow, sparse architecture of the autoencoder used in each subdomain NM-ROM allows application of hyper-reduction (HR), reducing the complexity caused by nonlinearity and yielding computational speedup of the NM-ROM. This paper provides the first application of NM-ROM (with HR) to a DD problem. In particular, it details an algebraic DD formulation of the FOM, trains a NM-ROM with HR for each subdomain, and develops a sequential quadratic programming (SQP) solver to evaluate the coupled global NM-ROM. Theoretical convergence results for the SQP method and a priori and a posteriori error estimates for the DD NM-ROM with HR are provided. The proposed DD NM-ROM with HR approach is numerically compared to a DD LS-ROM with HR on 2D steady-state Burgers' equation, showing an order of magnitude improvement in accuracy of the proposed DD NM-ROM over the DD LS-ROM

Reduced order output feedback control design for PDE systems using proper orthogonal decomposition and nonlinear semidefinite programming

AbstractThe design of an optimal (output feedback) reduced order control (ROC) law for a dynamic control system is an important example of a difficult and in general non-convex (nonlinear) optimal control problem. In this paper we present a novel numerical strategy to the solution of the ROC design problem if the control system is described by partial differential equations (PDE). The discretization of the ROC problem with PDE constraints leads to a large scale (non-convex) nonlinear semidefinite program (NSDP). For reducing the size of the high dimensional control system, first, we apply a proper orthogonal decomposition (POD) method to the discretized PDE. The POD approach leads to a low dimensional model of the control system. Thereafter, we solve the corresponding small-sized NSDP by a fully iterative interior point constraint trust region (IPCTR) algorithm. IPCTR is designed to take advantage of the special structure of the NSDP. Finally, the solution is a ROC for the low dimensional approximation of the control system. In our numerical examples we demonstrate that the reduced order controller computed from the small scaled problem can be used to control the large scale approximation of the PDE system

A Multigrid Method for the Efficient Numerical Solution of Optimization Problems Constrained by Partial Differential Equations

We study the minimization of a quadratic functional subject to constraints given by a linear or semilinear elliptic partial differential equation with distributed control. Further, pointwise inequality constraints on the control are accounted for. In the linear-quadratic case, the discretized optimality conditions yield a large, sparse, and indefinite system with saddle point structure. One main contribution of this thesis consists in devising a coupled multigrid solver which avoids full constraint elimination. To this end, we define a smoothing iteration incorporating elements from constraint preconditioning. A local mode analysis shows that for discrete optimality systems, we can expect smoothing rates close to those obtained with respect to the underlying constraint PDE. Our numerical experiments include problems with constraints where standard pointwise smoothing is known to fail for the underlying PDE. In particular, we consider anisotropic diffusion and convection-diffusion problems. The framework of our method allows to include line smoothers or ILU-factorizations, which are suitable for such problems. In all cases, numerical experiments show that convergence rates do not depend on the mesh size of the finest level and discrete optimality systems can be solved with a small multiple of the computational cost which is required to solve the underlying constraint PDE. Employing the full multigrid approach, the computational cost is proportional to the number of unknowns on the finest grid level. We discuss the role of the regularization parameter in the cost functional and show that the convergence rates are robust with respect to both the fine grid mesh size and the regularization parameter under a mild restriction on the next to coarsest mesh size. Incorporating spectral filtering for the reduced Hessian in the control smoothing step allows us to weaken the mesh size restriction. As a result, problems with near-vanishing regularization parameter can be treated efficiently with a negligible amount of additional computational work. For fine discretizations, robust convergence is obtained with rates which are independent of the regularization parameter, the coarsest mesh size, and the number of levels. In order to treat linear-quadratic problems with pointwise inequality constraints on the control, the multigrid approach is modified to solve subproblems generated by a primal-dual active set strategy (PDAS). Numerical experiments demonstrate the high efficiency of this approach due to mesh-independent convergence of both the outer PDAS method and the inner multigrid solver. The PDAS-multigrid method is incorporated in the sequential quadratic programming (SQP) framework. Inexact Newton techniques further enhance the computational efficiency. Globalization is implemented with a line search based on the augmented Lagrangian merit function. Numerical experiments highlight the efficiency of the resulting SQP-multigrid approach. In all cases, locally superlinear convergence of the SQP method is observed. In combination with the mesh-independent convergence rate of the inner solver, a solution method with optimal efficiency is obtained

The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach

We study the combination of the alternating direction method of multipliers (ADMM) with physics-informed neural networks (PINNs) for a general class of nonsmooth partial differential equation (PDE)-constrained optimization problems, where additional regularization can be employed for constraints on the control or design variables. The resulting ADMM-PINNs algorithmic framework substantially enlarges the applicable range of PINNs to nonsmooth cases of PDE-constrained optimization problems. The application of the ADMM makes it possible to untie the PDE constraints and the nonsmooth regularization terms for iterations. Accordingly, at each iteration, one of the resulting subproblems is a smooth PDE-constrained optimization which can be efficiently solved by PINNs, and the other is a simple nonsmooth optimization problem which usually has a closed-form solution or can be efficiently solved by various standard optimization algorithms or pre-trained neural networks. The ADMM-PINNs algorithmic framework does not require to solve PDEs repeatedly, and it is mesh-free, easy to implement, and scalable to different PDE settings. We validate the efficiency of the ADMM-PINNs algorithmic framework by different prototype applications, including inverse potential problems, source identification in elliptic equations, control constrained optimal control of the Burgers equation, and sparse optimal control of parabolic equations

A Direct Integral Pseudospectral Method for Solving a Class of Infinite-Horizon Optimal Control Problems Using Gegenbauer Polynomials and Certain Parametric Maps

We present a novel direct integral pseudospectral (PS) method (a direct IPS method) for solving a class of continuous-time infinite-horizon optimal control problems (IHOCs). The method transforms the IHOCs into finite-horizon optimal control problems (FHOCs) in their integral forms by means of certain parametric mappings, which are then approximated by finite-dimensional nonlinear programming problems (NLPs) through rational collocations based on Gegenbauer polynomials and Gegenbauer-Gauss-Radau (GGR) points. The paper also analyzes the interplay between the parametric maps, barycentric rational collocations based on Gegenbauer polynomials and GGR points, and the convergence properties of the collocated solutions for IHOCs. Some novel formulas for the construction of the rational interpolation weights and the GGR-based integration and differentiation matrices in barycentric-trigonometric forms are derived. A rigorous study on the error and convergence of the proposed method is presented. A stability analysis based on the Lebesgue constant for GGR-based rational interpolation is investigated. Two easy-to-implement pseudocodes of computational algorithms for computing the barycentric-trigonometric rational weights are described. Two illustrative test examples are presented to support the theoretical results. We show that the proposed collocation method leveraged with a fast and accurate NLP solver converges exponentially to near-optimal approximations for a coarse collocation mesh grid size. The paper also shows that typical direct spectral/PS- and IPS-methods based on classical Jacobi polynomials and certain parametric maps usually diverge as the number of collocation points grow large, if the computations are carried out using floating-point arithmetic and the discretizations use a single mesh grid whether they are of Gauss/Gauss-Radau (GR) type or equally-spaced.Comment: 33 pages, 19 figure

The automation of PDE-constrained optimisation and its applications

This thesis is concerned with the automation of solving optimisation problems constrained by partial differential equations (PDEs). Gradient-based optimisation algorithms are the key to solve optimisation problems of practical interest. The required derivatives can be efficiently computed with the adjoint approach. However, current methods for the development of adjoint models often require a significant amount of effort and expertise, in particular for non-linear time-dependent problems. This work presents a new high-level reinterpretation of algorithmic differentiation to develop adjoint models. This reinterpretation considers the discrete system as a sequence of equation solves. Applying this approach to a general finite-element framework results in an automatic and robust way of deriving and solving adjoint models. This drastically reduces the development effort compared to traditional methods. Based on this result, a new framework for rapidly defining and solving optimisation problems constrained by PDEs is developed. The user specifies the discrete optimisation problem in a compact high-level language that resembles the mathematical structure of the underlying system. All remaining steps, including parameter updates, PDE solves and derivative computations, are performed without user intervention. The framework can be applied to a wide range of governing PDEs, and interfaces to various gradient-free and gradient-based optimisation algorithms. The capabilities of this framework are demonstrated through the application to two PDE-constrained optimisation problems. The first is concerned with the optimal layout of turbines in tidal stream farms; this optimisation problem is one of the main challenges facing the marine renewable energy industry. The second application applies data assimilation to reconstruct the profile of tsunami waves based on inundation observations. This provides the first step towards the general reconstruction of tsunami signals from satellite information

Asymptotic Stability of POD based Model Predictive Control for a semilinear parabolic PDE

In this article a stabilizing feedback control is computed for a semilinear parabolic partial differential equation utilizing a nonlinear model predictive (NMPC) method. In each level of the NMPC algorithm the finite time horizon open loop problem is solved by a reduced-order strategy based on proper orthogonal decomposition (POD). A stability analysis is derived for the combined POD-NMPC algorithm so that the lengths of the finite time horizons are chosen in order to ensure the asymptotic stability of the computed feedback controls. The proposed method is successfully tested by numerical examples