Convex operator-theoretic methods in stochastic control

This paper is about operator-theoretic methods for solving nonlinear stochastic optimal control problems to global optimality. These methods leverage on the convex duality between optimally controlled diffusion processes and Hamilton-Jacobi-Bellman (HJB) equations for nonlinear systems in an ergodic Hilbert-Sobolev space. In detail, a generalized Bakry-Emery condition is introduced under which one can establish the global exponential stabilizability of a large class of nonlinear systems. It is shown that this condition is sufficient to ensure the existence of solutions of the ergodic HJB for stochastic optimal control problems on infinite time horizons. Moreover, a novel dynamic programming recursion for bounded linear operators is introduced, which can be used to numerically solve HJB equations by a Galerkin projection

Asymptotic Stability of POD based Model Predictive Control for a semilinear parabolic PDE

In this article a stabilizing feedback control is computed for a semilinear parabolic partial differential equation utilizing a nonlinear model predictive (NMPC) method. In each level of the NMPC algorithm the finite time horizon open loop problem is solved by a reduced-order strategy based on proper orthogonal decomposition (POD). A stability analysis is derived for the combined POD-NMPC algorithm so that the lengths of the finite time horizons are chosen in order to ensure the asymptotic stability of the computed feedback controls. The proposed method is successfully tested by numerical examples

The turnpike property in finite-dimensional nonlinear optimal control

Turnpike properties have been established long time ago in finite-dimensional optimal control problems arising in econometry. They refer to the fact that, under quite general assumptions, the optimal solutions of a given optimal control problem settled in large time consist approximately of three pieces, the first and the last of which being transient short-time arcs, and the middle piece being a long-time arc staying exponentially close to the optimal steady-state solution of an associated static optimal control problem. We provide in this paper a general version of a turnpike theorem, valuable for nonlinear dynamics without any specific assumption, and for very general terminal conditions. Not only the optimal trajectory is shown to remain exponentially close to a steady-state, but also the corresponding adjoint vector of the Pontryagin maximum principle. The exponential closedness is quantified with the use of appropriate normal forms of Riccati equations. We show then how the property on the adjoint vector can be adequately used in order to initialize successfully a numerical direct method, or a shooting method. In particular, we provide an appropriate variant of the usual shooting method in which we initialize the adjoint vector, not at the initial time, but at the middle of the trajectory

Analysis of unconstrained nonlinear MPC schemes with time varying control horizon

For discrete time nonlinear systems satisfying an exponential or finite time controllability assumption, we present an analytical formula for a suboptimality estimate for model predictive control schemes without stabilizing terminal constraints. Based on our formula, we perform a detailed analysis of the impact of the optimization horizon and the possibly time varying control horizon on stability and performance of the closed loop

On the existence of oscillating solutions in non-monotone Mean-Field Games

For non-monotone single and two-populations time-dependent Mean-Field Game systems we obtain the existence of an infinite number of branches of non-trivial solutions. These non-trivial solutions are in particular shown to exhibit an oscillatory behaviour when they are close to the trivial (constant) one. The existence of such branches is derived using local and global bifurcation methods, that rely on the analysis of eigenfunction expansions of solutions to the associated linearized problem. Numerical analysis is performed on two different models to observe the oscillatory behaviour of solutions predicted by bifurcation theory, and to study further properties of branches far away from bifurcation points.Comment: 24 pages, 10 figure