Convex operator-theoretic methods in stochastic control

This paper is about operator-theoretic methods for solving nonlinear stochastic optimal control problems to global optimality. These methods leverage on the convex duality between optimally controlled diffusion processes and Hamilton-Jacobi-Bellman (HJB) equations for nonlinear systems in an ergodic Hilbert-Sobolev space. In detail, a generalized Bakry-Emery condition is introduced under which one can establish the global exponential stabilizability of a large class of nonlinear systems. It is shown that this condition is sufficient to ensure the existence of solutions of the ergodic HJB for stochastic optimal control problems on infinite time horizons. Moreover, a novel dynamic programming recursion for bounded linear operators is introduced, which can be used to numerically solve HJB equations by a Galerkin projection

A Laguerre spectral method for quadratic optimal control of nonlinear systems in a semi-infinite interval

This paper presents a Laguerre homotopy method for quadratic optimal control problems in semi-infinite intervals (LaHOC), with particular interests given to nonlinear interconnected large-scale dynamic systems. In LaHOC, the spectral homotopy analysis method is used to derive an iterative solver for the nonlinear two-point boundary value problem derived from Pontryagin\u27s maximum principle. A proof of local convergence of the LaHOC is provided. Numerical comparisons are made between the LaHOC, Matlab BVP5C generated results and results from the literature for two nonlinear optimal control problems. The results show that LaHOC is superior in both accuracy and efficiency

Implicit sampling for path integral control, Monte Carlo localization, and SLAM

The applicability and usefulness of implicit sampling in stochastic optimal control, stochastic localization, and simultaneous localization and mapping (SLAM), is explored; implicit sampling is a recently-developed variationally-enhanced sampling method. The theory is illustrated with examples, and it is found that implicit sampling is significantly more efficient than current Monte Carlo methods in test problems for all three applications

A Direct Integral Pseudospectral Method for Solving a Class of Infinite-Horizon Optimal Control Problems Using Gegenbauer Polynomials and Certain Parametric Maps

We present a novel direct integral pseudospectral (PS) method (a direct IPS method) for solving a class of continuous-time infinite-horizon optimal control problems (IHOCs). The method transforms the IHOCs into finite-horizon optimal control problems (FHOCs) in their integral forms by means of certain parametric mappings, which are then approximated by finite-dimensional nonlinear programming problems (NLPs) through rational collocations based on Gegenbauer polynomials and Gegenbauer-Gauss-Radau (GGR) points. The paper also analyzes the interplay between the parametric maps, barycentric rational collocations based on Gegenbauer polynomials and GGR points, and the convergence properties of the collocated solutions for IHOCs. Some novel formulas for the construction of the rational interpolation weights and the GGR-based integration and differentiation matrices in barycentric-trigonometric forms are derived. A rigorous study on the error and convergence of the proposed method is presented. A stability analysis based on the Lebesgue constant for GGR-based rational interpolation is investigated. Two easy-to-implement pseudocodes of computational algorithms for computing the barycentric-trigonometric rational weights are described. Two illustrative test examples are presented to support the theoretical results. We show that the proposed collocation method leveraged with a fast and accurate NLP solver converges exponentially to near-optimal approximations for a coarse collocation mesh grid size. The paper also shows that typical direct spectral/PS- and IPS-methods based on classical Jacobi polynomials and certain parametric maps usually diverge as the number of collocation points grow large, if the computations are carried out using floating-point arithmetic and the discretizations use a single mesh grid whether they are of Gauss/Gauss-Radau (GR) type or equally-spaced.Comment: 33 pages, 19 figure

Nonlinear model predictive control for aggressive maneuvers in a variable pitch quadrotor based on the extended modal series method

Este trabaja presenta el diseño y la implementación de un control predictivo basado en las series modales extendidas en un cuadrotor de ángulo de ataque variable.This work presents the design and implementation of a predictive control based on the extended modal series method in a variable pitch quadrotor.Magíster en Ingeniería ElectrónicaMaestrí

Sum-of-Squares approach to feedback control of laminar wake flows

A novel nonlinear feedback control design methodology for incompressible fluid flows aiming at the optimisation of long-time averages of flow quantities is presented. It applies to reduced-order finite-dimensional models of fluid flows, expressed as a set of first-order nonlinear ordinary differential equations with the right-hand side being a polynomial function in the state variables and in the controls. The key idea, first discussed in Chernyshenko et al. 2014, Philos. T. Roy. Soc. 372(2020), is that the difficulties of treating and optimising long-time averages of a cost are relaxed by using the upper/lower bounds of such averages as the objective function. In this setting, control design reduces to finding a feedback controller that optimises the bound, subject to a polynomial inequality constraint involving the cost function, the nonlinear system, the controller itself and a tunable polynomial function. A numerically tractable approach to the solution of such optimisation problems, based on Sum-of-Squares techniques and semidefinite programming, is proposed. To showcase the methodology, the mitigation of the fluctuation kinetic energy in the unsteady wake behind a circular cylinder in the laminar regime at Re=100, via controlled angular motions of the surface, is numerically investigated. A compact reduced-order model that resolves the long-term behaviour of the fluid flow and the effects of actuation, is derived using Proper Orthogonal Decomposition and Galerkin projection. In a full-information setting, feedback controllers are then designed to reduce the long-time average of the kinetic energy associated with the limit cycle. These controllers are then implemented in direct numerical simulations of the actuated flow. Control performance, energy efficiency, and physical control mechanisms identified are analysed. Key elements, implications and future work are discussed