Series Solution of Discrete Time Stochastic Optimal Control Problems

In this paper we consider discrete time stochastic optimal control problems over infinite and finite time horizons. We show that for a large class of such problems the Taylor polynomials of the solutions to the associated Dynamic Programming Equations can be computed degree by degree.Comment: arXiv admin note: text overlap with arXiv:1806.0412

Stochastic HJB Equations and Regular Singular Points

IIn this paper we show that some HJB equations arising from both finite and infinite horizon stochastic optimal control problems have a regular singular point at the origin. This makes them amenable to solution by power series techniques. This extends the work of Al'brecht who showed that the HJB equations of an infinite horizon deterministic optimal control problem can have a regular singular point at the origin, Al'brekht solved the HJB equations by power series, degree by degree. In particular, we show that the infinite horizon stochastic optimal control problem with linear dynamics, quadratic cost and bilinear noise leads to a new type of algebraic Riccati equation which we call the Stochastic Algebraic Riccati Equation (SARE). If SARE can be solved then one has a complete solution to this infinite horizon stochastic optimal control problem. We also show that a finite horizon stochastic optimal control problem with linear dynamics, quadratic cost and bilinear noise leads to a Stochastic Differential Riccati Equation (SDRE) that is well known. If these problems are the linear-quadratic-bilinear part of a nonlinear finite horizon stochastic optimal control problem then we show how the higher degree terms of the solutions can be computed degree by degree. To our knowledge this computation is new

The patchy Method for the Infinite Horizon Hamilton-Jacobi-Bellman Equation and its Accuracy

We introduce a modification to the patchy method of Navasca and Krener for solving the stationary Hamilton Jacobi Bellman equation. The numerical solution that we generate is a set of polynomials that approximate the optimal cost and optimal control on a partition of the state space. We derive an error bound for our numerical method under the assumption that the optimal cost is a smooth strict Lyupanov function. The error bound is valid when the number of subsets in the partition is not too large.Comment: 50 pages, 5 figure

Simplicial Nonlinear Principal Component Analysis

We present a new manifold learning algorithm that takes a set of data points lying on or near a lower dimensional manifold as input, possibly with noise, and outputs a simplicial complex that fits the data and the manifold. We have implemented the algorithm in the case where the input data can be triangulated. We provide triangulations of data sets that fall on the surface of a torus, sphere, swiss roll, and creased sheet embedded in a fifty dimensional space. We also discuss the theoretical justification of our algorithm.Comment: 21 pages, 6 figure

Model Predictive Regulation

We show how optimal nonlinear regulation can be achieved in a model predictive control fashion

Optimal Boundary Control of a Nonlinear Reaction Diffusion Equation via Completing the Square and Al'brekht's Method

The two contributions of this paper are as follows. The first is the solution of an infinite dimensional, boundary controlled Linear Quadratic Regulator by the simple and constructive method of completing the square. The second contribution is the extension of Al'brekht's method to the optimal stabilization of a boundary controlled, nonlinear Reaction Diffusion system

Optimal Boundary Control of a Nonlinear Reaction Diffusion Equation via Completing the Square and Al’brekht’s Method

The two contributions of this paper are as follows. The first is the solution of an infinite dimensional, boundary controlled Linear Quadratic Regulator by the simple and constructive method of completing the square. The second contribution is the extension of Al’brekht’s method to the optimal stabilization of a boundary controlled, nonlinear Reaction Diffusion system.AFOS

Boundary Control of the Beam Equation by Linear Quadratic Regulation

We present and solve a Linear Quadratic Regulator (LQR) for the boundary control of the beam equation. We use the simple technique of completing the square to get an explicit solution. By decoupling the spatial frequencies we are able to reduce an infinite dimensional LQR to an infinte family of two two dimensional LQRs each of which can be solved explicitly