6,034 research outputs found
Approximate Dynamic Programming via Sum of Squares Programming
We describe an approximate dynamic programming method for stochastic control
problems on infinite state and input spaces. The optimal value function is
approximated by a linear combination of basis functions with coefficients as
decision variables. By relaxing the Bellman equation to an inequality, one
obtains a linear program in the basis coefficients with an infinite set of
constraints. We show that a recently introduced method, which obtains convex
quadratic value function approximations, can be extended to higher order
polynomial approximations via sum of squares programming techniques. An
approximate value function can then be computed offline by solving a
semidefinite program, without having to sample the infinite constraint. The
policy is evaluated online by solving a polynomial optimization problem, which
also turns out to be convex in some cases. We experimentally validate the
method on an autonomous helicopter testbed using a 10-dimensional helicopter
model.Comment: 7 pages, 5 figures. Submitted to the 2013 European Control
Conference, Zurich, Switzerlan
Al'brekht's Method in Infinite Dimensions
In 1961 E. G. Albrekht presented a method for the optimal stabilization of smooth, nonlinear, finite dimensional, continuous time control systems. This method has been extended to similar systems in discrete time and to some stochastic systems in continuous and discrete time. In this paper we extend Albrekht's method to the optimal stabilization of some smooth, nonlinear, infinite dimensional, continuous time control systems whose nonlinearities are described by Fredholm integral operators
Stochastic HJB Equations and Regular Singular Points
IIn this paper we show that some HJB equations arising from both finite and
infinite horizon stochastic optimal control problems have a regular singular
point at the origin. This makes them amenable to solution by power series
techniques. This extends the work of Al'brecht who showed that the HJB
equations of an infinite horizon deterministic optimal control problem can have
a regular singular point at the origin, Al'brekht solved the HJB equations by
power series, degree by degree. In particular, we show that the infinite
horizon stochastic optimal control problem with linear dynamics, quadratic cost
and bilinear noise leads to a new type of algebraic Riccati equation which we
call the Stochastic Algebraic Riccati Equation (SARE). If SARE can be solved
then one has a complete solution to this infinite horizon stochastic optimal
control problem. We also show that a finite horizon stochastic optimal control
problem with linear dynamics, quadratic cost and bilinear noise leads to a
Stochastic Differential Riccati Equation (SDRE) that is well known. If these
problems are the linear-quadratic-bilinear part of a nonlinear finite horizon
stochastic optimal control problem then we show how the higher degree terms of
the solutions can be computed degree by degree. To our knowledge this
computation is new
Series Solution of Discrete Time Stochastic Optimal Control Problems
In this paper we consider discrete time stochastic optimal control problems
over infinite and finite time horizons. We show that for a large class of such
problems the Taylor polynomials of the solutions to the associated Dynamic
Programming Equations can be computed degree by degree.Comment: arXiv admin note: text overlap with arXiv:1806.0412
Finite-time behavior of inner systems
In this paper, we investigate how nonminimum phase characteristics of a dynamical system affect its controllability and tracking properties. For the class of linear time-invariant dynamical systems, these characteristics are determined by transmission zeros of the inner factor of the system transfer function. The relation between nonminimum phase zeros and Hankel singular values of inner systems is studied and it is shown how the singular value structure of a suitably defined operator provides relevant insight about system invertibility and achievable tracking performance. The results are used to solve various tracking problems both on finite as well as on infinite time horizons. A typical receding horizon control scheme is considered and new conditions are derived to guarantee stabilizability of a receding horizon controller
Optimal control of discrete-time switched linear systems via continuous parameterization
The paper presents a novel method for designing an optimal controller for
discrete-time switched linear systems. The problem is formulated as one of
computing the discrete mode sequence and the continuous input sequence that
jointly minimize a quadratic performance index. State-of-art methods for
solving such a control problem suffer in general from a high computational
requirement due to the fact that an exponential number of switching sequences
must be explored. The method of this paper addresses the challenge of the
switching law design by introducing auxiliary continuous input variables and
then solving a non-smooth block-sparsity inducing optimization problem.Comment: 6 pages, 2 figures, 2 tables; To appear in the Proceedings of IFAC
World Congress, 201
- …