2 research outputs found
On the Convergence of the Iterative Linear Exponential Quadratic Gaussian Algorithm to Stationary Points
A classical method for risk-sensitive nonlinear control is the iterative
linear exponential quadratic Gaussian algorithm. We present its convergence
analysis from a first-order optimization viewpoint. We identify the objective
that the algorithm actually minimizes and we show how the addition of a
proximal term guarantees convergence to a stationary point
Primal-dual Learning for the Model-free Risk-constrained Linear Quadratic Regulator
Risk-aware control, though with promise to tackle unexpected events, requires
a known exact dynamical model. In this work, we propose a model-free framework
to learn a risk-aware controller with a focus on the linear system. We
formulate it as a discrete-time infinite-horizon LQR problem with a state
predictive variance constraint. To solve it, we parameterize the policy with a
feedback gain pair and leverage primal-dual methods to optimize it by solely
using data. We first study the optimization landscape of the Lagrangian
function and establish the strong duality in spite of its non-convex nature.
Alongside, we find that the Lagrangian function enjoys an important local
gradient dominance property, which is then exploited to develop a convergent
random search algorithm to learn the dual function. Furthermore, we propose a
primal-dual algorithm with global convergence to learn the optimal
policy-multiplier pair. Finally, we validate our results via simulations.Comment: To appear in the Annual Conference on Learning for Dynamics and
Control (L4DC) 202