1,134 research outputs found
Log-Concave Duality in Estimation and Control
In this paper we generalize the estimation-control duality that exists in the
linear-quadratic-Gaussian setting. We extend this duality to maximum a
posteriori estimation of the system's state, where the measurement and
dynamical system noise are independent log-concave random variables. More
generally, we show that a problem which induces a convex penalty on noise terms
will have a dual control problem. We provide conditions for strong duality to
hold, and then prove relaxed conditions for the piecewise linear-quadratic
case. The results have applications in estimation problems with nonsmooth
densities, such as log-concave maximum likelihood densities. We conclude with
an example reconstructing optimal estimates from solutions to the dual control
problem, which has implications for sharing solution methods between the two
types of problems
Controller Synthesis for Discrete-Time Polynomial Systems via Occupation Measures
In this paper, we design nonlinear state feedback controllers for
discrete-time polynomial dynamical systems via the occupation measure approach.
We propose the discrete-time controlled Liouville equation, and use it to
formulate the controller synthesis problem as an infinite-dimensional linear
programming problem on measures, which is then relaxed as finite-dimensional
semidefinite programming problems on moments of measures and their duals on
sums-of-squares polynomials. Nonlinear controllers can be extracted from the
solutions to the relaxed problems. The advantage of the occupation measure
approach is that we solve convex problems instead of generally non-convex
problems, and the computational complexity is polynomial in the state and input
dimensions, and hence the approach is more scalable. In addition, we show that
the approach can be applied to over-approximating the backward reachable set of
discrete-time autonomous polynomial systems and the controllable set of
discrete-time polynomial systems under known state feedback control laws. We
illustrate our approach on several dynamical systems
Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRs
While the techniques in optimal control theory are often model-based, the
policy optimization (PO) approach can directly optimize the performance metric
of interest without explicit dynamical models, and is an essential approach for
reinforcement learning problems. However, it usually leads to a non-convex
optimization problem in most cases, where there is little theoretical
understanding on its performance. In this paper, we focus on the
risk-constrained Linear Quadratic Regulator (LQR) problem with noisy input via
the PO approach, which results in a challenging non-convex problem. To this
end, we first build on our earlier result that the optimal policy has an affine
structure to show that the associated Lagrangian function is locally gradient
dominated with respect to the policy, based on which we establish strong
duality. Then, we design policy gradient primal-dual methods with global
convergence guarantees to find an optimal policy-multiplier pair in both
model-based and sample-based settings. Finally, we use samples of system
trajectories in simulations to validate our policy gradient primal-dual
methods
- …