400 research outputs found
Data-driven Economic NMPC using Reinforcement Learning
Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal
control without relying on a model of the system. However, RL struggles to
provide hard guarantees on the behavior of the resulting control scheme. In
contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC)
are standard tools for the closed-loop optimal control of complex systems with
constraints and limitations, and benefit from a rich theory to assess their
closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the
quality of the model underlying the control scheme. In this paper, we show that
an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system
even when using a wrong model. This result also holds for real systems having
stochastic dynamics. This entails that ENMPC can be used as a new type of
function approximator within RL. Furthermore, we investigate our results in the
context of ENMPC and formally connect them to the concept of dissipativity,
which is central for the ENMPC stability. Finally, we detail how these results
can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply
these tools on both a classical linear MPC setting and a standard nonlinear
example from the ENMPC literature
Global Convergence of Policy Gradient Primal-dual Methods for Risk-constrained LQRs
While the techniques in optimal control theory are often model-based, the
policy optimization (PO) approach can directly optimize the performance metric
of interest without explicit dynamical models, and is an essential approach for
reinforcement learning problems. However, it usually leads to a non-convex
optimization problem in most cases, where there is little theoretical
understanding on its performance. In this paper, we focus on the
risk-constrained Linear Quadratic Regulator (LQR) problem with noisy input via
the PO approach, which results in a challenging non-convex problem. To this
end, we first build on our earlier result that the optimal policy has an affine
structure to show that the associated Lagrangian function is locally gradient
dominated with respect to the policy, based on which we establish strong
duality. Then, we design policy gradient primal-dual methods with global
convergence guarantees to find an optimal policy-multiplier pair in both
model-based and sample-based settings. Finally, we use samples of system
trajectories in simulations to validate our policy gradient primal-dual
methods
A Parallel Dual Fast Gradient Method for MPC Applications
We propose a parallel adaptive constraint-tightening approach to solve a
linear model predictive control problem for discrete-time systems, based on
inexact numerical optimization algorithms and operator splitting methods. The
underlying algorithm first splits the original problem in as many independent
subproblems as the length of the prediction horizon. Then, our algorithm
computes a solution for these subproblems in parallel by exploiting auxiliary
tightened subproblems in order to certify the control law in terms of
suboptimality and recursive feasibility, along with closed-loop stability of
the controlled system. Compared to prior approaches based on constraint
tightening, our algorithm computes the tightening parameter for each subproblem
to handle the propagation of errors introduced by the parallelization of the
original problem. Our simulations show the computational benefits of the
parallelization with positive impacts on performance and numerical conditioning
when compared with a recent nonparallel adaptive tightening scheme.Comment: This technical report is an extended version of the paper "A Parallel
Dual Fast Gradient Method for MPC Applications" by the same authors submitted
to the 54th IEEE Conference on Decision and Contro
Distributed Design for Decentralized Control using Chordal Decomposition and ADMM
We propose a distributed design method for decentralized control by
exploiting the underlying sparsity properties of the problem. Our method is
based on chordal decomposition of sparse block matrices and the alternating
direction method of multipliers (ADMM). We first apply a classical
parameterization technique to restrict the optimal decentralized control into a
convex problem that inherits the sparsity pattern of the original problem. The
parameterization relies on a notion of strongly decentralized stabilization,
and sufficient conditions are discussed to guarantee this notion. Then, chordal
decomposition allows us to decompose the convex restriction into a problem with
partially coupled constraints, and the framework of ADMM enables us to solve
the decomposed problem in a distributed fashion. Consequently, the subsystems
only need to share their model data with their direct neighbours, not needing a
central computation. Numerical experiments demonstrate the effectiveness of the
proposed method.Comment: 11 pages, 8 figures. Accepted for publication in the IEEE
Transactions on Control of Network System
TinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers
Model-predictive control (MPC) is a powerful tool for controlling highly
dynamic robotic systems subject to complex constraints. However, MPC is
computationally demanding, and is often impractical to implement on small,
resource-constrained robotic platforms. We present TinyMPC, a high-speed MPC
solver with a low memory footprint targeting the microcontrollers common on
small robots. Our approach is based on the alternating direction method of
multipliers (ADMM) and leverages the structure of the MPC problem for
efficiency. We demonstrate TinyMPC both by benchmarking against the
state-of-the-art solver OSQP, achieving nearly an order of magnitude speed
increase, as well as through hardware experiments on a 27 g quadrotor,
demonstrating high-speed trajectory tracking and dynamic obstacle avoidance.Comment: First three authors contributed equally and are ordered
alphabeticall
- …