Data-driven Economic NMPC using Reinforcement Learning

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closed-loop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature

Technical Report: Control of Nonlinear Systems with Explicit-MPC-like Controllers

This paper describes synthesis of controllers involving Quadratic Programming (QP) optimization problems for control of nonlinear systems. The QP structure allows an implementation of the controller as a piecewise affine function, pre-computed offline, which is a technique extensively studied in the field of explicit model predictive control (EMPC). The nonlinear systems being controlled are assumed to be described by polynomial functions and the synthesis also generates a polynomial Lyapunov function for the closed-loop system involving the obtained controller. The synthesis is based on a sum-of-squares (SOS) stability verification for polynomial discrete-time systems, described in continuous-time in this paper. The presented synthesis method allows a design of EMPC controllers with closed-loop stability guarantees without relying on a terminal cost and/or constraint, and even without using the prediction horizon concept to formulate the control optimization problem. In particular, for a specified QP structure the method directly searches for the stabilizing coefficients in the cost and/or the constraint set. The method involves two phases, where the first searches for stabilizing controllers by minimizing a polynomial slack function introduced to the SOS stability condition and the second phase optimizes some user-specified performance criteria. The two phases are formulated as optimization problems which can be tackled by using a black-box optimization technique such as Bayesian optimization, which is used in this paper. The synthesis is demonstrated on a numerical example involving a bilinear model of a permanent magnet synchronous machine (PMSM), where in order to demonstrate the modeling flexibility of the proposed synthesis method a QP-based controller for speed regulation of PMSM is synthesized that is robust to parametric uncertainty coming from the temperature-dependent stator resistance of the PMSM

Stability analysis of Model Predictive Controllers using Mixed Integer Linear Programming

It is a well known fact that finite time optimal controllers, such as MPC do not necessarily result in closed loop stable systems. Within the MPC community it is common practice to add a final state constraint and/or a final state penalty in order to obtain guaranteed stability. However, for more advanced controller structures it can be difficult to show stability using these techniques. Additionally in some cases the final state constraint set consists of so many inequalities that the complexity of the MPC problem is too big for use in certain fast and time critical applications. In this paper we instead focus on deriving a tool for a-postiori analysis of the closed loop stability for linear systems controlled with MPC controllers. We formulate an optimisation problem that gives a sufficient condition for stability of the closed loop system and we show that the problem can be written as a Mixed Integer Linear Programming Problem (MILP).Funding agencies: Swedish Governmental Agency for Innovation Systems (VINNOVA); Centrum for indnstriell informationsteknologi (CENIIT)</p