Data-driven Economic NMPC using Reinforcement Learning

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closed-loop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature

An exponential turnpike theorem for dissipative discrete time optimal control problems

revised 2013, 23 p.International audienceWe investigate the exponential turnpike property for nite horizon undercounted discrete time optimal control problems without any terminal constraints. Considering a class of strictly dissipative systems we derive a boundedness condition for an auxiliary optimal value function which implies the exponential turnpike property. Two theorems illustrate how this boundedness condition can be concluded from structural properties like controllability and stabilizability of the control system under consideration

A general dissipativity constraint for feedback system design, with emphasis on MPC

A ‘General Dissipativity Constraint’ (GDC) is introduced to facilitate the design of stable feedback systems. A primary application is to MPC controllers when it is preferred to avoid the use of ‘stabilising ingredients’ such as terminal constraint sets or long prediction horizons. Some very general convergence results are proved under mild conditions. The use of quadratic functions, replacing GDC by ‘Quadratic Dissipation Constraint’ (QDC), is introduced to allow implementation using linear matrix inequalities. The use of QDC is illustrated for several scenarios: state feedback for a linear time-invariant system, MPC of a linear system, MPC of an input-affine system, and MPC with persistent disturbances. The stability that is guaranteed by GDC is weaker than Lyapunov stability, being ‘Lagrange stability plus convergence’. Input-to-state stability is obtained if the control law is continuous in the state. An example involving an open-loop unstable helicopter illustrates the efficacy of the approach in practice.National Research Foundation Singapor

Multiplexed model predictive control of interconnected systems

A Multiplexed Model Predictive Control (MMPC) scheme with Quadratic Dissipativity Constraint (QDC) for interconnected systems is presented in this paper. A centralized MMPC is designed for the global system, wherein the controls of subsystems are updated sequentially to reduce the computational time. In MMPC, the global state vector of the interconnected system is required by the optimization. The QDC is converted into an enforced stability constraint for the MMPC as an alternative to the terminal constraint and terminal cost in this approach. The nominal recursive feasibility for the global system and the iterative feasibility for the local subsystems are obtained via set operations on the invariant sets. The admissible sets for the control inputs are obtained and employed in this approach for the QDC-based stability constraint. The set operations are speed up by multiple magnitudes thanks to the implementation of multiplexed inputs in MMPC. Numerical simulations with Automatic Generation Control (AGC) in power systems having tie-lines demonstrate the theoretical development.The authors acknowledge the support by the Singapore National Research Foundation (NRF) under its Campus for Research Excellence And Technological Enterprise (CREATE) programme and the Cambridge Centre for Advanced Research in Energy Efficiency in Singapore (Cambridge CARES), C4T project.This is the author accepted manuscript. The final version is available from IEEE via http://dx.doi.org/10.1109/CDC.2015.740256