Data-driven Economic NMPC using Reinforcement Learning

Reinforcement Learning (RL) is a powerful tool to perform data-driven optimal control without relying on a model of the system. However, RL struggles to provide hard guarantees on the behavior of the resulting control scheme. In contrast, Nonlinear Model Predictive Control (NMPC) and Economic NMPC (ENMPC) are standard tools for the closed-loop optimal control of complex systems with constraints and limitations, and benefit from a rich theory to assess their closed-loop behavior. Unfortunately, the performance of (E)NMPC hinges on the quality of the model underlying the control scheme. In this paper, we show that an (E)NMPC scheme can be tuned to deliver the optimal policy of the real system even when using a wrong model. This result also holds for real systems having stochastic dynamics. This entails that ENMPC can be used as a new type of function approximator within RL. Furthermore, we investigate our results in the context of ENMPC and formally connect them to the concept of dissipativity, which is central for the ENMPC stability. Finally, we detail how these results can be used to deploy classic RL tools for tuning (E)NMPC schemes. We apply these tools on both a classical linear MPC setting and a standard nonlinear example from the ENMPC literature

Learning unification-based grammars using the Spoken English Corpus

This paper describes a grammar learning system that combines model-based and data-driven learning within a single framework. Our results from learning grammars using the Spoken English Corpus (SEC) suggest that combined model-based and data-driven learning can produce a more plausible grammar than is the case when using either learning style isolation.Comment: 10 page

A model of factors affecting independent learners’ engagement with feedback on language learning tasks

In independent learning contexts, the effectiveness of the feedback dialogue between student and tutor or, in the absence of a tutor, the quality of the learning materials, is essential to successful learning. Using the voices of participants as the prime source of data through a combination of data-driven and concept-driven approaches, this investigation attempts to gain deeper insights into the dynamics of the learning process as students express emotional reactions to the learning environment and in particular the written feedback from their tutors and the learning materials. To account for the different ways in which adult learners studying independently engage both cognitively and emotionally with external feedback, we propose a model based on four key drivers: goal relevance, knowledge, self-confidence, and roles. We conclude that only when these key drivers are aligned with each other can learners in independent settings engage with external feedback and learn from it