Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Optimal predictive control of water transport systems: Arrêt-Darré/Arros case study

This paper proposes the use of predictive optimal control as a suitable methodology to manage efficiently transport water networks. The predictive optimal controller is implemented using MPC control techniques. The Arrêt-Darré/Arros dam-river system located in the Southwest region of France is proposed as case study. A high-fidelity dynamic simulator based on the full Saint-Venant equations and able to reproduce this system is developed in MATLAB/SIMULINK to validate the performance of the developed predictive optimal control system. The control objective in the Arrêt-Darré/Arros dam-river system is to guarantee an ecological flow rate at a control point downstream of the Arrêt-Darré dam by controlling the outflow of this dam in spite of the unmeasured disturbances introduced by rainfalls incomings and farmer withdrawals

Deep Reinforcement Learning for Event-Triggered Control

Event-triggered control (ETC) methods can achieve high-performance control with a significantly lower number of samples compared to usual, time-triggered methods. These frameworks are often based on a mathematical model of the system and specific designs of controller and event trigger. In this paper, we show how deep reinforcement learning (DRL) algorithms can be leveraged to simultaneously learn control and communication behavior from scratch, and present a DRL approach that is particularly suitable for ETC. To our knowledge, this is the first work to apply DRL to ETC. We validate the approach on multiple control tasks and compare it to model-based event-triggering frameworks. In particular, we demonstrate that it can, other than many model-based ETC designs, be straightforwardly applied to nonlinear systems