Event-triggered Learning for Resource-efficient Networked Control

Common event-triggered state estimation (ETSE) algorithms save communication in networked control systems by predicting agents' behavior, and transmitting updates only when the predictions deviate significantly. The effectiveness in reducing communication thus heavily depends on the quality of the dynamics models used to predict the agents' states or measurements. Event-triggered learning is proposed herein as a novel concept to further reduce communication: whenever poor communication performance is detected, an identification experiment is triggered and an improved prediction model learned from data. Effective learning triggers are obtained by comparing the actual communication rate with the one that is expected based on the current model. By analyzing statistical properties of the inter-communication times and leveraging powerful convergence results, the proposed trigger is proven to limit learning experiments to the necessary instants. Numerical and physical experiments demonstrate that event-triggered learning improves robustness toward changing environments and yields lower communication rates than common ETSE.Comment: 7 pages, 4 figures, to appear in the 2018 American Control Conference (ACC

Event-triggered Pulse Control with Model Learning (if Necessary)

In networked control systems, communication is a shared and therefore scarce resource. Event-triggered control (ETC) can achieve high performance control with a significantly reduced amount of samples compared to classical, periodic control schemes. However, ETC methods usually rely on the availability of an accurate dynamics model, which is oftentimes not readily available. In this paper, we propose a novel event-triggered pulse control strategy that learns dynamics models if necessary. In addition to adapting to changing dynamics, the method also represents a suitable replacement for the integral part typically used in periodic control.Comment: Accepted final version to appear in: Proc. of the American Control Conference, 201

Deep Reinforcement Learning for Event-Triggered Control

Event-triggered control (ETC) methods can achieve high-performance control with a significantly lower number of samples compared to usual, time-triggered methods. These frameworks are often based on a mathematical model of the system and specific designs of controller and event trigger. In this paper, we show how deep reinforcement learning (DRL) algorithms can be leveraged to simultaneously learn control and communication behavior from scratch, and present a DRL approach that is particularly suitable for ETC. To our knowledge, this is the first work to apply DRL to ETC. We validate the approach on multiple control tasks and compare it to model-based event-triggering frameworks. In particular, we demonstrate that it can, other than many model-based ETC designs, be straightforwardly applied to nonlinear systems

Event-triggered Learning

The efficient exchange of information is an essential aspect of intelligent collective behavior. Event-triggered control and estimation achieve some efficiency by replacing continuous data exchange between agents with intermittent, or event-triggered communication. Typically, model-based predictions are used at times of no data transmission, and updates are sent only when the prediction error grows too large. The effectiveness in reducing communication thus strongly depends on the quality of the prediction model. In this article, we propose event-triggered learning as a novel concept to reduce communication even further and to also adapt to changing dynamics. By monitoring the actual communication rate and comparing it to the one that is induced by the model, we detect a mismatch between model and reality and trigger model learning when needed. Specifically, for linear Gaussian dynamics, we derive different classes of learning triggers solely based on a statistical analysis of inter-communication times and formally prove their effectiveness with the aid of concentration inequalities

Linear Regression over Networks with Communication Guarantees

A key functionality of emerging connected autonomous systems such as smart cities, smart transportation systems, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations. This is increasingly attracting attention under the terms of distributed learning and federated learning. However, in connected autonomous systems, data transfer takes place over communication networks with often limited resources. This paper examines algorithms for communication-efficient learning for linear regression tasks by exploiting the informativeness of the data. The developed algorithms enable a tradeoff between communication and learning with theoretical performance guarantees and efficient practical implementations.Comment: Accepted at 3rd Annual Learning for Dynamics & Control Conference (L4DC) 2021. arXiv admin note: substantial text overlap with arXiv:2101.1000

Hierarchical Event-triggered Learning for Cyclically Excited Systems with Application to Wireless Sensor Networks

Communication load is a limiting factor in many real-time systems. Event-triggered state estimation and event-triggered learning methods reduce network communication by sending information only when it cannot be adequately predicted based on previously transmitted data. This paper proposes an event-triggered learning approach for nonlinear discrete-time systems with cyclic excitation. The method automatically recognizes cyclic patterns in data - even when they change repeatedly - and reduces communication load whenever the current data can be accurately predicted from previous cycles. Nonetheless, a bounded error between original and received signal is guaranteed. The cyclic excitation model, which is used for predictions, is updated hierarchically, i.e., a full model update is only performed if updating a small number of model parameters is not sufficient. A nonparametric statistical test enforces that model updates happen only if the cyclic excitation changed with high probability. The effectiveness of the proposed methods is demonstrated using the application example of wireless real-time pitch angle measurements of a human foot in a feedback-controlled neuroprosthesis. The experimental results show that communication load can be reduced by 70 % while the root-mean-square error between measured and received angle is less than 1{\deg}.Comment: 6 pages and 6 figures; to appear in IEEE Control Systems Letter

Event-triggered Learning for Linear Quadratic Control

When models are inaccurate, the performance of model-based control will degrade. For linear quadratic control, an event-triggered learning framework is proposed that automatically detects inaccurate models and triggers the learning of a new process model when needed. This is achieved by analyzing the probability distribution of the linear quadratic cost and designing a learning trigger that leverages Chernoff bounds. In particular, whenever empirically observed cost signals are located outside the derived confidence intervals, we can provably guarantee that this is with high probability due to a model mismatch. With the aid of numerical and hardware experiments, we demonstrate that the proposed bounds are tight and that the event-triggered learning algorithm effectively distinguishes between inaccurate models and probabilistic effects such as process noise. Thus, a structured approach is obtained that decides when model learning is beneficial.Comment: 13 pages, 8 figures, accepted for publication in IEEE Transactions on Automatic Contro