Learning Event-triggered Control from Data through Joint Optimization

We present a framework for model-free learning of event-triggered control strategies. Event-triggered methods aim to achieve high control performance while only closing the feedback loop when needed. This enables resource savings, e.g., network bandwidth if control commands are sent via communication networks, as in networked control systems. Event-triggered controllers consist of a communication policy, determining when to communicate, and a control policy, deciding what to communicate. It is essential to jointly optimize the two policies since individual optimization does not necessarily yield the overall optimal solution. To address this need for joint optimization, we propose a novel algorithm based on hierarchical reinforcement learning. The resulting algorithm is shown to accomplish high-performance control in line with resource savings and scales seamlessly to nonlinear and high-dimensional systems. The method's applicability to real-world scenarios is demonstrated through experiments on a six degrees of freedom real-time controlled manipulator. Further, we propose an approach towards evaluating the stability of the learned neural network policies

Lazy global feedbacks for quantized nonlinear event systems

We consider nonlinear event systems with quantized state information and design a globally stabilizing controller from which only the minimal required number of control value changes along the feedback trajectory to a given initial condition is transmitted to the plant. In addition, we present a non-optimal heuristic approach which might reduce the number of control value changes and requires a lower computational effort. The constructions are illustrated by two numerical examples

Cross-Layer Adaptive Feedback Scheduling of Wireless Control Systems

There is a trend towards using wireless technologies in networked control systems. However, the adverse properties of the radio channels make it difficult to design and implement control systems in wireless environments. To attack the uncertainty in available communication resources in wireless control systems closed over WLAN, a cross-layer adaptive feedback scheduling (CLAFS) scheme is developed, which takes advantage of the co-design of control and wireless communications. By exploiting cross-layer design, CLAFS adjusts the sampling periods of control systems at the application layer based on information about deadline miss ratio and transmission rate from the physical layer. Within the framework of feedback scheduling, the control performance is maximized through controlling the deadline miss ratio. Key design parameters of the feedback scheduler are adapted to dynamic changes in the channel condition. An event-driven invocation mechanism for the feedback scheduler is also developed. Simulation results show that the proposed approach is efficient in dealing with channel capacity variations and noise interference, thus providing an enabling technology for control over WLAN.Comment: 17 pages, 12 figures; Open Access at http://www.mdpi.org/sensors/papers/s8074265.pd

Deep Reinforcement Learning for Event-Triggered Control

Event-triggered control (ETC) methods can achieve high-performance control with a significantly lower number of samples compared to usual, time-triggered methods. These frameworks are often based on a mathematical model of the system and specific designs of controller and event trigger. In this paper, we show how deep reinforcement learning (DRL) algorithms can be leveraged to simultaneously learn control and communication behavior from scratch, and present a DRL approach that is particularly suitable for ETC. To our knowledge, this is the first work to apply DRL to ETC. We validate the approach on multiple control tasks and compare it to model-based event-triggering frameworks. In particular, we demonstrate that it can, other than many model-based ETC designs, be straightforwardly applied to nonlinear systems

Feedback and time are essential for the optimal control of computing systems

The performance, reliability, cost, size and energy usage of computing systems can be improved by one or more orders of magnitude by the systematic use of modern control and optimization methods. Computing systems rely on the use of feedback algorithms to schedule tasks, data and resources, but the models that are used to design these algorithms are validated using open-loop metrics. By using closed-loop metrics instead, such as the gap metric developed in the control community, it should be possible to develop improved scheduling algorithms and computing systems that have not been over-engineered. Furthermore, scheduling problems are most naturally formulated as constraint satisfaction or mathematical optimization problems, but these are seldom implemented using state of the art numerical methods, nor do they explicitly take into account the fact that the scheduling problem itself takes time to solve. This paper makes the case that recent results in real-time model predictive control, where optimization problems are solved in order to control a process that evolves in time, are likely to form the basis of scheduling algorithms of the future. We therefore outline some of the research problems and opportunities that could arise by explicitly considering feedback and time when designing optimal scheduling algorithms for computing systems

Manufacturing systems considered as time domain control systems : receding horizon control and observers

This thesis considers manufacturing systems and model-based controller design, as well as their combinations. The objective of a manufacturing system is to create products from a selected group of raw materials and semifinished goods. In the field of manufacturing systems control is an important issue appearing at various operation levels. At the level of fabrication, for example, control is necessary in order to assure properly working production processes such that products are being fabricated in the desired way. At a higher level in the hierarchy of manufacturing system control, the product streams through the system are controlled in order to satisfy, for example, customer demands in an optimal way. Here, the definition of optimal can be interpreted in various ways, such as "with the least possible costs in terms of money" or "in the shortest possible time". In this research, the attention is focussed on this higher hierarchy level of manufacturing system control. In the literature, many heuristic methods have been developed for the control of a manufacturing system. Nowadays, some heuristicmethods are still being used in combination with operator experience for management of resources and planning of production. However, as the complexity of the manufacturing systems increases rapidly, the (simple) heuristic methods and operator experience will at some point become incapable of finding an optimal control strategy. In this dissertation the potential of consideringmanufacturing system control from a control systems point of view is investigated. The ultimate goal of the research is to eventually obtain a more constructive way to address controller design for manufacturing systems. One control strategy from control systems theory, on which is in particularly focused in this research, is a model-based receding horizon control strategy, known in literature as Model Predictive Control (MPC). Since in manufacturing systems a lot of physical system constraints are involved, like for example finite machine process capacities, finite product storage capacities, finite product arrival rates, etc., the capability for a manufacturing control strategy to handle those constraints is a necessity. One of the key features of model predictive control is the capability of handling constraints in the controller design. This is one of the major motivations to investigate the model predictive control principle as a control strategy for manufacturing systems. Other issues that are important and that the model predictive control design methodology can handle is to enforce optimality, to introduce feedback, and the capability of allowing for mixed continuous and discrete model structures. The later are typically encountered when models of manufacturing systems are derived. The main results that are obtained in this dissertation and that are relevant in the context of manufacturing systems control, but are certainly also relevant beyond this field are: • One has developed an robust computationally friendly nonlinear model predictive control algorithm that can handle model structures with mixed continuous and discrete dynamics. The algorithm can be designed for additive disturbance rejection purposes; • Robustness (with respect to measurement noise) results that are in particulary of interest in the field of nonlinear model predictive control are obtained; • An asymptotically stabilizing output based nonlinear model predictive control scheme for a class of nonlinear discrete-time systems is developed. Results that are relevant in the context of manufacturing systems control are: • It is illustrated howthe aforementioned developed robust computationally friendly nonlinear model predictive control algorithm can be employed to solve a large scale manufacturing control problem in an efficient decentralized manner; • The relation between the so-called event domain modeling approaches for a class of discrete-eventmanufacturing systems to time domainmodels is derived. This results enables one to solve seemingly untractable time domain formulated optimal control problems for a class of manufacturing systems in a tractable manner; • An observer theory for a class of discrete-event manufacturing systems is developed