Sparse Wide-Area Control of Power Systems using Data-driven Reinforcement Learning

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems.Comment: Submitted to IEEE ACC 2019. 8 pages, 4 figure

Active Sampling-based Binary Verification of Dynamical Systems

Nonlinear, adaptive, or otherwise complex control techniques are increasingly relied upon to ensure the safety of systems operating in uncertain environments. However, the nonlinearity of the resulting closed-loop system complicates verification that the system does in fact satisfy those requirements at all possible operating conditions. While analytical proof-based techniques and finite abstractions can be used to provably verify the closed-loop system's response at different operating conditions, they often produce conservative approximations due to restrictive assumptions and are difficult to construct in many applications. In contrast, popular statistical verification techniques relax the restrictions and instead rely upon simulations to construct statistical or probabilistic guarantees. This work presents a data-driven statistical verification procedure that instead constructs statistical learning models from simulated training data to separate the set of possible perturbations into "safe" and "unsafe" subsets. Binary evaluations of closed-loop system requirement satisfaction at various realizations of the uncertainties are obtained through temporal logic robustness metrics, which are then used to construct predictive models of requirement satisfaction over the full set of possible uncertainties. As the accuracy of these predictive statistical models is inherently coupled to the quality of the training data, an active learning algorithm selects additional sample points in order to maximize the expected change in the data-driven model and thus, indirectly, minimize the prediction error. Various case studies demonstrate the closed-loop verification procedure and highlight improvements in prediction error over both existing analytical and statistical verification techniques.Comment: 23 page

A probabilistic approach to model-based adaptive control for damping of interarea oscillations

Published versio

Learning and Management for Internet-of-Things: Accounting for Adaptivity and Scalability

Internet-of-Things (IoT) envisions an intelligent infrastructure of networked smart devices offering task-specific monitoring and control services. The unique features of IoT include extreme heterogeneity, massive number of devices, and unpredictable dynamics partially due to human interaction. These call for foundational innovations in network design and management. Ideally, it should allow efficient adaptation to changing environments, and low-cost implementation scalable to massive number of devices, subject to stringent latency constraints. To this end, the overarching goal of this paper is to outline a unified framework for online learning and management policies in IoT through joint advances in communication, networking, learning, and optimization. From the network architecture vantage point, the unified framework leverages a promising fog architecture that enables smart devices to have proximity access to cloud functionalities at the network edge, along the cloud-to-things continuum. From the algorithmic perspective, key innovations target online approaches adaptive to different degrees of nonstationarity in IoT dynamics, and their scalable model-free implementation under limited feedback that motivates blind or bandit approaches. The proposed framework aspires to offer a stepping stone that leads to systematic designs and analysis of task-specific learning and management schemes for IoT, along with a host of new research directions to build on.Comment: Submitted on June 15 to Proceeding of IEEE Special Issue on Adaptive and Scalable Communication Network

On Using a Support Vector Machine in Learning Feed-Forward Control

For mechatronic motion systems, the performance increases significantly if, besides feedback control, also feed-forward control is used. This feed-forward part should contain the (stable part of the) inverse of the plant. This inverse is difficult to obtain if non-linear dynamics are present. To overcome this problem, learning feed-forward control can be applied. The properties of the learning mechanism are of importance in this setting. In the paper, a support vector machine is proposed as the learning mechanism. It is shown that this mechanism has several advantages over other learning techniques when applied to learning feed-forward control. The method is tested with simulation

A Survey of Prediction and Classification Techniques in Multicore Processor Systems

In multicore processor systems, being able to accurately predict the future provides new optimization opportunities, which otherwise could not be exploited. For example, an oracle able to predict a certain application\u27s behavior running on a smart phone could direct the power manager to switch to appropriate dynamic voltage and frequency scaling modes that would guarantee minimum levels of desired performance while saving energy consumption and thereby prolonging battery life. Using predictions enables systems to become proactive rather than continue to operate in a reactive manner. This prediction-based proactive approach has become increasingly popular in the design and optimization of integrated circuits and of multicore processor systems. Prediction transforms from simple forecasting to sophisticated machine learning based prediction and classification that learns from existing data, employs data mining, and predicts future behavior. This can be exploited by novel optimization techniques that can span across all layers of the computing stack. In this survey paper, we present a discussion of the most popular techniques on prediction and classification in the general context of computing systems with emphasis on multicore processors. The paper is far from comprehensive, but, it will help the reader interested in employing prediction in optimization of multicore processor systems

Classifying intelligence in machines : a taxonomy of intelligent control

The quest to create machines that can solve problems as humans do leads us to intelligent control. This field encompasses control systems that can adapt to changes and learn to improve their actions—traits typically associated with human intelligence. In this work we seek to determine how intelligent these classes of control systems are by quantifying their level of adaptability and learning. First we describe the stages of development towards intelligent control and present a definition based on literature. Based on the key elements of this definition, we propose a novel taxonomy of intelligent control methods, which assesses the extent to which they handle uncertainties in three areas: the environment, the controller, and the goals. This taxonomy is applicable to a variety of robotic and other autonomous systems, which we demonstrate through several examples of intelligent control methods and their classifications. Looking at the spread of classifications based on this taxonomy can help researchers identify where control systems can be made more intelligent

A MRAS-based Learning Feed-forward Controller

Inspired by learning feed–forward control structures, this paper considers the adaptation of the parameters of a model–reference based learning feed–forward controller that realizes an inverse model of the process. The actual process response is determined by a setpoint generator. For linear systems it can be proved that the controlled system is asymptotically stable in the sense of Liapunov. Compared with more standard model reference configurations this system has a superior performance. It is fast, robust and relatively insensitive for noisy measurements. Simulations with an arbitrary second–order process and with a model of a typical fourth–ordermechatronics process demonstrate this