Optimal Ensemble Control of Loads in Distribution Grids with Network Constraints

Flexible loads, e.g. thermostatically controlled loads (TCLs), are technically feasible to participate in demand response (DR) programs. On the other hand, there is a number of challenges that need to be resolved before it can be implemented in practice en masse. First, individual TCLs must be aggregated and operated in sync to scale DR benefits. Second, the uncertainty of TCLs needs to be accounted for. Third, exercising the flexibility of TCLs needs to be coordinated with distribution system operations to avoid unnecessary power losses and compliance with power flow and voltage limits. This paper addresses these challenges. We propose a network-constrained, open-loop, stochastic optimal control formulation. The first part of this formulation represents ensembles of collocated TCLs modelled by an aggregated Markov Process (MP), where each MP state is associated with a given power consumption or production level. The second part extends MPs to a multi-period distribution power flow optimization. In this optimization, the control of TCL ensembles is regulated by transition probability matrices and physically enabled by local active and reactive power controls at TCL locations. The optimization is solved with a Spatio-Temporal Dual Decomposition (ST-D2) algorithm. The performance of the proposed formulation and algorithm is demonstrated on the IEEE 33-bus distribution model.Comment: 7 pages, 6 figures, accepted PSCC 201

Smart Finite State Devices: A Modeling Framework for Demand Response Technologies

We introduce and analyze Markov Decision Process (MDP) machines to model individual devices which are expected to participate in future demand-response markets on distribution grids. We differentiate devices into the following four types: (a) optional loads that can be shed, e.g. light dimming; (b) deferrable loads that can be delayed, e.g. dishwashers; (c) controllable loads with inertia, e.g. thermostatically-controlled loads, whose task is to maintain an auxiliary characteristic (temperature) within pre-defined margins; and (d) storage devices that can alternate between charging and generating. Our analysis of the devices seeks to find their optimal price-taking control strategy under a given stochastic model of the distribution market.Comment: 8 pages, 8 figures, submitted IEEE CDC 201

Learning-based Predictive Control via Real-time Aggregate Flexibility

Aggregators have emerged as crucial tools for the coordination of distributed, controllable loads. To be used effectively, an aggregator must be able to communicate the available flexibility of the loads they control, as known as the aggregate flexibility to a system operator. However, most of existing aggregate flexibility measures often are slow-timescale estimations and much less attention has been paid to real-time coordination between an aggregator and an operator. In this paper, we consider solving an online optimization in a closed-loop system and present a design of real-time aggregate flexibility feedback, termed the maximum entropy feedback (MEF). In addition to deriving analytic properties of the MEF, combining learning and control, we show that it can be approximated using reinforcement learning and used as a penalty term in a novel control algorithm -- the penalized predictive control (PPC), which modifies vanilla model predictive control (MPC). The benefits of our scheme are (1). Efficient Communication. An operator running PPC does not need to know the exact states and constraints of the loads, but only the MEF. (2). Fast Computation. The PPC often has much less number of variables than an MPC formulation. (3). Lower Costs. We show that under certain regularity assumptions, the PPC is optimal. We illustrate the efficacy of the PPC using a dataset from an adaptive electric vehicle charging network and show that PPC outperforms classical MPC.Comment: 13 pages, 5 figures, extension of arXiv:2006.1381

Robust Engineering of Dynamic Structures in Complex Networks

Populations of nearly identical dynamical systems are ubiquitous in natural and engineered systems, in which each unit plays a crucial role in determining the functioning of the ensemble. Robust and optimal control of such large collections of dynamical units remains a grand challenge, especially, when these units interact and form a complex network. Motivated by compelling practical problems in power systems, neural engineering and quantum control, where individual units often have to work in tandem to achieve a desired dynamic behavior, e.g., maintaining synchronization of generators in a power grid or conveying information in a neuronal network; in this dissertation, we focus on developing novel analytical tools and optimal control policies for large-scale ensembles and networks. To this end, we first formulate and solve an optimal tracking control problem for bilinear systems. We developed an iterative algorithm that synthesizes the optimal control input by solving a sequence of state-dependent differential equations that characterize the optimal solution. This iterative scheme is then extended to treat isolated population or networked systems. We demonstrate the robustness and versatility of the iterative control algorithm through diverse applications from different fields, involving nuclear magnetic resonance (NMR) spectroscopy and imaging (MRI), electrochemistry, neuroscience, and neural engineering. For example, we design synchronization controls for optimal manipulation of spatiotemporal spike patterns in neuron ensembles. Such a task plays an important role in neural systems. Furthermore, we show that the formation of such spatiotemporal patterns is restricted when the network of neurons is only partially controllable. In neural circuitry, for instance, loss of controllability could imply loss of neural functions. In addition, we employ the phase reduction theory to leverage the development of novel control paradigms for cyclic deferrable loads, e.g., air conditioners, that are used to support grid stability through demand response (DR) programs. More importantly, we introduce novel theoretical tools for evaluating DR capacity and bandwidth. We also study pinning control of complex networks, where we establish a control-theoretic approach to identifying the most influential nodes in both undirected and directed complex networks. Such pinning strategies have extensive practical implications, e.g., identifying the most influential spreaders in epidemic and social networks, and lead to the discovery of degenerate networks, where the most influential node relocates depending on the coupling strength. This phenomenon had not been discovered until our recent study

Centralized and Decentralized Optimal Control of Variable Speed Heat Pumps

Utility service providers are often challenged with the synchronization of thermostatically controlled loads. Load synchronization, as a result of naturally occurring and demand-response events, has the potential to damage power distribution equipment. Because thermostatically controlled loads constitute most of the power consumed by the grid at any given time, the proper control of such devices can lead to significant energy savings and improved grid stability. The contribution of this paper is the development of an optimal control algorithm for commonly used variable speed heat pumps. By means of selective peer-to-peer communication, our control architecture allows for the regulation of home temperatures while simultaneously minimizing aggregate power consumption, and aggregate load volatility. An optimal centralized controller is also explored and compared against its decentralized counterpart

Fast and Reliable Primary Frequency Reserves From Refrigerators with Decentralized Stochastic Control

Due to increasing shares of renewable energy sources, more frequency reserves are required to maintain power system stability. In this paper, we present a decentralized control scheme that allows a large aggregation of refrigerators to provide Primary Frequency Control (PFC) reserves to the grid based on local frequency measurements and without communication. The control is based on stochastic switching of refrigerators depending on the frequency deviation. We develop methods to account for typical lockout constraints of compressors and increased power consumption during the startup phase. In addition, we propose a procedure to dynamically reset the thermostat temperature limits in order to provide reliable PFC reserves, as well as a corrective temperature feedback loop to build robustness to biased frequency deviations. Furthermore, we introduce an additional randomization layer in the controller to account for thermostat resolution limitations, and finally, we modify the control design to account for refrigerator door openings. Extensive simulations with actual frequency signal data and with different aggregation sizes, load characteristics, and control parameters, demonstrate that the proposed controller outperforms a relevant state-of-the-art controller.Comment: 44 pages, 17 figures, 9 Tables, submitted to IEEE Transactions on Power System

Demand side participation for frequency containment in the web of cells architecture

A large number of demand side management schemes have been proposed in literature for provision of frequency control ancillary services to the network. However, it is assumed that all the flexible devices within the network are managed and controlled under one demand side management (DSM) scheme. In this paper, two independent demand side management schemes control the portfolio of flexible devices within a web of cells architecture. A methodology and scenarios for analysis of the performance of more than one DSM scheme within the same network have been realized using a real-time power hardware-in-the-loop co-simulation platform, and the paper presents this as a basis for investigations of such arrangements

Optimal provision of distributed reserves under dynamic energy service preferences

We propose and solve a stochastic dynamic programming (DP) problem addressing the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic pricing work is that we pioneer the relaxation of static, uniformly distributed utility of demand. In this paper we model explicitly the dynamics of energy service preferences leading to a non-uniform and time varying probability distribution of demand utility. More explicitly, we model active and idle duty cycle appliances in a smart building as a closed queuing system with price-controlled arrival rates into the active appliance queue. Focusing on cooling appliances, we model the utility associated with the transition from idle to active as a non-uniform time varying function. We (i) derive an analytic characterization of the optimal policy and the differential cost function, and (ii) prove optimal policy monotonicity and value function convexity. These properties enable us to propose and implement a smart assisted value iteration (AVI) algorithm and an approximate DP (ADP) that exploits related functional approximations. Numerical results demonstrate the validity of the solution techniques and the computational advantage of the proposed ADP on realistic, large-state-space problems