Machine Learning Based Simulation and Control Framework for Power Distribution Systems with Transactive Elements

Transactive Energy (TE) has been recognized as a promising combination of techniques for improving the efficiency of modern power grids through market-based transactive exchanges between energy producers and energy consumers. It is of significant interest to identify optimal strategy to control the transactive load in TE systems. The behaviors of transactive loads are affected by the energy market values which in return impact the operation and stability of the distribution system. To evaluate the benefits and impacts of transactive loads and new control mechanisms, time series simulations are commonly used. These simulations consider the pricing response and the physical constraints of the system simultaneously. Such simulations are computationally demanding due to the information exchange among various participants and the complex co-simulation environments. This dissertation first explores the reduced order models to support quasi-static time-series (QSTS) simulations for power distribution systems with independent dynamic non-responsive load to address the limitations of the order reduction methods. Further, a reduced order model for transactive systems with responsive load is proposed. The proposed model consists of an aggregate responsive load (ARL) agent which utilizes two Recurrent Neural Networks (RNN) with Long Short-Term Memory units (LSTMs) to represent the transactive elements in TE systems. The developed ARL agent generates load behavior for transactive elements and interacts with the electricity market. In addition, for individual transactive elements, a control strategy for the residential Heating, Ventilation, and Air Conditioning (HVAC) is introduced through the solution of an optimization problem that balances between the energy cost and consumer’s dissatisfaction. A reinforcement learning (RL) algorithm based on Deep Deterministic Policy Gradients (DDPG) is used to obtain the optimal control strategy for the HVAC systems. The reduced order model and the DDPG RL-based control are both implemented in the Transactive Energy Simulation Platform (TESP). The reduced order model is able to produce transactive behavior very close to the full simulation model while achieving significant simulation time reduction. Moreover, simulation results demonstrated that the proposed control method for HVACs reduces the energy cost and improves the customers’ comfort simultaneously