A Constraint Enforcement Deep Reinforcement Learning Framework for Optimal Energy Storage Systems Dispatch

The optimal dispatch of energy storage systems (ESSs) presents formidable challenges due to the uncertainty introduced by fluctuations in dynamic prices, demand consumption, and renewable-based energy generation. By exploiting the generalization capabilities of deep neural networks (DNNs), deep reinforcement learning (DRL) algorithms can learn good-quality control models that adaptively respond to distribution networks' stochastic nature. However, current DRL algorithms lack the capabilities to enforce operational constraints strictly, often even providing unfeasible control actions. To address this issue, we propose a DRL framework that effectively handles continuous action spaces while strictly enforcing the environments and action space operational constraints during online operation. Firstly, the proposed framework trains an action-value function modeled using DNNs. Subsequently, this action-value function is formulated as a mixed-integer programming (MIP) formulation enabling the consideration of the environment's operational constraints. Comprehensive numerical simulations show the superior performance of the proposed MIP-DRL framework, effectively enforcing all constraints while delivering high-quality dispatch decisions when compared with state-of-the-art DRL algorithms and the optimal solution obtained with a perfect forecast of the stochastic variables.Comment: This paper has been submitted to a publication in a journal. This corresponds to the submitted version. After acceptance, it may be removed depending on the journal's requirements for copyrigh

Quantum Neural Networks for Power Flow Analysis

This paper explores the potential application of quantum and hybrid quantum-classical neural networks in power flow analysis. Experiments are conducted using two small-size datasets based on the IEEE 4-bus and 33-bus test systems. A systematic performance comparison is also conducted among quantum, hybrid quantum-classical, and classical neural networks. The comparison is based on (i) generalization ability, (ii) robustness, (iii) training dataset size needed, (iv) training error. (v) training computational time, and (vi) training process stability. The results show that the developed quantum-classical neural network outperforms both quantum and classical neural networks, and hence can improve deep learning-based power flow analysis in the noisy-intermediate-scale quantum (NISQ) era.Comment: 7 pages, 15 figure

EV2Gym: A Flexible V2G Simulator for EV Smart Charging Research and Benchmarking

As electric vehicle (EV) numbers rise, concerns about the capacity of current charging and power grid infrastructure grow, necessitating the development of smart charging solutions. While many smart charging simulators have been developed in recent years, only a few support the development of Reinforcement Learning (RL) algorithms in the form of a Gym environment, and those that do usually lack depth in modeling Vehicle-to-Grid (V2G) scenarios. To address the aforementioned issues, this paper introduces the EV2Gym, a realistic simulator platform for the development and assessment of small and large-scale smart charging algorithms within a standardized platform. The proposed simulator is populated with comprehensive EV, charging station, power transformer, and EV behavior models validated using real data. EV2Gym has a highly customizable interface empowering users to choose from pre-designed case studies or craft their own customized scenarios to suit their specific requirements. Moreover, it incorporates a diverse array of RL, mathematical programming, and heuristic algorithms to speed up the development and benchmarking of new solutions. By offering a unified and standardized platform, EV2Gym aims to provide researchers and practitioners with a robust environment for advancing and assessing smart charging algorithms.Comment: 10 pages, 9 figures, and 6 table

Conversations with my washing machine: an in-the-wild study of demand-shifting with self-generated energy

Domestic microgeneration is the onsite generation of low- and zero-carbon heat and electricity by private households to meet their own needs. In this paper we explore how an everyday household routine – that of doing laundry – can be augmented by digital technologies to help households with photovoltaic solar energy generation to make better use of self-generated energy. This paper presents an 8-month in-the-wild study that involved 18 UK households in longitudinal energy data collection, prototype deployment and participatory data analysis. Through a series of technology interventions mixing energy feedback, proactive suggestions and direct control the study uncovered opportunities, potential rewards and barriers for families to shift energy consuming household activities and highlights how digital technology can act as mediator between household laundry routines and energy demand-shifting behaviors. Finally, the study provides insights into how a “smart” energy-aware washing machine shapes organization of domestic life and how people “communicate” with their washing machine

Smart homes and their users:a systematic analysis and key challenges

Published research on smart homes and their users is growing exponentially, yet a clear understanding of who these users are and how they might use smart home technologies is missing from a field being overwhelmingly pushed by technology developers. Through a systematic analysis of peer-reviewed literature on smart homes and their users, this paper takes stock of the dominant research themes and the linkages and disconnects between them. Key findings within each of nine themes are analysed, grouped into three: (1) views of the smart home-functional, instrumental, socio-technical; (2) users and the use of the smart home-prospective users, interactions and decisions, using technologies in the home; and (3) challenges for realising the smart home-hardware and software, design, domestication. These themes are integrated into an organising framework for future research that identifies the presence or absence of cross-cutting relationships between different understandings of smart homes and their users. The usefulness of the organising framework is illustrated in relation to two major concerns-privacy and control-that have been narrowly interpreted to date, precluding deeper insights and potential solutions. Future research on smart homes and their users can benefit by exploring and developing cross-cutting relationships between the research themes identified

Cyber-physical energy systems modeling, test specification, and co-simulation based testing

The gradual deployment of intelligent and coordinated devices in the electrical power system needs careful investigation of the interactions between the various domains involved. Especially due to the coupling between ICT and power systems a holistic approach for testing and validating is required. Taking existing (quasi-) standardised smart grid system and test specification methods as a starting point, we are developing a holistic testing and validation approach that allows a very flexible way of assessing the system level aspects by various types of experiments (including virtual, real, and mixed lab settings). This paper describes the formal holistic test case specification method and applies it to a particular co-simulation experimental setup. The various building blocks of such a simulation (i.e., FMI, mosaik, domain-specific simulation federates) are covered in more detail. The presented method addresses most modeling and specification challenges in cyber-physical energy systems and is extensible for future additions such as uncertainty quantification

An Integrated Research Infrastructure for Validating Cyber-Physical Energy Systems

Renewables are key enablers in the plight to reduce greenhouse gas emissions and cope with anthropogenic global warming. The intermittent nature and limited storage capabilities of renewables culminate in new challenges that power system operators have to deal with in order to regulate power quality and ensure security of supply. At the same time, the increased availability of advanced automation and communication technologies provides new opportunities for the derivation of intelligent solutions to tackle the challenges. Previous work has shown various new methods of operating highly interconnected power grids, and their corresponding components, in a more effective way. As a consequence of these developments, the traditional power system is being transformed into a cyber-physical energy system, a smart grid. Previous and ongoing research have tended to mainly focus on how specific aspects of smart grids can be validated, but until there exists no integrated approach for the analysis and evaluation of complex cyber-physical systems configurations. This paper introduces integrated research infrastructure that provides methods and tools for validating smart grid systems in a holistic, cyber-physical manner. The corresponding concepts are currently being developed further in the European project ERIGrid.Comment: 8th International Conference on Industrial Applications of Holonic and Multi-Agent Systems (HoloMAS 2017