637 research outputs found
A Stochastic Game Framework for Efficient Energy Management in Microgrid Networks
We consider the problem of energy management in microgrid networks. A
microgrid is capable of generating a limited amount of energy from a renewable
resource and is responsible for handling the demands of its dedicated
customers. Owing to the variable nature of renewable generation and the demands
of the customers, it becomes imperative that each microgrid optimally manages
its energy. This involves intelligently scheduling the demands at the customer
side, selling (when there is a surplus) and buying (when there is a deficit)
the power from its neighboring microgrids depending on its current and future
needs. Typically, the transaction of power among the microgrids happens at a
pre-decided price by the central grid. In this work, we formulate the problems
of demand and battery scheduling, energy trading and dynamic pricing (where we
allow the microgrids to decide the price of the transaction depending on their
current configuration of demand and renewable energy) in the framework of
stochastic games. Subsequently, we propose a novel approach that makes use of
independent learners Deep Q-learning algorithm to solve this problem. Through
extensive empirical evaluation, we show that our proposed framework is more
beneficial to the majority of the microgrids and we provide a detailed analysis
of the results
Reinforcement Learning Based Cooperative P2P Energy Trading between DC Nanogrid Clusters with Wind and PV Energy Resources
In order to replace fossil fuels with the use of renewable energy resources,
unbalanced resource production of intermittent wind and photovoltaic (PV) power
is a critical issue for peer-to-peer (P2P) power trading. To resolve this
problem, a reinforcement learning (RL) technique is introduced in this paper.
For RL, graph convolutional network (GCN) and bi-directional long short-term
memory (Bi-LSTM) network are jointly applied to P2P power trading between
nanogrid clusters based on cooperative game theory. The flexible and reliable
DC nanogrid is suitable to integrate renewable energy for distribution system.
Each local nanogrid cluster takes the position of prosumer, focusing on power
production and consumption simultaneously. For the power management of nanogrid
clusters, multi-objective optimization is applied to each local nanogrid
cluster with the Internet of Things (IoT) technology. Charging/discharging of
electric vehicle (EV) is performed considering the intermittent characteristics
of wind and PV power production. RL algorithms, such as deep Q-learning network
(DQN), deep recurrent Q-learning network (DRQN), Bi-DRQN, proximal policy
optimization (PPO), GCN-DQN, GCN-DRQN, GCN-Bi-DRQN, and GCN-PPO, are used for
simulations. Consequently, the cooperative P2P power trading system maximizes
the profit utilizing the time of use (ToU) tariff-based electricity cost and
system marginal price (SMP), and minimizes the amount of grid power
consumption. Power management of nanogrid clusters with P2P power trading is
simulated on the distribution test feeder in real-time and proposed GCN-PPO
technique reduces the electricity cost of nanogrid clusters by 36.7%.Comment: 22 pages, 8 figures, to be submitted to Applied Energy of Elsevie
Multi-Microgrid Collaborative Optimization Scheduling Using an Improved Multi-Agent Soft Actor-Critic Algorithm
The implementation of a multi-microgrid (MMG) system with multiple renewable
energy sources enables the facilitation of electricity trading. To tackle the
energy management problem of a MMG system, which consists of multiple renewable
energy microgrids belonging to different operating entities, this paper
proposes a MMG collaborative optimization scheduling model based on a
multi-agent centralized training distributed execution framework. To enhance
the generalization ability of dealing with various uncertainties, we also
propose an improved multi-agent soft actor-critic (MASAC) algorithm, which
facilitates en-ergy transactions between multi-agents in MMG, and employs
automated machine learning (AutoML) to optimize the MASAC hyperparameters to
further improve the generalization of deep reinforcement learning (DRL). The
test results demonstrate that the proposed method successfully achieves power
complementarity between different entities, and reduces the MMG system
operating cost. Additionally, the proposal significantly outperforms other
state-of-the-art reinforcement learning algorithms with better economy and
higher calculation efficiency.Comment: Accepted by Energie
Decentralized Microgrid Energy Management: A Multi-agent Correlated Q-learning Approach
Microgrids (MG) are anticipated to be important players in the future smart
grid. For proper operation of MGs an Energy Management System (EMS) is
essential. The EMS of an MG could be rather complicated when renewable energy
resources (RER), energy storage system (ESS) and demand side management (DSM)
need to be orchestrated. Furthermore, these systems may belong to different
entities and competition may exist between them. Nash equilibrium is most
commonly used for coordination of such entities however the convergence and
existence of Nash equilibrium can not always be guaranteed. To this end, we use
the correlated equilibrium to coordinate agents, whose convergence can be
guaranteed. In this paper, we build an energy trading model based on mid-market
rate, and propose a correlated Q-learning (CEQ) algorithm to maximize the
revenue of each agent. Our results show that CEQ is able to balance the revenue
of agents without harming total benefit. In addition, compared with Q-learning
without correlation, CEQ could save 19.3% cost for the DSM agent and 44.2% more
benefits for the ESS agent.Comment: Accepted by 2020 IEEE International Conference on SmartGridComm,
978-1-7281-6127-3/20/$31.00 copyright 2020 IEE
A systematic review of machine learning techniques related to local energy communities
In recent years, digitalisation has rendered machine learning a key tool for improving processes in several sectors, as in the case of electrical power systems. Machine learning algorithms are data-driven models based on statistical learning theory and employed as a tool to exploit the data generated by the power system and its users. Energy communities are emerging as novel organisations for consumers and prosumers in the distribution grid. These communities may operate differently depending on their objectives and the potential service the community wants to offer to the distribution system operator. This paper presents the conceptualisation of a local energy community on the basis of a review of 25 energy community projects. Furthermore, an extensive literature review of machine learning algorithms for local energy community applications was conducted, and these algorithms were categorised according to forecasting, storage optimisation, energy management systems, power stability and quality, security, and energy transactions. The main algorithms reported in the literature were analysed and classified as supervised, unsupervised, and reinforcement learning algorithms. The findings demonstrate the manner in which supervised learning can provide accurate models for forecasting tasks. Similarly, reinforcement learning presents interesting capabilities in terms of control-related applications.publishedVersio
- …