Realistic Peer-to-Peer Energy Trading Model for Microgrids Using Deep Reinforcement Learning

In this paper, we integrate deep reinforcement learning with our realistic peer-to-peer (P2P) energy trading model to address a decision-making problem for microgrids (MGs) in the local energy market. First, an hour-ahead P2P energy trading model with a set of critical physical constraints is formed. Then, the decision-making process of energy trading is built as a Markov decision process, which is used to find the optimal strategies for MGs using a deep reinforcement learning (DRL) algorithm. Specifically, a modified deep Q-network (DQN) algorithm helps the MGs to utilise their resources and make better strategies. Finally, we choose several real-world electricity data sets to perform the simulations. The DQN-based energy trading strategies improve the utilities of the MGs and significantly reduce the power plant schedule with a virtual penalty function. Moreover, the model can determine the best battery for the selected MG. The results show that this P2P energy trading model can be applied to real-world situations

Correlated Deep Q-learning based Microgrid Energy Management

Microgrid (MG) energy management is an important part of MG operation. Various entities are generally involved in the energy management of an MG, e.g., energy storage system (ESS), renewable energy resources (RER) and the load of users, and it is crucial to coordinate these entities. Considering the significant potential of machine learning techniques, this paper proposes a correlated deep Q-learning (CDQN) based technique for the MG energy management. Each electrical entity is modeled as an agent which has a neural network to predict its own Q-values, after which the correlated Q-equilibrium is used to coordinate the operation among agents. In this paper, the Long Short Term Memory networks (LSTM) based deep Q-learning algorithm is introduced and the correlated equilibrium is proposed to coordinate agents. The simulation result shows 40.9% and 9.62% higher profit for ESS agent and photovoltaic (PV) agent, respectively.Comment: Accepted by 2020 IEEE 25th International Workshop on CAMAD, 978-1-7281-6339-0/20/$31.00 \copyright 2020 IEE

Community energy groups: Can they shield consumers from the risks of using blockchain for peer-to-peer energy trading?

Peer-to-peer (P2P) energy trading is emerging as a new mechanism for settling the ex-change of energy between renewable energy generators and consumers. P2P provides a mechanism for local balancing when it is facilitated through distributed ledgers (‘blockchains’). Energy communities across Europe have uncovered the potential of this technology and are currently running pi-lots to test its applicability in P2P energy trading. The aim of this paper is to assess, using legal literature and legislation, whether the legal forms available to energy communities in the United Kingdom (UK) can help resolve some of the uncertainties around the individual use of blockchain for P2P energy trading. This includes the legal recognition of ‘prosumers’, the protection of their personal data, as well as the validity of ‘smart contracts’ programmed to trade energy on the block-chain network. The analysis has shown that legal entities, such as Limited Liability Partnerships and Co-operative Societies, can play a crucial role in providing the necessary framework to protect consumers engaging in these transactions. This is particularly the case for co-operatives, given that they can hold members liable for not respecting the rules set out in their (compulsory) governing document. These findings are relevant to other European countries, where the energy co-operative model is also used

Reinforcement learning in local energy markets

Local energy markets (LEMs) are well suited to address the challenges of the European energy transition movement. They incite investments in renewable energy sources (RES), can improve the integration of RES into the energy system, and empower local communities. However, as electricity is a low involvement good, residential households have neither the expertise nor do they want to put in the time and effort to trade themselves on their own on short-term LEMs. Thus, machine learning algorithms are proposed to take over the bidding for households under realistic market information. We simulate a LEM on a 15 min merit-order market mechanism and deploy reinforcement learning as strategic learning for the agents. In a multi-agent simulation of 100 households including PV, micro-cogeneration, and demand shifting appliances, we show how participants in a LEM can achieve a self-sufficiency of up to 30% with trading and 41,4% with trading and demand response (DR) through an installation of only 5kWp PV panels in 45% of the households under affordable energy prices. A sensitivity analysis shows how the results differ according to the share of renewable generation and degree of demand flexibility

Small-Scale Communities Are Sufficient for Cost- and Data-Efficient Peer-to-Peer Energy Sharing

Due to ever lower cost, investments in renewable electricity generation and storage have become more attractive to electricity consumers in recent years. At the same time, electricity generation and storage have become something to share or trade locally in energy communities or microgrid systems. In this context, peer-to-peer (P2P) sharing has gained attention, since it offers a way to optimize the cost-benefits from distributed resources, making them financially more attractive. However, it is not yet clear in which situations consumers do have interests to team up and how much cost is saved through cooperation in practical instances. While introducing realistic continuous decisions, through detailed analysis based on large-scale measured household data, we show that the financial benefit of cooperation does not require an accurate forecasting. Furthermore, we provide strong evidence, based on analysis of the same data, that even P2P networks with only 2--5 participants can reach a high fraction (96% in our study) of the potential gain, i.e., of the ideal offline (i.e., non-continuous) achievable gain. Maintaining such small communities results in much lower associated costs and better privacy, as each participant only needs to share its data with 1--4 other peers. These findings shed new light and motivate requirements for distributed, continuous and dynamic P2P matching algorithms for energy trading and sharing

New actor types in electricity market simulation models: Deliverable D4.4

Project TradeRES - New Markets Design & Models for 100% Renewable Power Systems: https://traderes.eu/about/ABSTRACT: The modelling of agents in the simulation models and tools is of primary importance if the quality and the validity of the simulation outcomes are at stake. This is the first version of the report that deals with the representation of electricity market actors’ in the agent based models (ABMs) used in TradeRES project. With the AMIRIS, the EMLab-Generation (EMLab), the MASCEM and the RESTrade models being in the centre of the analysis, the subject matter of this report has been the identification of the actors’ characteristics that are already covered by the initial (with respect to the project) version of the models and the presentation of the foreseen modelling enhancements. For serving these goals, agent attributes and representation methods, as found in the literature of agent-driven models, are considered initially. The detailed review of such aspects offers the necessary background and supports the formation of a context that facilitates the mapping of actors’ characteristics to agent modelling approaches. Emphasis is given in several approaches and technics found in the literature for the development of a broader environment, on which part of the later analysis is deployed. Although the ABMs that are used in the project constitute an important part of the literature, they have not been included in the review since they are the subject of another section.N/

D4.4 - New actor types in electricity market simulation models

The modelling of agents in the simulation models and tools is of primary importance if the quality and the validity of the simulation outcomes are at stake. This is the final version of the report that deals with the representation of electricity market actors’ in the agent-based models (ABMs) used in TradeRES project and it was developed within the scope of Task 4.2 - Representation of new actors, markets and policies. With the ABMs available in the consortium (AMIRIS, the EMLab, the MASCEM and the RESTrade) being in the centre of the analysis, the subject matter of this report has been the identification of the actors’ characteristics that are already covered by the initial (with respect to the project) version of the models and the presentation of the foreseen modelling enhancements