Reinforcement Learning Based Cooperative P2P Energy Trading between DC Nanogrid Clusters with Wind and PV Energy Resources

In order to replace fossil fuels with the use of renewable energy resources, unbalanced resource production of intermittent wind and photovoltaic (PV) power is a critical issue for peer-to-peer (P2P) power trading. To resolve this problem, a reinforcement learning (RL) technique is introduced in this paper. For RL, graph convolutional network (GCN) and bi-directional long short-term memory (Bi-LSTM) network are jointly applied to P2P power trading between nanogrid clusters based on cooperative game theory. The flexible and reliable DC nanogrid is suitable to integrate renewable energy for distribution system. Each local nanogrid cluster takes the position of prosumer, focusing on power production and consumption simultaneously. For the power management of nanogrid clusters, multi-objective optimization is applied to each local nanogrid cluster with the Internet of Things (IoT) technology. Charging/discharging of electric vehicle (EV) is performed considering the intermittent characteristics of wind and PV power production. RL algorithms, such as deep Q-learning network (DQN), deep recurrent Q-learning network (DRQN), Bi-DRQN, proximal policy optimization (PPO), GCN-DQN, GCN-DRQN, GCN-Bi-DRQN, and GCN-PPO, are used for simulations. Consequently, the cooperative P2P power trading system maximizes the profit utilizing the time of use (ToU) tariff-based electricity cost and system marginal price (SMP), and minimizes the amount of grid power consumption. Power management of nanogrid clusters with P2P power trading is simulated on the distribution test feeder in real-time and proposed GCN-PPO technique reduces the electricity cost of nanogrid clusters by 36.7%.Comment: 22 pages, 8 figures, to be submitted to Applied Energy of Elsevie

Demand response performance and uncertainty: A systematic literature review

The present review has been carried out, resorting to the PRISMA methodology, analyzing 218 published articles. A comprehensive analysis has been conducted regarding the consumer's role in the energy market. Moreover, the methods used to address demand response uncertainty and the strategies used to enhance performance and motivate participation have been reviewed. The authors find that participants will be willing to change their consumption pattern and behavior given that they have a complete awareness of the market environment, seeking the optimal decision. The authors also find that a contextual solution, giving the right signals according to the different behaviors and to the different types of participants in the DR event, can improve the performance of consumers' participation, providing a reliable response. DR is a mean of demand-side management, so both these concepts are addressed in the present paper. Finally, the pathways for future research are discussed.This article is a result of the project RETINA (NORTE-01-0145- FEDER-000062), supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF). We also acknowledge the work facilities and equipment provided by GECAD research center (UIDB/00760/2020) to the project team, and grants CEECIND/02887/2017 and SFRH/BD/144200/2019.info:eu-repo/semantics/publishedVersio

Industrial Multi-Energy and Production Management Scheme in Cyber-Physical Environments: A Case Study in a Battery Manufacturing Plant

Among the various electricity consumer sectors, the consumption level of the industrial sector is often considered as the largest portion of electricity consumption, highlighting the urgent need to implement demand response (DR) energy management. However, implementation of DR for the industrial sector requires a more sophisticated and different scheme compared to the residential and commercial sector. This study explores all the elastic segments of plant multi-energy production, conversion, and consumption. We then construct a real-time industrial facilities management problem as an optimal dispatch model to enclose these elastic segments and production constraints in cyber-physical environments. Moreover, a model predictive-based centralised dispatch scheme is proposed to address the uncertainties of real-time price and renewable energy forecasting while considering the sequence of the production process. Numerical results demonstrate that the proposed scheme can enhance energy efficiency and economics of lithium battery manufacturing plant through responding to the real-time price whilst ensuring the completion of production tasks

A novel multi-level and community-based agent ecosystem to support customers dynamic decision-making in smart grids

Electrical systems have evolved at a fast pace over the past years, particularly in response to the current environmental and climate challenges. Consequently, the European Union and the United Nations have encouraged the development of a more sustainable energy strategy. This strategy triggered a paradigm shift in energy consumption and production, which becoming increasingly distributed, resulted in the development and emergence of smart energy grids. Multi-agent systems are one of the most widely used artificial intelligence concepts in smart grids. Both multi-agent systems and smart grids are distributed, so there is correspondence between the used technology and the network's complex reality. Due to the wide variety of multi-agent systems applied to smart grids, which typically have very specific goals, the ability to model the network as a whole may be compromised, as communication between systems is typically non-existent. This dissertation, therefore, proposes an agent-based ecosystem to model smart grids in which different agent-based systems can coexist. This dissertation aims to conceive, implement, test, and validate a new agent-based ecosystem, entitled A4SG (agent-based ecosystem for smart grids modelling), which combines the concepts of multi-agent systems and agent communities to enable the modelling and representation of smart grids and the entities that compose them. The proposed ecosystem employs an innovative methodology for managing static or dynamic interactions present in smart grids. The creation of a solution that allows the integration of existing systems into an ecosystem, enables the representation of smart grids in a realistic and comprehensive manner. A4SG integrates several functionalities that support the ecosystem's management, also conceived, implemented, tested, and validated in this dissertation. Two mobility functionalities are proposed: one that allows agents to move between physical machines and another that allows "virtual" mobility, where agents move between agent communities to improve the context for the achievement of their objectives. In order to prevent an agent from becoming overloaded, a novel functionality is proposed to enable the creation of agents that function as extensions of the main agent (i.e., branch agents), allowing the distribution of objectives among the various extensions of the main agent. Several case studies, which test the proposed services and functionalities individually and the ecosystem as a whole, were used to test and validate the proposed solution. These case studies were conducted in realistic contexts using data from multiple sources, including energy communities. The results indicate that the used methodologies can increase participation in demand response events, increasing the fitting between consumers and aggregators from 12 % to 69 %, and improve the strategies used in energy transaction markets, allowing an energy community of 50 customers to save 77.0 EUR per week.Os últimos anos têm sido de mudança nos sistemas elétricos, especialmente devido aos atuais desafios ambientais e climáticos. A procura por uma estratégia mais sustentável para o domínio da energia tem sido promovida pela União Europeia e pela Organização das Nações Unidas. A mudança de paradigma no que toca ao consumo e produção de energia, que acontece, cada vez mais, de forma distribuída, tem levado à emergência das redes elétricas inteligentes. Os sistemas multi-agente são um dos conceitos, no domínio da inteligência artificial, mais aplicados em redes inteligentes. Tanto os sistemas multi-agente como as redes inteligentes têm uma natureza distribuída, existindo por isso um alinhamento entre a tecnologia usada e a realidade complexa da rede. Devido a existir uma vasta oferta de sistemas multi-agente aplicados a redes inteligentes, normalmente com objetivos bastante específicos, a capacidade de modelar a rede como um todo pode ficar comprometida, porque a comunicação entre sistemas é, geralmente, inexistente. Por isso, esta dissertação propõe um ecossistema baseado em agentes para modelar as redes inteligentes, onde vários sistemas de agentes coexistem. Esta dissertação pretende conceber, implementar, testar, e validar um novo ecossistema multiagente, intitulado A4SG (agent-based ecosystem for smart grids modelling), que combina os conceitos de sistemas multi-agente e comunidades de agentes, permitindo a modelação e representação de redes inteligentes e das suas entidades. O ecossistema proposto utiliza uma metodologia inovadora para gerir as interações presentes nas redes inteligentes, sejam elas estáticas ou dinâmicas. A criação de um ecossistema que permite a integração de sistemas já existentes, cria a possibilidade de uma representação realista e detalhada das redes de energia. O A4SG integra diversas funcionalidades, também estas concebidas, implementadas, testadas, e validadas nesta dissertação, que suportam a gestão do próprio ecossistema. São propostas duas funcionalidades de mobilidade, uma que permite aos agentes mover-se entre máquinas físicas, e uma que permite uma mobilidade “virtual”, onde os agentes se movem entre comunidades de agentes, de forma a melhorar o contexto para a execução dos seus objetivos. É também proposta uma nova funcionalidade que permite a criação de agentes que funcionam como uma extensão de um agente principal, com o objetivo de evitar a sobrecarga de um agente, permitindo a distribuição de objetivos entre as várias extensões do agente principal. A solução proposta foi testada e validada por vários casos de estudo, que testam os serviços e funcionalidades propostas individualmente, e o ecossistema como um todo. Estes casos de estudo foram executados em contextos realistas, usando dados provenientes de diversas fontes, tais como comunidades de energia. Os resultados demonstram que as metodologias utilizadas podem melhorar a participação em eventos de demand response, subindo a adequação entre consumidores e agregadores de 12 % para 69 %, e melhorar as estratégias utilizadas em mercados de transações de energia, permitindo a uma comunidade de energia com 50 consumidores poupar 77,0 EUR por semana

ASSESSMENT OF CONSUMERS POWER CONSUMPTION OPTIMIZATION BASED ON DEMAND SIDE MANAGEMENT

To ensure the functioning of the energy system, coordination and increase the efficiency of its parts need new control mechanisms. Generation, transmission and consumption of electricity needed control mechanisms that include integration of self-organizing power and heat supply systems, built on multi-agent principle. Also they must correspond intellectual basis, monitoring and accumulation. This includes effectiveness assessment of the state and analysis of technical, technological and organizational management mechanisms. One of the main parts is interaction principles of energy systems in accordance with European Community policy at various levels at liberalized electricity market. In most developed countries, demand management programs are widely used as a means of harmonizing the modes of generation and consumption in the power supply system. The main direct methods are set in the form of electricity tariffs. Indirect methods are set in the form of programs to manage electricity demand and the possibility of their application to manage electricity demand. Methods for estimating the unevenness of the daily schedule of electricity consumption and the factors influencing the technological environment are presented. The work aims at scientific and applied problem – finding methods of estimation and features of managing the demand for electricity. The use of the proposed estimation methods of electricity consumption influence non-uniformity on the level of power supplies system losses based on Frize QF power and optimization of consumers’ operation modes in the power supply system is considered. Approaches and optimization mechanisms of the daily electricity consumption on the example of a residential complex with the possibility of energy accumulation are offere

Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review

Recent years have seen an increasing interest in Demand Response (DR) as a means to provide flexibility, and hence improve the reliability of energy systems in a cost-effective way. Yet, the high complexity of the tasks associated with DR, combined with their use of large-scale data and the frequent need for near real-time de-cisions, means that Artificial Intelligence (AI) and Machine Learning (ML) — a branch of AI — have recently emerged as key technologies for enabling demand-side response. AI methods can be used to tackle various challenges, ranging from selecting the optimal set of consumers to respond, learning their attributes and pref-erences, dynamic pricing, scheduling and control of devices, learning how to incentivise participants in the DR schemes and how to reward them in a fair and economically efficient way. This work provides an overview of AI methods utilised for DR applications, based on a systematic review of over 160 papers, 40 companies and commercial initiatives, and 21 large-scale projects. The papers are classified with regards to both the AI/ML algorithm(s) used and the application area in energy DR. Next, commercial initiatives are presented (including both start-ups and established companies) and large-scale innovation projects, where AI methods have been used for energy DR. The paper concludes with a discussion of advantages and potential limitations of reviewed AI techniques for different DR tasks, and outlines directions for future research in this fast-growing area

Demand Response in Smart Grids

The Special Issue “Demand Response in Smart Grids” includes 11 papers on a variety of topics. The success of this Special Issue demonstrates the relevance of demand response programs and events in the operation of power and energy systems at both the distribution level and at the wide power system level. This reprint addresses the design, implementation, and operation of demand response programs, with focus on methods and techniques to achieve an optimized operation as well as on the electricity consumer

Transitioning power distribution grid into nanostructured ecosystem : prosumer-centric sovereignty

PhD ThesisGrowing acceptance for in-house Distributed Energy Resource (DER) installations at lowvoltage level have gained much significance in recent years due to electricity market liberalisations and opportunities in reduced energy billings through personalised utilisation management for targeted business model. In consequence, modelling of passive customers’ electric power system are progressively transitioned into Prosumer-based settings where presidency for Transactive Energy (TE) system framework is favoured. It amplifies Prosumers’ commitments into annexing TE values during market participations and optimised energy management to earn larger rebates and incentives from TE programs. However, when dealing with mass Behind-The-Meter DER administrations, Utility foresee managerial challenges when dealing with distribution network analysis, planning, protection, and power quality security based on Prosumers’ flexibility in optimising their energy needs. This dissertation contributes prepositions into modelling Distributed Energy Resources Management System (DERMS) as an aggregator designed for Prosumer-centered cooperation, interoperating TE control and coordination as key parameters to market for both optimised energy trading and ancillary services in a Community setting. However, Prosumers are primarily driven to create a profitable business model when modelling their DERMS aggregator. Greedy-optimisation exploitations are negative concerns when decisions made resulted in detrimental-uncoordinated outcomes on Demand-Side Response (DSR) and capacity market engagements. This calls for policy decision makers to contract safe (i.e. cooperative yet competitive tendency) business models for Prosumers to maximise TE values while enhancing network’s power quality metrics and reliability performances. Firstly, digitalisation and nanostructuring of distribution network is suggested to identify Prosumer as a sole energy citizen while extending bilateral trading between Prosumer-to- Prosumer (PtP) with the involvements of other grid operators−TE system. Modelling of Nanogrid environment for DER integrations and establishment of local area network infrastructure for IoT security (i.e. personal computing solutions and data protection) are committed for communal engagements in a decentralise setting. Secondly, a multi-layered Distributed Control Framework (DCF) is proposed using Microsoft Azure cloud-edge platform that cascades energy actors into respective layers of TE control and coordination. Furthermore, modelling of flexi-edge computing architecture is proposed, comprising of Contract-Oriented Sensor-based Application Platform (COSAP) employing Multi-Agent System (MAS) to enhance data-sharing privacy and contract coalition agreements during PtP engagements. Lastly, the Agents of MAS are programmed with cooperative yet competitive intelligences attributed to Reinforcement Learning (RL) and Neural Networks (NN) algorithms to solve multimodal socio-economical and uncertainty problems that corresponded to Prosumers’ dynamic energy priorities within the TE framework. To verify the DERMS aggregator operations, three business models were proposed (i.e. greedy-profit margin, collegial-peak demand, reserved-standalone) to analyse comparative technical/physical and economic/social dimensions. Results showed that the proposed TE-valued DERMS aggregator provides participation versatility in the electricity market that enables competitive edginess when utilising Behind-The-Meter DERs in view of Prosumer’s asset scheduling, bidding strategy, and corroborative ancillary services. Performance metrics were evaluated on both domestic and industrial NG environments against IEEE Standard 2030.7-2017 & 2030.8-2018 compliances to ensure deployment practicability. Subsequently, proposed in-house protection system for DER installation serves as an add-on monitoring service which can be incorporated into existing Advance Distribution Management System (ADMS) for Distribution Service Operator (DSO) and field engineers use, ADMS aggregator. It provides early fault detections and isolation processes from allowing fault current to propagate upstream causing cascading power quality issues across the feeder line. In addition, ADMS aggregator also serves as islanding indicator that distinguishes Nanogrid’s islanding state from unintentional or intentional operations. Therefore, a Overcurrent Current Relay (OCR) is proposed using Fuzzy Logic (FL) algorithm to detect, profile, and provide decisional isolation processes using specified OCRs. Moreover, the proposed expert knowledge in FL is programmed to detect fault crises despite insufficient fault current level contributed by DER (i.e. solar PV system) which conventional OCR fails to trigger

Multi agent system framework for demand response management in distribution networks

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe inexorable increase in penetration of clean energies and responsive loads in the Distribution Network (DN), introduces new technical challenges for network operators. The responsibilities of Distribution Network Operators (DNOs) are being adjusted to cope with current system challenges and they are transitioning to Distribution System Operators (DSOs), taking a more active role in dynamically managing power flows across the network. Further, the advancement in distribution automation technologies provides greater opportunities for energy consumers to take more effective participation in demand reduction schemes and DSOs can be enablers of Demand Response (DR). Hence, the functionality of DR can be considered an alternative, lower cost, carbon-saving and flexible solution to defer network reinforcement. This forms the rationale behind this thesis, which aims to provide an in-depth investigation of the potential responsiveness in residential demand and its effect on constraint management of the DN. The main contribution of this thesis is the design, development and implementation of a Multi Agent System (MAS) framework for active DN management through residential DR. One advantage of the proposed platform is the capability of integrating both centralised and decentralised DR control mechanisms. It employs the responsiveness of demand from both loads shifting and shedding through price-based and incentive-based DR respectively. The feasibility and effectiveness of such a platform has been evaluated by developing and implementing the DR mechanism in three levels. The DR algorithm for several static and dynamic electricity tariffs, (Time of Use (ToU), Day-Ahead (DA) and Real Time Pricing (RTP)), is designed and developed in Low Voltage (LV) feeders. This is then expanded and implemented in a Medium Voltage (MV) feeder under an RTP environment. Finally, two incentive-based DR schemes: emergency and local community DR, are merged in the MV/LV network to improve its reliability and security. The implementation of the MAS framework demonstrated its configurability and scalability through three case studies under different scenarios. One novel aspect of this research is the consideration of customers’ characteristics in the design of the DR algorithms. In addition, at MV level, the tariffs and the required DR are allocated to each LV feeder specifically taking into account their DR potential and participation effects on the overall network performance. The simulation results at LV level show that maximum peak demand reduction and the most flattened load profile are achieved with RTP. At MV/LV network, MAS provides a community environment where the consumers can collaborate to decrease their overall demand. Moreover, the local community can reduce their dependency on the grid during daytime with PV generation. It is concluded that DR trading can benefit all players economically and also lessen DN violations from stipulated limits