Deep Reinforcement Learning for Distribution Network Operation and Electricity Market

The conventional distribution network and electricity market operation have become challenging under complicated network operating conditions, due to emerging distributed electricity generations, coupled energy networks, and new market behaviours. These challenges include increasing dynamics and stochastics, and vast problem dimensions such as control points, measurements, and multiple objectives, etc. Previously the optimization models were often formulated as conventional programming problems and then solved mathematically, which could now become highly time-consuming or sometimes infeasible. On the other hand, with the recent advancement of artificial intelligence technologies, deep reinforcement learning (DRL) algorithms have demonstrated their excellent performances in various control and optimization fields. This indicates a potential alternative to address these challenges. In this thesis, DRL-based solutions for distribution network operation and electricity market have been investigated and proposed. Firstly, a DRL-based methodology is proposed for Volt/Var Control (VVC) optimization in a large distribution network, to effectively control bus voltages and reduce network power losses. Further, this thesis proposes a multi-agent (MA)DRL-based methodology under a complex regional coordinated VVC framework, and it can address spatial and temporal uncertainties. The DRL algorithm is also improved to adapt to the applications. Then, an integrated energy and heating systems (IEHS) optimization problem is solved by a MADRL-based methodology, where conventionally this could only be solved by simplifications or iterations. Beyond the applications in distribution network operation, a new electricity market service pricing method based on a DRL algorithm is also proposed. This DRL-based method has demonstrated good performance in this virtual storage rental service pricing problem, whereas this bi-level problem could hardly be solved directly due to a non-convex and non-continuous lower-level problem. These proposed methods have demonstrated advantageous performances under comprehensive case studies, and numerical simulation results have validated the effectiveness and high efficiency under different sophisticated operation conditions, solution robustness against temporal and spatial uncertainties, and optimality under large problem dimensions

ENERGY & STORAGE SHARING STRATEGIES IN AN ELECTRICITY MARKET ENVIRONMENT

The rapid growth of renewable energy generation (REG) and energy storage systems (ESS) has created a need to further develop the electricity market for distributed energy, to stimulate the technology and application of REG and battery energy storage systems (BESS). Considering that the investment cost is still high at this stage, a window of opportunity exists for the development of a sharing economy. In light of this, this thesis focuses on energy and storage sharing strategies in an electricity market environment. A distributed energy sharing strategy is proposed for a peer-to-peer (P2P) model on a microgrid. In addition, the pricing model for users in this proposed strategy has been optimised using game theory—with the Bayesian Nash Equilibrium (GM-BNE) algorithm. Based on the basic call auction trading model, the energy trading mechanism has been modified. Meanwhile, an energy sharing cloud service is proposed based on a decentralised approach, in which the cloud energy management strategy can be customised for each participant. Rigorous proofs are also given. A detailed energy storage sharing strategy of the hybrid electricity and gas energy is proposed in the distribution network, which considers the energy operation of BESS and thermal energy storage system (TESS). The techno-economic analysis based on the BESS and TESS sizing model is conducted for storage sharing between users. When considering the battery firm in the joint storage sharing strategy, a novel sharing model is proposed based on the classic per-use sharing economy business model. Rigorous mathematical proofs are given for the application of the sharing economy model to BESS, in which the sharing pricing model is validated for technical feasibility and accuracy. The proposed energy and storage sharing strategies are applicable to distributed users, in the cases of the hospitality industry and smart home. The proposed sharing strategies are also beneficial for investors, as demonstrated in the case for a battery firm. In the case of the battery firm, this per-use rental service can open new benefits. The case studies results show that the proposed energy and storage sharing strategies provide a 'win-win' situation for customers, the battery sales firm and energy networks

Risk Hedging Strategies in New Energy Markets

In recent years, two typical developments have been witnessed in the energy market. On the one hand, the penetration of renewable generations has gradually replaced parts of the traditional ways to generate energy. The intermittent nature of renewable generation can lead to energy supply uncertainty, which might exacerbate the imbalance between energy supply and demand. As a result, the problem of energy price risks might occur. On the other hand, with the introduction of distributed energy resources (DERs), new categories of markets besides traditional wholesale and retail markets are emerging. The main benefits of the penetration of DERs are threefold. First, DERs can increase power system reliability. Second, the cost of transmission can be reduced. Third, end users can directly participate in some of these new types of markets according to their energy demand, excess energy, and cost function without third-party intervention. However, energy market participants might encounter various types of uncertainties. Therefore, it is necessary to develop proper risk-hedging strategies for different energy market participants in emerging new markets. Thus, we propose risk-hedging strategies that can be used to guide various market participants to hedge risks and enhance utilities in the new energy market. These participants can be categorized into the supply side and demand side. Regarding the wide range of hedging tools analyzed in this thesis, four main types of hedging strategies are developed, including the application of ESS, financial tools, DR management, and pricing strategy. Several benchmark test systems have been applied to demonstrate the effectiveness of the proposed risk-hedging strategies. Comparative studies of existing risk hedging approaches in the literature, where applicable, have also been conducted. The real applicability of the proposed approach has been verified by simulation results