Electrical and Electronic Engineering, Imperial College London
Doi
Abstract
Demand side response are proposed to incentivise customers to shift their electricity usage from peak demand periods to off-peak demand periods and to curtail their electricity usage during peak demand periods, which show great potential to reduce the peak loads, electricity prices, customers’ bills and further stabilize the power systems. The investigation of this effect on the pricing strategies and the profits of electricity retailers has recently emerged as a highly interesting research area. However, the state-of-the-art, bi-level optimization modelling approach makes the unrealistic assumption that retailers treat wholesale market prices as exogenous, fixed parameters.
On the other hand, distributed energy resources (DER) in electricity markets are proposed to bring the significant operating flexibility which can support system balancing and reduce demand peaks, thereby limiting the balancing costs of conventional generators and the investments costs of new generation and network assets. And, local energy markets (LEM) have recently attracted great interest as they enable effective coordination of small-scale DER at the customer side, and avoidance of distribution network reinforcements. However, the introduction of LEM has also significant implications on the strategic interactions between the customers and incumbent electricity retailers, which has not been explored.
Furthermore, a specific demand response technology of electric vehicles (EV) exhibits the potential to support system balancing and limit demand peaks, thus improving significantly the cost-effectiveness of low-carbon electricity systems. And the effective pricing of EV charging by aggregators constitutes a key problem towards the realization of the significant EV flexibility potential in deregulated electricity systems and has been addressed by previous work through bi-level optimization formulations. However, the solution approach adopted in previous work cannot capture the discrete nature of the EV charging / discharging levels. Furthermore, aggregators suffering from communication and privacy limitations are hard to acquire the perfect knowledge of EV operating characteristics and traveling patterns.
Given such a context, this thesis aims at addressing the above challenges and proposing strategic retail pricing-based energy response programs to study the interactions between the electricity retailer / aggregator and its served flexible customers / EV based on game theoretic modeling and learning based approaches. We conduct the research in three different application scenarios:
1) This thesis proposes a novel bi-level optimization problem which represents endogenously the wholesale market clearing process as an additional lower-level problem, thus capturing the realistic implications of a retailer’s pricing strategies and the resulting demand response on the wholesale market prices. This bi-level optimization problem is solved through converting it to a single-level Mathematical Programs with Equilibrium Constraints (MPEC). The scope of the examined case studies is threefold. First of all, they demonstrate the interactions between the retailer, the flexible consumers and the wholesale market and analyse the fundamental effects of the consumers’ time-shifting flexibility on the retailer’s revenue from the consumers, its cost in the wholesale market, and its overall profit. Furthermore, they analyse how these effects of demand flexibility depend on the retailer’s relative size in the market and the strictness of the regulatory framework. Finally, they highlight the added value of the proposed bi-level model by comparing its outcomes against the state-of-the-art bi-level modelling approach.
2) This thesis explores for the first time the interaction between electricity retailer and LEM by proposing a novel bi-level optimization problem, which captures the pricing decisions of a strategic retailer in the upper-level problem and the response of both independent customers and the LEM (both including flexible consumers, micro- generators and energy storages) in the lower-level problems. Since the lower-level problem representing the LEM is non-convex, a new analytical approach is employed for solving the developed bi-level optimization problem. The examined case studies demonstrate that the introduction of an LEM reduces the customers’ energy dependency on the retailer and limits the retailer’s strategic potential of exploiting the customers through large differentials between buy and sell prices. As a result, the profit of the retailer is significantly reduced while the customers, primarily the LEM participants and to a lower extent non-participating customer, achieve significant economic benefits.
3) This thesis proposes a reinforcement learning (RL) method that the EV aggregator gradually learns how to improve its pricing strategies by utilizing experiences acquired from its repeated interactions with the EV and the wholesale market. Although RL can tackle the challenge of imperfect information and MPEC reformulation, the state-of-the- art RL methods require discretization of state and / or action spaces and thus exhibit limitations in terms of solution optimality and computational requirements. This thesis proposes a novel deep reinforcement learning (DRL) method to solve the examined EV pricing problem, combining deep deterministic policy gradient (DDPG) principles with a prioritized experience replay (PER) strategy, and setting up the problem in multi-dimensional continuous state and action spaces. Case studies demonstrate that the proposed method outperforms state-of-the-art RL methods in terms of both solution optimality and computational requirements, and comprehensively analyze the economic impacts of smart-charging and vehicle-to-grid (V2G) flexibility on both aggregators and EV owners.Open Acces
