A double-deck deep reinforcement learning-based energy dispatch strategy for an integrated electricity and district heating system embedded with thermal inertial and operational flexibility

With the high penetration of wind power connected to the integrated electricity and district heating systems (IEDHSs), wind power curtailment still inevitably occurs in the traditional IEDHS dispatch. Focusing on the flexibilities of the IEDHS is considered to be a beneficial solution to further promote the integration of wind power. In the district heating network, the thermal inertia is utilized to improve such flexibility. Therefore, an IEDHS dispatch model considering the thermal inertia of district heating network and operational flexibility of generators is proposed in this paper. In addition, to avoid the tendency of traditional reinforcement learning (RL) to fall into local optimality when solving high-dimensional problems, a double-deck deep RL (D3RL) framework is proposed in this study. D3RL combines with a deep deterministic policy gradient (DDPG) agent in the upper level and a conventional optimization solver in the lower level to simplify the action and reward design. In the simulation, the proposed model considering the transmission time delay characteristics of the district heating network and the operational flexibility of generators is verified in four scheduling scenarios. Besides, the superiority of the proposed D3RL method is validated in a larger IEDHS. Numerical results show that the considered scheduling model can use the heat storage characteristics of heating pipelines, reduce operating costs, improve the operational flexibility and encourage wind power utilization. Compared with traditional RL, the proposed optimization method can improve its training speed and convergence performance.Ministry of Education (MOE)Nanyang Technological UniversityPublished versionThis work was supported by the School of Electrical and Electronic Engineering at Nanyang Technological University, Ministry of Education, Singapore, under Grant AcRF TIER 1 RG50/21

Coordinated carbon capture systems and power-to-gas dynamic economic energy dispatch strategy for electricity–gas coupled systems considering system uncertainty:An improved soft actor–critic approach

Due to uncertainties in renewable energy generation and load demands, traditional energy dispatch schemes for an integrated electricity–gas system (IEGS) considerably depend on explicit forecast mathematical models. In this study, a novel data-driven deep reinforcement learning method is applied to solve the IEGS dynamic dispatch problem with the targets of minimizing carbon emission and operating cost. Moreover, a flexible operation of carbon capture system and power-to-gas facility is proposed to attain low operating costs. The IEGS dynamic dispatch problem is formulated as a Markov game, and a soft actor–critic (SAC) algorithm is applied to learn the optimal dispatch solution. To improve training efficiency and convergence, prioritized experience replay (PER) is employed. In the simulation, the proposed PER–SAC algorithm compared with deep Q-network and SAC has fast and stable learning performance. In contrast to a modified sequential quadratic programming based on uncertainty prediction, the proposed method can reduce the target cost by 11.62% when the prediction error exceeds 10%. The computational time of scenario analysis solution on the same hardware platform is 4.58 times than that of training the PER–SAC method. Finally, the simulation results under different scenarios demonstrate that the PER–SAC-based dispatch strategy has satisfactory generalization and adaptability

Low-frequency voltage ripples in the flying capacitors of the nested neutral-point-clamped converter

The flying capacitors (FCs) of the nested neutral-point-clamped (NNPC) converter show an inherent voltage ripple at fundamental frequency. This ripple can be significantly large under some operating conditions of the converter. In this paper, the amplitudes of the low-frequency voltage ripples in the FCs are determined. An averaged model of the NNPC converter is introduced and used in the analysis. The amplitudes of the capacitor voltage ripples are provided using normalized variables so that this information can be used to size the FCs of the converter in different applications. The results of the analysis are validated experimentally in a laboratory prototype

Novel Data-Driven decentralized coordination model for electric vehicle aggregator and energy hub entities in multi-energy system using an improved multi-agent DRL approach

Energy hub (EH) is an independent entity that benefits to the efficiency, flexibility, and reliability of integrated energy systems (IESs). On the other hand, the rapid emerging of electric vehicles (EVs) drives the EV aggregator (EVAGG) as another independent entity to facilitate the electricity exchange with the grid. However, due to privacy consideration for different owners, it is challenging to investigate the optimal coordinated strategies for such interconnected entities only by exchanging the information of electrical energy. Besides, the existence of parameter uncertainties (load demands, EVs’ charging behaviors, wind power and photovoltaic generation), continuous decision space, dynamic energy flows, and non-convex multi-objective function is difficult to solve. To this end, this paper proposes a novel model-free multi-agent deep reinforcement learning (MADRL) -based decentralized coordination model to minimize the energy costs of EH entities and maximize profits of EVAGGs. First, a long short-term memory (LSTM) module is used to capture the future trend of uncertainties. Then, the coordination problem is formulated as Markov games and solved by the attention enabled MADRL algorithm, where the EH or EVAGG entity is modeled as an adaptive agent. An attention mechanism makes each agent only focus on state information related to the reward. The proposed MADRL adopts the forms of offline centralized training to learn the optimal coordinated control strategy, and decentralized execution to enable agents’ online decisions to only require local measurements. A safety network is employed to cope with equality constraints (demand–supply balance). Simulation results illustrate that the proposed method achieves similar results compared to the traditional model-based method with perfect knowledge of system models, and the computation performance is at least two orders of magnitudes shorter than the traditional method. The testing results of the proposed method are better than those of the Concurrent and other MADRL method, with 10.79%/3.06% lower energy cost and 17.11%/6.82% higher profits of aggregator. Besides, the electric equality constraint of the proposed method is only 0.25 MW averaged per day, which is a small and acceptable violation

Reliability Worth Analysis of Distribution Systems Using Cascade Correlation Neural Networks

Reliability worth analysis is of great importance in the area of distribution network planning and operation. The reliability worth’s precision can be affected greatly by the customer interruption cost model used. The choice of the cost models can change system and load point reliability indices. In this study, a cascade correlation neural network is adopted to further develop two cost models comprising a probabilistic distribution model and an average or aggregate model. A contingency-based analytical technique is adopted to conduct the reliability worth analysis. Furthermore, the possible effects of adding distributed generation units into the network are evaluated. The proposed approach has been tested on a radial distribution test network evaluating the reliability worth. The results show that the probabilistic distribution model provides a more realistic model for the reliability analysis