Optimal scheduling for charging and discharging of electric vehicles based on deep reinforcement learning

The growing scale of electric vehicles (EVs) brings continuous challenges to the energy trading market. In the process of grid-connected charging of EVs, disorderly charging behavior of a large number of EVs will have a substantial impact on the grid load. Aiming to solve the problem of optimal scheduling for charging and discharging of EVs, this paper first establishes a model for the charging and discharging scheduling of EVs involving the grid, charging equipment, and EVs. Then, the established scheduling model is described as a partially observable Markov decision process (POMDP) in the multi-agent environment. This paper proposes an optimization objective that comprehensively considers various factors such as the cost of charging and discharging EVs, grid load stability, and user usage requirements. Finally, this paper introduces the long short-term memory enhanced multi-agent deep deterministic policy gra dient (LEMADDPG) algorithm to obtain the optimal scheduling strategy of EVs. Simulation results prove that the proposed LEMADDPG algorithm can obtain the fastest convergence speed, the smallest fluctuation and the highest cumulative reward compared with traditional deep deterministic policy gradient and DQN algorithms

Incentive Design for Direct Load Control Programs

We study the problem of optimal incentive design for voluntary participation of electricity customers in a Direct Load Scheduling (DLS) program, a new form of Direct Load Control (DLC) based on a three way communication protocol between customers, embedded controls in flexible appliances, and the central entity in charge of the program. Participation decisions are made in real-time on an event-based basis, with every customer that needs to use a flexible appliance considering whether to join the program given current incentives. Customers have different interpretations of the level of risk associated with committing to pass over the control over the consumption schedule of their devices to an operator, and these risk levels are only privately known. The operator maximizes his expected profit of operating the DLS program by posting the right participation incentives for different appliance types, in a publicly available and dynamically updated table. Customers are then faced with the dynamic decision making problem of whether to take the incentives and participate or not. We define an optimization framework to determine the profit-maximizing incentives for the operator. In doing so, we also investigate the utility that the operator expects to gain from recruiting different types of devices. These utilities also provide an upper-bound on the benefits that can be attained from any type of demand response program.Comment: 51st Annual Allerton Conference on Communication, Control, and Computing, 201

Optimal charging strategy of electric vehicles customers in a smart electrical car park

© 2016, Institution of Engineering and Technology. All rights reserved. A smart electrical car park with electric vehicles (EVs) parking there, regarded as a short-term storage system, could minimize the costs of EV customers and improve the main grid stability simultaneously. This system, including numerous bidirectional AC/DC converters, a local energy storage unit and a monitoring room, is firstly established. As the hourly prices of electricity fluctuating with time, EV owners would like to charge energy from the main grid during the low-price periods to save money, while discharging energy to the main grid during high-price periods. In order to achieve this, an optimal charging scheme is proposed to determine the charging rate of each EV based on the fluctuation of hourly prices and requirements of customers. Thus, this charging/discharging strategy can reduce the costs for EV owners and help keep the balance of supply and demand for the main grid. A comparison between the cost of EVs customers with and without the developed smart charging/discharging strategy in this smart electrical car park is presented and analysed in Matlab with an optimization problem solver named Cplex in this study. It is demonstrated that the proposed charging/discharging strategy can not only reduce EV owner's cost but also improve the main grid stability as well

Estimating the Benefits of Electric Vehicle Smart Charging at Non-Residential Locations: A Data-Driven Approach

In this paper, we use data collected from over 2000 non-residential electric vehicle supply equipments (EVSEs) located in Northern California for the year of 2013 to estimate the potential benefits of smart electric vehicle (EV) charging. We develop a smart charging framework to identify the benefits of non-residential EV charging to the load aggregators and the distribution grid. Using this extensive dataset, we aim to improve upon past studies focusing on the benefits of smart EV charging by relaxing the assumptions made in these studies regarding: (i) driving patterns, driver behavior and driver types; (ii) the scalability of a limited number of simulated vehicles to represent different load aggregation points in the power system with different customer characteristics; and (iii) the charging profile of EVs. First, we study the benefits of EV aggregations behind-the-meter, where a time-of-use pricing schema is used to understand the benefits to the owner when EV aggregations shift load from high cost periods to lower cost periods. For the year of 2013, we show a reduction of up to 24.8% in the monthly bill is possible. Then, following a similar aggregation strategy, we show that EV aggregations decrease their contribution to the system peak load by approximately 40% when charging is controlled within arrival and departure times. Our results also show that it could be expected to shift approximately 0.25kWh (~2.8%) of energy per non-residential EV charging session from peak periods (12PM-6PM) to off-peak periods (after 6PM) in Northern California for the year of 2013.Comment: Pre-print, under review at Applied Energ