Deep Reinforcement Learning Based Energy Storage Arbitrage With Accurate Lithium-ion Battery Degradation Model

Accurate estimation of battery degradation cost is one of the main barriers for battery participating on the energy arbitrage market. This paper addresses this problem by using a model-free deep reinforcement learning (DRL) method to optimize the battery energy arbitrage considering an accurate battery degradation model. Firstly, the control problem is formulated as a Markov Decision Process (MDP). Then a noisy network based deep reinforcement learning approach is proposed to learn an optimized control policy for storage charging/discharging strategy. To address the uncertainty of electricity price, a hybrid Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) model is adopted to predict the price for the next day. Finally, the proposed approach is tested on the the historical UK wholesale electricity market prices. The results compared with model based Mixed Integer Linear Programming (MILP) have demonstrated the effectiveness and performance of the proposed framework

Techno-economic evaluation of battery storage systems in industry

In the context of a changing energy system towards one dominated by renewable energy sources, the demand for flexible energy generation and consumption will increase. Battery storage systems can provide a significant share of this energy flexibility, especially when combined with an industrial manufacturing plant to shift the industrial electricity demand over time. This paper contributes to a better understanding of the business decision when investing in a battery storage system and when marketing energy flexibility. For this purpose, the work considers the techno-economic and regulatory framework for flexibility measures and examines the optimal investment and dispatch planning for a battery storage system in an industrial company. The studies in this thesis focus on three central aspects. As a first aspect, the various revenue streams for the stored electricity are analysed and how these influence the profitability of a battery storage system. In particular, the provision of frequency containment reserve power, peak load shifting or peak shaving, arbitrage trading on the energy markets and the increase in self-consumption through photovoltaic self-generation are addressed. For this purpose, an optimisation model is formulated as a discrete, linear programme that maps the economic framework of the flexibility markets and integrates the technological constraints of the battery storage system. As a second aspect, uncertainties about market prices, load and generation behaviour are integrated into the optimisation model and the influence on the investment decision is investigated. This is done on the one hand by a two-stage robust optimisation model, which represents the uncertainty about the market success on the intraday market. On the other hand, the significance of the sequence of uncertain market decisions is illuminated through a multi-stage stochastic optimisation model. As a third aspect, the trade-off between the economic and ecological use of a battery storage system is analysed. For this purpose, an ecological, CO₂-minimal dispatch is calculated by deriving national CO₂-emission factors and compared with an economically optimal dispatch. The case studies are analysed based on real industrial load data from small, medium and large enterprises. The thesis discusses the technical and economic framework conditions, with the main focus on Germany. However, a comparison between the countries Germany, Denmark, and Croatia is also presented. The results show that peak shaving and the provision of frequency containment reserve are complementary and make the investment in a battery storage system economically viable. Self-generation through a photovoltaic system can reduce the risk arising from uncertain energy market prices. However, the sequence of uncertain decisions has a significant impact on the design of the battery storage system. Economically feasible operation through arbitrage trading, on the other hand, is not possible due to the small price differences in the markets and limitations due to battery ageing and efficiency. These battery characteristics also influence the use of a battery storage system for CO₂-reduction. Due to the limited number of cycles and relatively high charging losses, battery technology is currently unsuitable for CO₂-minimal storage use. Nevertheless, the economic and ecological potential of battery storage systems strongly depends on individual factors such as local grid charges, the selected battery technology and the individual industrial load profile. Advances in battery technology, such as increased lifetime, and possible new flexibility markets, such as dynamic grid charges, offer new application and marketing opportunities that could increase the economic viability of a battery storage system

Demand Response in Smart Grids

The Special Issue “Demand Response in Smart Grids” includes 11 papers on a variety of topics. The success of this Special Issue demonstrates the relevance of demand response programs and events in the operation of power and energy systems at both the distribution level and at the wide power system level. This reprint addresses the design, implementation, and operation of demand response programs, with focus on methods and techniques to achieve an optimized operation as well as on the electricity consumer

Energy Management of Prosumer Communities

The penetration of distributed generation, energy storages and smart loads has resulted in the emergence of prosumers: entities capable of adjusting their electricity production and consumption in order to meet environmental goals and to participate profitably in the available electricity markets. Significant untapped potential remains in the exploitation and coordination of small and medium-sized distributed energy resources. However, such resources usually have a primary purpose, which imposes constraints on the exploitation of the resource; for example, the primary purpose of an electric vehicle battery is for driving, so the battery could be used as temporary storage for excess photovoltaic energy only if the vehicle is available for driving when the owner expects it to be. The aggregation of several distributed energy resources is a solution for coping with the unavailability of one resource. Solutions are needed for managing the electricity production and consumption characteristics of diverse distributed energy resources in order to obtain prosumers with more generic capabilities and services for electricity production, storage, and consumption. This collection of articles studies such prosumers and the emergence of prosumer communities. Demand response-capable smart loads, battery storages and photovoltaic generation resources are forecasted and optimized to ensure energy-efficient and, in some cases, profitable operation of the resources

Passenger-Centric Urban Air Mobility: Fairness Trade-Offs and Operational Efficiency

Urban Air Mobility (UAM) has the potential to revolutionize transportation. It will exploit the third dimension to help smooth ground traffic in densely populated areas. To be successful, it will require an integrated approach able to balance efficiency and safety while harnessing common resources and information. In this work we focus on future urban air-taxi services, and present the first methods and algorithms to efficiently operate air-taxi at scale. Our approach is twofold. First, we use a passenger-centric perspective which introduces traveling classes, and information sharing between transport modes to differentiate quality of services. This helps smooth multimodal journeys and increase passenger satisfaction. Second, we provide a flight routing and recharging solution which minimizes direct operational costs while preserving long term battery life through reduced energy-intense recharging. Our methods, which surpass the performance of a general state-of-the-art commercial solver, are also used to gain meaningful insights on the design space of the air-taxi problem, including solutions to hidden fairness issues.Comment: Submitted to Transportation Research Part C: Emerging Technologie

Optimization-based framework for low-voltage grid reinforcement assessment under various levels of flexibility and coordination

The rapid electrification of residential heating and mobility sectors is expected to drive the existing distribution grid assets beyond their planned operating conditions. This change will also reveal new potentials through sector coupling, flexibilities, and the local exchange of decentralized generation. This paper thus presents an optimization framework for multi-modal energy systems at the low voltage (LV) distribution grid level. In this, we focus on the reinforcement requirements of the grid and the techno-economic assessment of flexibility components and coordination between agents. By employing a multi-level approach, computational complexity is reduced, and various levels of coordination and flexibilities are implemented. We conclude the work with a case study for a representative rural grid in Germany, in which we observe high economic potential in the flexible operation of buildings, majorly thanks to better integration of photovoltaics. Across all paradigms barring a best-case benchmark, grid reinforcement based on a mix of passive and active measures was necessary. A synergy effect is observed between flexibilities and coordination, as their combination reduces the peaks to the extent of completely avoiding grid reinforcement. The presented framework can be applied with a wide range of grid and component types to outline the broad landscape of future LV distribution grids

Energy storage systems and grid code requirements for large-scale renewables integration in insular grids

This thesis addresses the topic of energy storage systems supporting increased penetration of renewables in insular systems. An overview of energy storage management, forecasting tools and demand side solutions is carried out, comparing the strategic utilization of storage and other competing strategies. Particular emphasis is given to energy storage systems on islands, as a new contribution to earlier studies, addressing their particular requirements, the most appropriate technologies and existing operating projects throughout the world. Several real-world case studies are presented and discussed in detail. Lead-acid battery design parameters are assessed for energy storage applications on insular grids, comparing different battery models. The wind curtailment mitigation effect by means of energy storage resources is also explored. Grid code requirements for large-scale integration of renewables are discussed in an island context, as another new contribution to earlier studies. The current trends on grid code formulation, towards an improved integration of distributed renewable resources in island systems, are addressed. Finally, modeling and control strategies with energy storage systems are addressed. An innovative energy management technique to be used in the day-ahead scheduling of insular systems with Vanadium Redox Flow battery is presented.Esta tese aborda a temática dos sistemas de armazenamento de energia visando o aumento da penetração de energias renováveis em sistemas insulares. Uma visão geral é apresentada acerca da gestão do armazenamento de energia, ferramentas de previsão e soluções do lado da procura de energia, comparando a utilização estratégica do armazenamento e outras estratégias concorrentes. É dada ênfase aos sistemas de armazenamento de energia em ilhas, como uma nova contribuição no estado da arte, abordando as suas necessidades específicas, as tecnologias mais adequadas e os projetos existentes e em funcionamento a nível mundial. Vários casos de estudos reais são apresentados e discutidos em detalhe. Parâmetros de projeto de baterias de chumbo-ácido são avaliados para aplicações de armazenamento de energia em redes insulares, comparando diferentes modelos de baterias. O efeito de redução do potencial de desperdício de energia do vento, recorrendo ao armazenamento de energia, também é perscrutado. As especificidades subjacentes aos códigos de rede para a integração em larga escala de energias renováveis são discutidas em contexto insular, sendo outra nova contribuição no estado da arte. As tendências atuais na elaboração de códigos de rede, no sentido de uma melhor integração da geração distribuída renovável em sistemas insulares, são abordadas. Finalmente, é estudada a modelação e as estratégias de controlo com sistemas de armazenamento de energia. Uma metodologia de gestão de energia inovadora é apresentada para a exploração de curto prazo de sistemas insulares com baterias de fluxo Vanádio Redox