Computing an Optimal Control Policy for an Energy Storage

We introduce StoDynProg, a small library created to solve Optimal Control problems arising in the management of Renewable Power Sources, in particular when coupled with an Energy Storage System. The library implements generic Stochastic Dynamic Programming (SDP) numerical methods which can solve a large class of Dynamic Optimization problems. We demonstrate the library capabilities with a prototype problem: smoothing the power of an Ocean Wave Energy Converter. First we use time series analysis to derive a stochastic Markovian model of this system since it is required by Dynamic Programming. Then, we briefly describe the "policy iteration" algorithm we have implemented and the numerical tools being used. We show how the API design of the library is generic enough to address Dynamic Optimization problems outside the field of Energy Management. Finally, we solve the power smoothing problem and compare the optimal control with a simpler heuristic control.Comment: Part of the Proceedings of the 6th European Conference on Python in Science (EuroSciPy 2013), Pierre de Buyl and Nelle Varoquaux editors, (2014

A state-of-the-art review on torque distribution strategies aimed at enhancing energy efficiency for fully electric vehicles with independently actuated drivetrains

© 2019, Levrotto and Bella. All rights reserved. Electric vehicles are the future of private passenger transportation. However, there are still several technological barriers that hinder the large scale adoption of electric vehicles. In particular, their limited autonomy motivates studies on methods for improving the energy efficiency of electric vehicles so as to make them more attractive to the market. This paper provides a concise review on the current state-of-the-art of torque distribution strategies aimed at enhancing energy efficiency for fully electric vehicles with independently actuated drivetrains (FEVIADs). Starting from the operating principles, which include the "control allocation" problem, the peculiarities of each proposed solution are illustrated. All the existing techniques are categorized based on a selection of parameters deemed relevant to provide a comprehensive overview and understanding of the topic. Finally, future concerns and research perspectives for FEVIAD are discussed

Optimization of Battery Energy Storage to Improve Power System Oscillation Damping

A placement problem for multiple Battery Energy Storage System (BESS) units is formulated towards power system transient voltage stability enhancement in this paper. The problem is solved by the Cross-Entropy (CE) optimization method. A simulation-based approach is adopted to incorporate higher-order dynamics and nonlinearities of generators and loads. The objective is to maximize the voltage stability index, which is setup based on certain grid-codes. Formulations of the optimization problem are then discussed. Finally, the proposed approach is implemented in MATLAB/DIgSILENT and tested on the New England 39-Bus system. Results indicate that installing BESS units at the optimized location can alleviate transient voltage instability issue compared with the original system with no BESS. The CE placement algorithm is also compared with the classic PSO (Particle Swarm Optimization) method, and its superiority is demonstrated in terms of a faster convergence rate with matched solution qualities.Comment: This paper has been accepted by IEEE Transactions on Sustainable Energy and now still in online-publication phase, IEEE Transactions on Sustainable Energy. 201

Heuristic Dynamic Programming for Adaptive Virtual Synchronous Generators

In this paper a neural network heuristic dynamic programing (HDP) is used for optimal control of the virtual inertia based control of grid connected three phase inverters. It is shown that the conventional virtual inertia controllers are not suited for non inductive grids. A neural network based controller is proposed to adapt to any impedance angle. Applying an adaptive dynamic programming controller instead of a supervised controlled method enables the system to adjust itself to different conditions. The proposed HDP consists of two subnetworks, critic network and action network. These networks can be trained during the same training cycle to decrease the training time. The simulation results confirm that the proposed neural network HDP controller performs better than the traditional direct fed voltage and reactive power controllers in virtual inertia control schemes.Comment: NAPS 2019 Conference. arXiv admin note: substantial text overlap with arXiv:1908.05191; text overlap with arXiv:1908.0519

A reinforcement learning based discrete supplementary control for power system transient stability enhancement

peer reviewedThis paper proposes an application of a Reinforcement Learning (RL) method to the control of a dynamic brake aimed to enhance power system transient stability. The control law of the resistive brake is in the form of switching strategies. In particular, the paper focuses on the application of a model based RL method, known as prioritized sweeping, a method proven to be suitable in applications in which computation is considered to be cheap. The curse of dimensionality problem is resolved by the system state dimensionality reduction based on the One Machine Infinite Bus (OMIB) transformation. Results obtained by using a synthetic four-machine power system are given to illustrate the performances of the proposed methodology

Swarm Intelligence Based Multi-phase OPF For Peak Power Loss Reduction In A Smart Grid

Recently there has been increasing interest in improving smart grids efficiency using computational intelligence. A key challenge in future smart grid is designing Optimal Power Flow tool to solve important planning problems including optimal DG capacities. Although, a number of OPF tools exists for balanced networks there is a lack of research for unbalanced multi-phase distribution networks. In this paper, a new OPF technique has been proposed for the DG capacity planning of a smart grid. During the formulation of the proposed algorithm, multi-phase power distribution system is considered which has unbalanced loadings, voltage control and reactive power compensation devices. The proposed algorithm is built upon a co-simulation framework that optimizes the objective by adapting a constriction factor Particle Swarm optimization. The proposed multi-phase OPF technique is validated using IEEE 8500-node benchmark distribution system.Comment: IEEE PES GM 2014, Washington DC, US

A Heuristic Dynamic Programming Based Power System Stabilizer for a Turbogenerator in a Single Machine Power System

Power system stabilizers (PSS) are used to generate supplementary control signals for the excitation system in order to damp the low frequency power system oscillations. To overcome the drawbacks of conventional PSS (CPSS), numerous techniques have been proposed in the literature. Based on the analysis of existing techniques, a novel design of power system stabilizer (PSS) based on heuristic dynamic programming (HDP) is proposed in this paper. HDP combining the concepts of dynamic programming and reinforcement learning is used in the design of a nonlinear optimal power system stabilizer. The proposed HDP based PSS is evaluated against the conventional power system stabilizer and indirect adaptive neurocontrol based PSS under small and large disturbances in a single machine infinite bus power system setup. Results are presented to show the effectiveness of this new technique

A Heuristic-Dynamic-programming-Based Power System Stabilizer for a Turbogenerator in a Single-Machine Power System

Power system stabilizers (PSSs) are used to generate supplementary control signals for the excitation system in order to damp the low-frequency power system oscillations. To overcome the drawbacks of a conventional PSS (CPSS), numerous techniques have been proposed in the literature. Based on the analysis of existing techniques, a novel design based on heuristic dynamic programming (HDP) is presented in this paper. HDP, combining the concepts of dynamic programming and reinforcement learning, is used in the design of a nonlinear optimal power system stabilizer. Results show the effectiveness of this new technique. The performance of the HDP-based PSS is compared with the CPSS and the indirect-adaptive-neurocontrol-based PSS under small and large disturbances. In addition, the impact of different discount factors in the HDP PSS\u27s performance is presented

Implementation of Adaptive Critic-Based Neurocontrollers for Turbogenerators in a Multimachine Power System

This paper presents the design and practical hardware implementation of optimal neurocontrollers that replace the conventional automatic voltage regulator (AVR) and the turbine governor of turbogenerators on multimachine power systems. The neurocontroller design uses a powerful technique of the adaptive critic design (ACD) family called dual heuristic programming (DHP). The DHP neurocontroller\u27s training and testing are implemented on the Innovative Integration M67 card consisting of the TMS320C6701 processor. The measured results show that the DHP neurocontrollers are robust and their performance does not degrade unlike the conventional controllers even when a power system stabilizer (PSS) is included, for changes in system operating conditions and configurations. This paper also shows that it is possible to design and implement optimal neurocontrollers for multiple turbogenerators in real time, without having to do continually online training of the neural networks, thus avoiding risks of instability