Mean Field Game-Based Control of Dispersed Energy Storage Devices with Constrained Inputs

RÉSUMÉ Avec plus de production d’énergie renouvelable (éoliennes, panneaux solaires) connectés au réseau électrique, il devient important d’équilibrer la charge et la production alors que la variabilité est un problème pour les ressources renouvelables. On a étudié les dispositifs de stockage d’énergie, en particulier ceux qui sont naturellement présents au réseau électrique (tels que les chauffe-eau électriques, les chauffages électriques, etc. dans les ménages) pourraient devenir des sources potentielles pour atténuer ce problème de variabilité. Afin de minimiser le besoin de communication et les efforts de calcul, un mécanisme de contrôle décentralisé est développé dans cette recherche pour contrôler la trajectoire de champ moyen d’une grande population de chauffages électriques pour suivre (ou atteindre) une cible de température. Le contrôle optimal pour chaque chauffage ou joueur est localement calculé et respecte certaines contraintes. Le problème de contrôle est formulé comme un problème de commande linéaire quadratique (LQ) avec des entrées contraintes dans la théorie du jeu de champ moyen, en anglais, Mean Field Game (MFG). Le mécanisme de contrôle décentralisé est développé à partir de la solution d’équilibre du modèle, cette dernière est obtenue lorsque chaque joueur individuel choisit la meilleure réponse au champ moyen basé sur l’état global de la population. En choisissant les stratégies optimales, tous les joueurs reproduisissent collectivement le champ moyen de la population. En ce sens, la solution d’équilibre est un point fixe du système. Dans une configuration de MFG, les joueurs sont faiblement couplés, ce qui signifie qu’un joueur a une influence négligeable sur le système global tandis que le comportement de la population entière a un effet significatif sur chaque joueur individuel.----------ABSTRACT With the increasing levels of renewable power generation (wind turbines, solar panels) connected to power grids, it is becoming important to balance the load and the generation while variability issue exists for renewable resources. It has been studied that energy storage devices, in particular those naturally present in the power system (such as electric water heaters, electric space heaters, etc. in households), can become potential sources to help mitigate such variability. In order to minimize the communication bandwidth and computation efforts, a decentralized control mechanism is developed in this research to shape the aggregate load profile of a large population of electric space heaters in a power system. Under the control mechanism, the mean-field trajectory (temperature) of the load population is controlled to follow a target temperature, and the control input for each device is generated locally and must respect certain constraint. We formulate the problem as a linear quadratic (LQ) tracking problem with constrained inputs under the mean field game (MFG) settings. The decentralized mechanism is then derived based on the equilibrium solution to the formulated model. The equilibrium solution is obtained when each individual agent of the game chooses a best response to the so-called mass effect of the population via couplings of their individual dynamics and cost functions; when all agents choose the optimal strategies, collectively the mass effect should be replicated. In this sense, the equilibrium solution is a fixed point of the system, and can be mathematically characterized by a mean-field (MF) fixed point equation system. In such MFG setup, agents are weakly coupled, meaning that an agent has a negligible influence on the overall system while the mean-field behavior of the population has a significant effect on any individual agent

Mean-Field-Type Games in Engineering

A mean-field-type game is a game in which the instantaneous payoffs and/or the state dynamics functions involve not only the state and the action profile but also the joint distributions of state-action pairs. This article presents some engineering applications of mean-field-type games including road traffic networks, multi-level building evacuation, millimeter wave wireless communications, distributed power networks, virus spread over networks, virtual machine resource management in cloud networks, synchronization of oscillators, energy-efficient buildings, online meeting and mobile crowdsensing.Comment: 84 pages, 24 figures, 183 references. to appear in AIMS 201

Local Water Storage Control for the Developing World

Most cities in India do not have water distribution networks that provide water throughout the entire day. As a result, it is common for homes and apartment buildings to utilize water storage systems that are filled during a small window of time in the day when the water distribution network is active. However, these water storage systems do not have disinfection capabilities, and so long durations of storage (i.e., as few as four days) of the same water leads to substantial increases in the amount of bacteria and viruses in that water. This paper considers the stochastic control problem of deciding how much water to store each day in the system, as well as deciding when to completely empty the water system, in order to tradeoff: the financial costs of the water, the health costs implicit in long durations of storing the same water, the potential for a shortfall in the quantity of stored versus demanded water, and water wastage from emptying the system. To solve this problem, we develop a new Binary Dynamic Search (BiDS) algorithm that is able to use binary search in one dimension to compute the value function of stochastic optimal control problems with controlled resets to a single state and with constraints on the maximum time span in between resets of the system

Energy Aware Algorithms for managing Wireless Sensor Networks

While the majority of the current Wireless Sensor Networks (WSNs) research has prioritized either the coverage of the monitored area or the energy efficiency of the network, it is clear that their relationship must be further studied in order to find optimal solutions that balance the two factors. Higher degrees of redundancy can be attained by increasing the number of active sensors monitoring a given area which results in better performance. However, this in turn increases the energy being consumed. In our research, we focus on attaining a solution that considers several optimization parameters such as the percentage of coverage, quality of coverage and energy consumption. The problem is modeled using a bipartite graph and employs an evolutionary algorithm to handle the activation and deactivation of the sensors. An accelerated version of the algorithm is also presented; this algorithm attempts to cleverly mutate the string being considered after analyzing the desired output conditions and performs a calculated crossover depending on the fitness of the parent strings. This results in a quicker convergence and a considerable reduction in the search time for attaining the desired solutions. Proficient cluster formation in wireless sensor networks reduces the total energy consumed by the network and prolongs the life of the network. There are various clustering approaches proposed, depending on the application and the objective to be attained. There are situations in which sensors are randomly dispersed over the area to be monitored. In our research, we also propose a solution for such scenarios using heterogeneous networks where a network has to self-organize itself depending on the physical allocations of sensors, cluster heads etc. The problem is modeled using a multi-stage graph and employs combinatorial algorithms to determine which cluster head a particular sensor would report to and which sink node a cluster head would report to. The solution proposed provides flexibility so that it can be applied to any network irrespective of density of resources deployed in the network. Finally we try to analyze how the modification of the sequence of execution of the two methods modifies the results. We also attempt to diagnose the reasons responsible for it and conclude by highlighting the advantages of each of the sequence

Enhancing Cyber-Resiliency of DER-based SmartGrid: A Survey

The rapid development of information and communications technology has enabled the use of digital-controlled and software-driven distributed energy resources (DERs) to improve the flexibility and efficiency of power supply, and support grid operations. However, this evolution also exposes geographically-dispersed DERs to cyber threats, including hardware and software vulnerabilities, communication issues, and personnel errors, etc. Therefore, enhancing the cyber-resiliency of DER-based smart grid - the ability to survive successful cyber intrusions - is becoming increasingly vital and has garnered significant attention from both industry and academia. In this survey, we aim to provide a systematical and comprehensive review regarding the cyber-resiliency enhancement (CRE) of DER-based smart grid. Firstly, an integrated threat modeling method is tailored for the hierarchical DER-based smart grid with special emphasis on vulnerability identification and impact analysis. Then, the defense-in-depth strategies encompassing prevention, detection, mitigation, and recovery are comprehensively surveyed, systematically classified, and rigorously compared. A CRE framework is subsequently proposed to incorporate the five key resiliency enablers. Finally, challenges and future directions are discussed in details. The overall aim of this survey is to demonstrate the development trend of CRE methods and motivate further efforts to improve the cyber-resiliency of DER-based smart grid.Comment: Submitted to IEEE Transactions on Smart Grid for Publication Consideratio

The Role of Deep Learning in Advancing Proactive Cybersecurity Measures for Smart Grid Networks: A Survey

As smart grids (SG) increasingly rely on advanced technologies like sensors and communication systems for efficient energy generation, distribution, and consumption, they become enticing targets for sophisticated cyberattacks. These evolving threats demand robust security measures to maintain the stability and resilience of modern energy systems. While extensive research has been conducted, a comprehensive exploration of proactive cyber defense strategies utilizing Deep Learning (DL) in {SG} remains scarce in the literature. This survey bridges this gap, studying the latest DL techniques for proactive cyber defense. The survey begins with an overview of related works and our distinct contributions, followed by an examination of SG infrastructure. Next, we classify various cyber defense techniques into reactive and proactive categories. A significant focus is placed on DL-enabled proactive defenses, where we provide a comprehensive taxonomy of DL approaches, highlighting their roles and relevance in the proactive security of SG. Subsequently, we analyze the most significant DL-based methods currently in use. Further, we explore Moving Target Defense, a proactive defense strategy, and its interactions with DL methodologies. We then provide an overview of benchmark datasets used in this domain to substantiate the discourse.{ This is followed by a critical discussion on their practical implications and broader impact on cybersecurity in Smart Grids.} The survey finally lists the challenges associated with deploying DL-based security systems within SG, followed by an outlook on future developments in this key field.Comment: To appear in the IEEE internet of Things journa

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Control and Optimization of Energy Storage in AC and DC Power Grids

Energy storage attracts attention nowadays due to the critical role it will play in the power generation and transportation sectors. Electric vehicles, as moving energy storage, are going to play a key role in the terrestrial transportation sector and help reduce greenhouse emissions. Bulk hybrid energy storage will play another critical role for feeding the new types of pulsed loads on ship power systems. However, to ensure the successful adoption of energy storage, there is a need to control and optimize the charging/discharging process, taking into consideration the customer preferences and the technical aspects. In this dissertation, novel control and optimization algorithms are developed and presented to address the various challenges that arise with the adoption of energy storage in the electricity and transportation sectors. Different decentralized control algorithms are proposed to manage the charging of a mass number of electric vehicles connected to different points of charging in the power distribution system. The different algorithms successfully satisfy the preferences of the customers without negatively impacting the technical constraints of the power grid. The developed algorithms were experimentally verified at the Energy Systems Research Laboratory at FIU. In addition to the charge control of electric vehicles, the optimal allocation and sizing of commercial parking lots are considered. A bi-layer Pareto multi-objective optimization problem is formulated to optimally allocate and size a commercial parking lot. The optimization formulation tries to maximize the profits of the parking lot investor, as well as minimize the losses and voltage deviations for the distribution system operator. Sensitivity analysis to show the effect of the different objectives on the selection of the optimal size and location is also performed. Furthermore, in this dissertation, energy management strategies of the onboard hybrid energy storage for a medium voltage direct current (MVDC) ship power system are developed. The objectives of the management strategies were to maintain the voltage of the MVDC bus, ensure proper power sharing, and ensure proper use of resources, where supercapacitors are used during the transient periods and batteries are used during the steady state periods. The management strategies were successfully validated through hardware in the loop simulation

Self-Evaluation Applied Mathematics 2003-2008 University of Twente

This report contains the self-study for the research assessment of the Department of Applied Mathematics (AM) of the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) at the University of Twente (UT). The report provides the information for the Research Assessment Committee for Applied Mathematics, dealing with mathematical sciences at the three universities of technology in the Netherlands. It describes the state of affairs pertaining to the period 1 January 2003 to 31 December 2008