Decentralized Smart Charging of Large-Scale EVs using Adaptive Multi-Agent Multi-Armed Bandits

The drastic growth of electric vehicles and photovoltaics can introduce new challenges, such as electrical current congestion and voltage limit violations due to peak load demands. These issues can be mitigated by controlling the operation of electric vehicles i.e., smart charging. Centralized smart charging solutions have already been proposed in the literature. But such solutions may lack scalability and suffer from inherent drawbacks of centralization, such as a single point of failure, and data privacy concerns. Decentralization can help tackle these challenges. In this paper, a fully decentralized smart charging system is proposed using the philosophy of adaptive multi-agent systems. The proposed system utilizes multi-armed bandit learning to handle uncertainties in the system. The presented system is decentralized, scalable, real-time, model-free, and takes fairness among different players into account. A detailed case study is also presented for performance evaluation.Comment: CIRED 2023 International Conference & Exhibition on Electricity Distribution, Jun 2023, Rome, Ital

Gestion optimisée d'un réseau de distribution actif par AMAS couplé à la méthode RL des bandits

Modern electrical power systems are evolving with the introduction of distributed energy resources and electric vehicles, promising sustainability. However, the uncontrolled integration of these technologies into legacy power grids can lead to real-time imbalances and peak load issues. Traditional grid reinforcement has drawbacks, including cost and deployment time concerns. Flexible solutions, enabled by grid digitization, offer an alternative by dynamically controlling grid elements. Yet, optimizing these solutions for diverse market actors is complex, and centralized approaches may struggle to manage large-scale smart grids in real-time. This thesis addresses these challenges by developing a decentralized system using adaptive multiagent systems for real-time control of flexible entities in distribution grids. Simulation experiments validate its effectiveness in overcoming centralization issues. Furthermore, integrating combinatorial multi-armed bandit learning enhances performance in stochastic environments. This research offers a promising approach to optimizing large-scale smart grids as they adapt to evolving energy landscapes.Les systèmes électriques modernes évoluent avec l’introduction des ressources énergétiques distribuées et des véhicules électriques, promettant la durabilité. Cependant, l’intégration non contrôlée de ces technologies dans les réseaux électriques existants peut entraîner des déséquilibres en temps réel et des problèmes de le pic de la demande. Le renforcement traditionnel du réseau présente des inconvénients, notamment des préoccupations liées au coût et au temps de déploiement. Des solutions flexibles, rendues possibles par la digitalisation du réseau, offrent une alternative en contrôlant dynamiquement les éléments du réseau. Cependant, l’optimisation de ces solutions pour les différents acteurs du marché est complexe,et les approches centralisées peuvent avoir du mal à gérer en temps réel de grands réseaux intelligents. Cette thèse aborde ces défis en développant un système décentralisé utilisant des systèmes multi-agents adaptatifs pour le contrôle en temps réel des entités flexibles dans les réseaux de distribution. Des expériences de simulation valident son efficacité pour surmonter les problèmes de centralisation. De plus, l’intégration de l’apprentissage combinatoire à bandit manchot améliore les performances dans des environnements stochastiques. Cette recherche propose une approche prometteuse pour l’optimisation de grands réseaux intelligents alors qu’ils s’adaptent aux évolutions du paysage énergétique

Multi-Armed Bandits Learning for Optimal Decentralized Control of Electric Vehicle Charging

International audienceOptimal control of new grid elements, such as electric vehicles, can ensure an efficient, and stable operation of distribution networks. Decentralization can result in scalability, higher reliability, and privacy (which may not be present in centralized or hierarchical control solutions). A decentralized multi-agent multi-armed combinatorial bandits system using Thompson Sampling is presented for smart charging of electric vehicles. The proposed system utilizes the concepts of bandits reinforcement learning to manage the uncertainties in the choice of other players' actions, and in the intermittent photovoltaic energy production. This proposed solution is fully decentralized, real-time, scalable, model-free, and fair. Its performance is evaluated through comparison with other charging strategies i.e., basic charging, and centralized optimization

Decentralized Smart Charging of Large-Scale EVs using Adaptive Multi-Agent Multi-Armed Bandits

The drastic growth of electric vehicles and photovoltaics can introduce new challenges, such as electrical current congestion and voltage limit violations due to peak load demands. These issues can be mitigated by controlling the operation of electric vehicles i.e., smart charging. Centralized smart charging solutions have already been proposed in the literature. But such solutions may lack scalability and suffer from inherent drawbacks of centralization, such as a single point of failure, and data privacy concerns. Decentralization can help tackle these challenges. In this paper, a fully decentralized smart charging system is proposed using the philosophy of adaptive multi-agent systems. The proposed system utilizes multi-armed bandit learning to handle uncertainties in the system. The presented system is decentralized, scalable, real-time, model-free, and takes fairness among different players into account. A detailed case study is also presented for performance evaluation

Decentralized optimal management of a large-scale EV fleet: optimality and computational complexity comparison between an Adaptive MAS and MILP

16p. ; Preprint submitted to International Journal of Electrical Power & Energy SystemsInternational audienceIncreasing the penetration of variable and uncertain renewables and electric vehicles in power systems may give rise to problems (such as network congestion and commitment mismatches) if not controlled strategically. This demands control solutions in the form of energy management strategies for active distribution networks which would control the connected distributed energy resources and storage units in real-time to address the mentioned challenges. Centralized strategies may fail to serve this purpose for large-scale distribution networks due to their inherent shortcomings like vulnerability to single point of failures and large computing times. Unlike centralized approaches, decentralized control strategies show more potential. This paper presents one such solution, based on an adaptive multi-agent system, to control a large-scale distribution network in real-time. Its performance is compared with the results obtained with the corresponding centralized optimization problem, modeled as a mixed integer linear programming problem. Both the centralized version and the decentralized multi-agent version of the problem under consideration are presented and a case study is designed for the comparison. The comparison shows that the designed multi-agent system produces a near-optimal solution in real-time while the centralized optimization strategy struggles in terms of computational complexities for larger distribution networks

Adaptive Multi-Agent System and Mixed Integer Linear Programming Optimization Comparison for Grid Stability and Commitment Mismatch in Smart Grids

International audienceExisting electrical networks are going through a transition and distributed energy resource, if not managed properly, can hinder this transition. Uncontrolled introduction of photovoltaics and electric vehicles in distribution networks would lead to substantial issues such as commitment mismatches, line congestions, voltage deviations, etc. This paper presents the use of a classical approach, mixed integer linear programming optimization, and a novel approach, adaptive multi-agent system, to solve the highlighted distribution side challenges by utilizing electric vehicles' storage capacity. This comparison serves as a great tool to benchmark the performance of the under-development adaptive multi-agent system methodology