Decentralized Multi-agent Filtering

This paper addresses the considerations that comes along with adopting decentralized communication for multi-agent localization applications in discrete state spaces. In this framework, we extend the original formulation of the Bayes filter, a foundational probabilistic tool for discrete state estimation, by appending a step of greedy belief sharing as a method to propagate information and improve local estimates' posteriors. We apply our work in a model-based multi-agent grid-world setting, where each agent maintains a belief distribution for every agents' state. Our results affirm the utility of our proposed extensions for decentralized collaborative tasks. The code base for this work is available in the following rep

A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control

This paper addresses the challenges associated with decentralized voltage control in power grids due to an increase in distributed generations (DGs). Traditional model-based voltage control methods struggle with the rapid energy fluctuations and uncertainties of these DGs. While multi-agent reinforcement learning (MARL) has shown potential for decentralized secondary control, scalability issues arise when dealing with a large number of DGs. This problem lies in the dominant centralized training and decentralized execution (CTDE) framework, where the critics take global observations and actions. To overcome these challenges, we propose a scalable network-aware (SNA) framework that leverages network structure to truncate the input to the critic's Q-function, thereby improving scalability and reducing communication costs during training. Further, the SNA framework is theoretically grounded with provable approximation guarantee, and it can seamlessly integrate with multiple multi-agent actor-critic algorithms. The proposed SNA framework is successfully demonstrated in a system with 114 DGs, providing a promising solution for decentralized voltage control in increasingly complex power grid systems

Optimal charging scheduling for battery electric vehicles under smart grid.

A projected high penetration of battery electric vehicles (BEV s) in the market will introduce an additional load in the electricity grid. Furthermore, uncontrolled BEV charging from residential users will exacerbate the existing peak load during evening hours. In this thesis, we propose two optimization models to alleviate the impact of extra demand from electric vehicles on the power grid. The first is a centralized charging scheduling model that coordinates the charging among BEV users under the goal of minimizing the total electricity cost for all users. The second model uses a decentralized agent-based approach to scheduling the BEV charging. This approach allows each user to minimize his/her own electricity cost through a learning process on a day-to-day basis. Our numerical results indicate that the centralized model is effective in reducing the total cost and peak-to-average ratios of the system load. Although the decentralized model is less effective compared to the centralized model, it is more appealing to public

Consensus-based approach to peer-to-peer electricity markets with product differentiation

With the sustained deployment of distributed generation capacities and the more proactive role of consumers, power systems and their operation are drifting away from a conventional top-down hierarchical structure. Electricity market structures, however, have not yet embraced that evolution. Respecting the high-dimensional, distributed and dynamic nature of modern power systems would translate to designing peer-to-peer markets or, at least, to using such an underlying decentralized structure to enable a bottom-up approach to future electricity markets. A peer-to-peer market structure based on a Multi-Bilateral Economic Dispatch (MBED) formulation is introduced, allowing for multi-bilateral trading with product differentiation, for instance based on consumer preferences. A Relaxed Consensus+Innovation (RCI) approach is described to solve the MBED in fully decentralized manner. A set of realistic case studies and their analysis allow us showing that such peer-to-peer market structures can effectively yield market outcomes that are different from centralized market structures and optimal in terms of respecting consumers preferences while maximizing social welfare. Additionally, the RCI solving approach allows for a fully decentralized market clearing which converges with a negligible optimality gap, with a limited amount of information being shared.Comment: Accepted for publication in IEEE Transactions on Power System

Decentralized energy management of power networks with distributed generation using periodical self-sufficient repartitioning approach

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.In this paper, we propose a decentralized model predictive control (MPC) method as the energy management strategy for a large-scale electrical power network with distributed generation and storage units. The main idea of the method is to periodically repartition the electrical power network into a group of self-sufficient interconnected microgrids. In this regard, a distributed graph-based partitioning algorithm is proposed. Having a group of self-sufficient microgrids allows the decomposition of the centralized dynamic economic dispatch problem into local economic dispatch problems for the microgrids. In the overall scheme, each microgrid must cooperate with its neighbors to perform repartitioning periodically and solve a decentralized MPC-based optimization problem at each time instant. In comparison to the approaches based on distributed optimization, the proposed scheme requires less intensive communication since the microgrids do not need to communicate at each time instant, at the cost of suboptimality of the solutions. The performance of the proposed scheme is shown by means of numerical simulations with a well-known benchmark case. © 2019 American Automatic Control Council.Peer ReviewedPostprint (author's final draft

Lightweight Blockchain Framework for Location-aware Peer-to-Peer Energy Trading

Peer-to-Peer (P2P) energy trading can facilitate integration of a large number of small-scale producers and consumers into energy markets. Decentralized management of these new market participants is challenging in terms of market settlement, participant reputation and consideration of grid constraints. This paper proposes a blockchain-enabled framework for P2P energy trading among producer and consumer agents in a smart grid. A fully decentralized market settlement mechanism is designed, which does not rely on a centralized entity to settle the market and encourages producers and consumers to negotiate on energy trading with their nearby agents truthfully. To this end, the electrical distance of agents is considered in the pricing mechanism to encourage agents to trade with their neighboring agents. In addition, a reputation factor is considered for each agent, reflecting its past performance in delivering the committed energy. Before starting the negotiation, agents select their trading partners based on their preferences over the reputation and proximity of the trading partners. An Anonymous Proof of Location (A-PoL) algorithm is proposed that allows agents to prove their location without revealing their real identity. The practicality of the proposed framework is illustrated through several case studies, and its security and privacy are analyzed in detail

A Game-theoretic Formulation of the Homogeneous Self-Reconfiguration Problem

In this paper we formulate the homogeneous two- and three-dimensional self-reconfiguration problem over discrete grids as a constrained potential game. We develop a game-theoretic learning algorithm based on the Metropolis-Hastings algorithm that solves the self-reconfiguration problem in a globally optimal fashion. Both a centralized and a fully distributed algorithm are presented and we show that the only stochastically stable state is the potential function maximizer, i.e. the desired target configuration. These algorithms compute transition probabilities in such a way that even though each agent acts in a self-interested way, the overall collective goal of self-reconfiguration is achieved. Simulation results confirm the feasibility of our approach and show convergence to desired target configurations.Comment: 8 pages, 5 figures, 2 algorithm

Task-Based Information Compression for Multi-Agent Communication Problems with Channel Rate Constraints

A collaborative task is assigned to a multiagent system (MAS) in which agents are allowed to communicate. The MAS runs over an underlying Markov decision process and its task is to maximize the averaged sum of discounted one-stage rewards. Although knowing the global state of the environment is necessary for the optimal action selection of the MAS, agents are limited to individual observations. The inter-agent communication can tackle the issue of local observability, however, the limited rate of the inter-agent communication prevents the agent from acquiring the precise global state information. To overcome this challenge, agents need to communicate their observations in a compact way such that the MAS compromises the minimum possible sum of rewards. We show that this problem is equivalent to a form of rate-distortion problem which we call the task-based information compression. We introduce a scheme for task-based information compression titled State aggregation for information compression (SAIC), for which a state aggregation algorithm is analytically designed. The SAIC is shown to be capable of achieving near-optimal performance in terms of the achieved sum of discounted rewards. The proposed algorithm is applied to a rendezvous problem and its performance is compared with several benchmarks. Numerical experiments confirm the superiority of the proposed algorithm.Comment: 13 pages, 9 figure