Constrained distributed optimization : A population dynamics approach

Large-scale network systems involve a large number of states, which makes the design of real-time controllers a challenging task. A distributed controller design allows to reduce computational requirements since tasks are divided into different systems, allowing real-time processing. This paper proposes a novel methodology for solving constrained optimization problems in a distributed way inspired by population dynamics. This methodology consists of an extension of a population dynamics equation and the introduction of a mass dynamics equation. The proposed methodology divides the problem into smaller sub-problems, whose feasible regions vary over time achieving an agreement to solve the global problem. The methodology also guarantees attraction to the feasible region and allows to have few changes in the decision-making design when a network suffers the addition/removal of nodes/edges. Then, distributed controllers are designed with the proposed methodology and applied to the large-scale Barcelona Drinking Water Network (BDWN). Some simulations are presented and discussed in order to illustrate the control performance.Peer ReviewedPostprint (author's final draft

Evolutionary-game-based dynamical tuning for multi-objective model predictive control

Model predictive control (MPC) is one of the most used optimization-based control strategies for large-scale systems, since this strategy allows to consider a large number of states and multi-objective cost functions in a straightforward way. One of the main issues in the design of multi-objective MPC controllers, which is the tuning of the weights associated to each objective in the cost function, is treated in this work. All the possible combinations of weights within the cost function affect the optimal result in a given Pareto front. Furthermore, when the system has time-varying parameters, e.g., periodic disturbances, the appropriate weight tuning might also vary over time. Moreover, taking into account the computational burden and the selected sampling time in the MPC controller design, the computation time to find a suitable tuning is limited. In this regard, the development of strategies to perform a dynamical tuning in function of the system conditions potentially improves the closed-loop performance. In order to adapt in a dynamical way the weights in the MPC multi-objective cost function, an evolutionary-game approach is proposed. This approach allows to vary the prioritization weights in the proper direction taking as a reference a desired region within the Pareto front. The proper direction for the prioritization is computed by only using the current system values, i.e., the current optimal control action and the measurement of the current states, which establish the system cost function over a certain point in the Pareto front. Finally, some simulations of a multi-objective MPC for a real multi-variable case study show a comparison between the system performance obtained with static and dynamical tuning.Peer ReviewedPostprint (author's final draft

Designing Coalition-Proof Reverse Auctions over Continuous Goods

This paper investigates reverse auctions that involve continuous values of different types of goods, general nonconvex constraints, and second stage costs. We seek to design the payment rules and conditions under which coalitions of participants cannot influence the auction outcome in order to obtain higher collective utility. Under the incentive-compatible Vickrey-Clarke-Groves mechanism, we show that coalition-proof outcomes are achieved if the submitted bids are convex and the constraint sets are of a polymatroid-type. These conditions, however, do not capture the complexity of the general class of reverse auctions under consideration. By relaxing the property of incentive-compatibility, we investigate further payment rules that are coalition-proof without any extra conditions on the submitted bids and the constraint sets. Since calculating the payments directly for these mechanisms is computationally difficult for auctions involving many participants, we present two computationally efficient methods. Our results are verified with several case studies based on electricity market data

Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learning

We examine online safe multi-agent reinforcement learning using constrained Markov games in which agents compete by maximizing their expected total rewards under a constraint on expected total utilities. Our focus is confined to an episodic two-player zero-sum constrained Markov game with independent transition functions that are unknown to agents, adversarial reward functions, and stochastic utility functions. For such a Markov game, we employ an approach based on the occupancy measure to formulate it as an online constrained saddle-point problem with an explicit constraint. We extend the Lagrange multiplier method in constrained optimization to handle the constraint by creating a generalized Lagrangian with minimax decision primal variables and a dual variable. Next, we develop an upper confidence reinforcement learning algorithm to solve this Lagrangian problem while balancing exploration and exploitation. Our algorithm updates the minimax decision primal variables via online mirror descent and the dual variable via projected gradient step and we prove that it enjoys sublinear rate $O((|X|+|Y|) L \sqrt{T(|A|+|B|)}))$ for both regret and constraint violation after playing $T$ episodes of the game. Here, $L$ is the horizon of each episode, $(|X|,|A|)$ and $(|Y|,|B|)$ are the state/action space sizes of the min-player and the max-player, respectively. To the best of our knowledge, we provide the first provably efficient online safe reinforcement learning algorithm in constrained Markov games.Comment: 59 pages, a full version of the main paper in the 5th Annual Conference on Learning for Dynamics and Contro

Open-Source, Open-Architecture SoftwarePlatform for Plug-InElectric Vehicle SmartCharging in California

This interdisciplinary eXtensible Building Operating System–Vehicles project focuses on controlling plug-in electric vehicle charging at residential and small commercial settings using a novel and flexible open-source, open-architecture charge communication and control platform. The platform provides smart charging functionalities and benefits to the utility, homes, and businesses.This project investigates four important areas of vehicle-grid integration research, integrating technical as well as social and behavioral dimensions: smart charging user needs assessment, advanced load control platform development and testing, smart charging impacts, benefits to the power grid, and smart charging ratepayer benefits