811 research outputs found
Opinion Dynamics in Social Networks through Mean-Field Games
Emulation, mimicry, and herding behaviors are phenomena that are observed when multiple social groups interact. To study such phenomena, we consider in this paper a large population of homogeneous social networks. Each such network is characterized by a vector state, a vector-valued controlled input and a vector-valued exogenous disturbance. The controlled input of each network is to align its state to the mean distribution of other networksâ states in spite of the actions of the disturbance. One of the contributions of this paper is a detailed analysis of the resulting mean field game for the cases of both polytopic and L2 bounds on controls and disturbances. A second contribution is the establishment of a robust mean-field equilibrium, that is, a solution including the worst-case value function, the state feedback best-responses for the controlled inputs and worst-case disturbances, and a density evolution. This solution is characterized by the property that no player can benefit from a unilateral deviation even in the presence of the disturbance. As a third contribution, microscopic and macroscopic analyses are carried out to show convergence properties of the population distribution using stochastic stability theory
Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies
Gradient-based methods have been widely used for system design and
optimization in diverse application domains. Recently, there has been a renewed
interest in studying theoretical properties of these methods in the context of
control and reinforcement learning. This article surveys some of the recent
developments on policy optimization, a gradient-based iterative approach for
feedback control synthesis, popularized by successes of reinforcement learning.
We take an interdisciplinary perspective in our exposition that connects
control theory, reinforcement learning, and large-scale optimization. We review
a number of recently-developed theoretical results on the optimization
landscape, global convergence, and sample complexity of gradient-based methods
for various continuous control problems such as the linear quadratic regulator
(LQR), control, risk-sensitive control, linear quadratic
Gaussian (LQG) control, and output feedback synthesis. In conjunction with
these optimization results, we also discuss how direct policy optimization
handles stability and robustness concerns in learning-based control, two main
desiderata in control engineering. We conclude the survey by pointing out
several challenges and opportunities at the intersection of learning and
control.Comment: To Appear in Annual Review of Control, Robotics, and Autonomous
System
Stochastic and Optimal Distributed Control for Energy Optimization and Spatially Invariant Systems
Improving energy efficiency and grid responsiveness of buildings requires sensing, computing and communication to enable stochastic decision-making and distributed operations. Optimal control synthesis plays a significant role in dealing with the complexity and uncertainty associated with the energy systems.
The dissertation studies general area of complex networked systems that consist of interconnected components and usually operate in uncertain environments. Specifically, the contents of this dissertation include tools using stochastic and optimal distributed control to overcome these challenges and improve the sustainability of electric energy systems.
The first tool is developed as a unifying stochastic control approach for improving energy efficiency while meeting probabilistic constraints. This algorithm is applied to demonstrate energy efficiency improvement in buildings and improving operational efficiency of virtualized web servers, respectively. Although all the optimization in this technique is in the form of convex optimization, it heavily relies on semidefinite programming (SP). A generic SP solver can handle only up to hundreds of variables. This being said, for a large scale system, the existing off-the-shelf algorithms may not be an appropriate tool for optimal control. Therefore, in the sequel I will exploit optimization in a distributed way.
The second tool is itself a concrete study which is optimal distributed control for spatially invariant systems. Spatially invariance means the dynamics of the system do not vary as we translate along some spatial axis. The optimal H2 [H-2] decentralized control problem is solved by computing an orthogonal projection on a class of Youla parameters with a decentralized structure. Optimal Hâ [H-infinity] performance is posed as a distance minimization in a general Lâ [L-infinity] space from a vector function to a subspace with a mixed Lâ and Hâ space structure. In this framework, the dual and pre-dual formulations lead to finite dimensional convex optimizations which approximate the optimal solution within desired accuracy. Furthermore, a mixed L2 [L-2] /Hâ synthesis problem for spatially invariant systems as trade-offs between transient performance and robustness. Finally, we pursue to deal with a more general networked system, i.e. the Non-Markovian decentralized stochastic control problem, using stochastic maximum principle via Malliavin Calculus
An integral control formulation of Mean-field game based large scale coordination of loads in smart grids
Pressure on ancillary reserves, i.e.frequency preserving, in power systems
has significantly mounted due to the recent generalized increase of the
fraction of (highly fluctuating) wind and solar energy sources in grid
generation mixes. The energy storage associated with millions of individual
customer electric thermal (heating-cooling) loads is considered as a tool for
smoothing power demand/generation imbalances. The piecewise constant level
tracking problem of their collective energy content is formulated as a linear
quadratic mean field game problem with integral control in the cost
coefficients. The introduction of integral control brings with it a robustness
potential to mismodeling, but also the potential of cost coefficient
unboundedness. A suitable Banach space is introduced to establish the existence
of Nash equilibria for the corresponding infinite population game, and
algorithms are proposed for reliably computing a class of desirable near Nash
equilibria. Numerical simulations illustrate the flexibility and robustness of
the approach
Mean Field Games and Applications.
This text is inspired from a âCours Bachelierâ held in January 2009 and taught by Jean-Michel Lasry. This course was based upon the articles of the three authors and upon unpublished materials they developed. Proofs were not presented during the conferences and are now available. So are some issues that were only rapidly tackled during class.Mean Field Games;
- âŠ