A Report on NISOCSol: an Algorithm For Approximating Markovian Equlibria in Dynamic Games with Coupled-Constraints

In this report, we outline a method for approximating a Markovian (or feedback-Nash) equilibrium of a dynamic game, possibly subject to coupled-constraints. We treat such a game as a "multiple" optimal control problem. A method for approximating a solution to a given optimal control problem via backward induction on Markov chains was developed in [Kra01]. A Markovian equilibrium may be obtained numerically by adapting this backward induction approach to a stage Nikaido-Isoda function (described in [KZ06])

Constrained Cost-Coupled Stochastic Games with Independent State Processes

We consider a non-cooperative constrained stochastic games with N players with the following special structure. With each player there is an associated controlled Markov chain. The transition probabilities of the i-th Markov chain depend only on the state and actions of controller i. The information structure that we consider is such that each player knows the state of its own MDP and its own actions. It does not know the states of, and the actions taken by other players. Finally, each player wishes to minimize a time-average cost function, and has constraints over other time-avrage cost functions. Both the cost that is minimized as well as those defining the constraints depend on the state and actions of all players. We study in this paper the existence of a Nash equilirium. Examples in power control in wireless communications are given.Comment: 7 pages, submitted in september 2006 to Operations Research Letter

Designing games to handle coupled constraints

The central goal in multiagent systems is to design local control laws for the individual agents to ensure that the emergent global behavior is desirable with respect to a given system level objective. In many systems, such as cooperative robotics or distributed power control, the design of these local control algorithms is further complicated by additional coupled constraints on the agents' actions. There are several approaches in the existing literature for designing such algorithms stemming from classical optimization theory; however, many of these approaches are not suitable for implementation in multiagent systems. This paper seeks to address the design of such algorithms using the field of game theory. Among other things, this design choice requires defining a local utility function for each decision maker in the system. This paper seeks to address the degree to which utility design can be effective for dealing with these coupled constraints. In particular, is it possible to design local agent utility functions such that all pure Nash equilibrium of the unconstrained game (i) optimize the given system level objective and (ii) satisfy the given coupled constraint. This design would greatly simplify the distributed control algorithms by eliminating the need to explicitly consider the constraints. Unfortunately, we illustrate that designing utility functions within the standard game theoretic framework is not suitable for this design objective. However, we demonstrate that by adding an additional state variable in the game environment, i.e., moving towards state based games, we can satisfy these performance criteria by utility design. We focus on the problem of consensus control to illustrate these results