Constrained Cost-Coupled Stochastic Games with Independent State Processes

We consider a non-cooperative constrained stochastic games with N players with the following special structure. With each player there is an associated controlled Markov chain. The transition probabilities of the i-th Markov chain depend only on the state and actions of controller i. The information structure that we consider is such that each player knows the state of its own MDP and its own actions. It does not know the states of, and the actions taken by other players. Finally, each player wishes to minimize a time-average cost function, and has constraints over other time-avrage cost functions. Both the cost that is minimized as well as those defining the constraints depend on the state and actions of all players. We study in this paper the existence of a Nash equilirium. Examples in power control in wireless communications are given.Comment: 7 pages, submitted in september 2006 to Operations Research Letter

Player aggregation in the traveling inspector model

We consider a model of dynamic inspection/surveillance of a number of facilities in different geographical locations. The inspector in this process travels from one facility to another and performs an inspection at each facility he visits. His aim is to devise an inspection/ travel schedule which minimizes the losses to society (or to his employer) resulting both from undetected violations of the regulations and from the costs of the policing operation. This model is formulated as a non-cooperative, single-controller, stochastic game. The existence of stationary Nash equilibria is established as a consequence of aggregating all the inspectees into a single “aggregated inspectee”. It is shown that such player aggregation causes no loss of generality under very mild assumptions. A notion of an “optimal Nash equilibrium” for the inspector is introduced and proven to be well-defined in this context. The issue of the inspector’s power to “enforce” such an equilibrium is also discussed

Modeling and Control of Rare Segments in BitTorrent with Epidemic Dynamics

Despite its existing incentives for leecher cooperation, BitTorrent file sharing fundamentally relies on the presence of seeder peers. Seeder peers essentially operate outside the BitTorrent incentives, with two caveats: slow downlinks lead to increased numbers of "temporary" seeders (who left their console, but will terminate their seeder role when they return), and the copyright liability boon that file segmentation offers for permanent seeders. Using a simple epidemic model for a two-segment BitTorrent swarm, we focus on the BitTorrent rule to disseminate the (locally) rarest segments first. With our model, we show that the rarest-segment first rule minimizes transition time to seeder (complete file acquisition) and equalizes the segment populations in steady-state. We discuss how alternative dissemination rules may {\em beneficially increase} file acquisition times causing leechers to remain in the system longer (particularly as temporary seeders). The result is that leechers are further enticed to cooperate. This eliminates the threat of extinction of rare segments which is prevented by the needed presence of permanent seeders. Our model allows us to study the corresponding trade-offs between performance improvement, load on permanent seeders, and content availability, which we leave for future work. Finally, interpreting the two-segment model as one involving a rare segment and a "lumped" segment representing the rest, we study a model that jointly considers control of rare segments and different uplinks causing "choking," where high-uplink peers will not engage in certain transactions with low-uplink peers.Comment: 18 pages, 6 figures, A shorter version of this paper that did not include the N-segment lumped model was presented in May 2011 at IEEE ICC, Kyot