Learning Equilibria with Partial Information in Decentralized Wireless Networks

In this article, a survey of several important equilibrium concepts for decentralized networks is presented. The term decentralized is used here to refer to scenarios where decisions (e.g., choosing a power allocation policy) are taken autonomously by devices interacting with each other (e.g., through mutual interference). The iterative long-term interaction is characterized by stable points of the wireless network called equilibria. The interest in these equilibria stems from the relevance of network stability and the fact that they can be achieved by letting radio devices to repeatedly interact over time. To achieve these equilibria, several learning techniques, namely, the best response dynamics, fictitious play, smoothed fictitious play, reinforcement learning algorithms, and regret matching, are discussed in terms of information requirements and convergence properties. Most of the notions introduced here, for both equilibria and learning schemes, are illustrated by a simple case study, namely, an interference channel with two transmitter-receiver pairs.Comment: 16 pages, 5 figures, 1 table. To appear in IEEE Communication Magazine, special Issue on Game Theor

Quality-Of-Service Provisioning in Decentralized Networks: A Satisfaction Equilibrium Approach

This paper introduces a particular game formulation and its corresponding notion of equilibrium, namely the satisfaction form (SF) and the satisfaction equilibrium (SE). A game in SF models the case where players are uniquely interested in the satisfaction of some individual performance constraints, instead of individual performance optimization. Under this formulation, the notion of equilibrium corresponds to the situation where all players can simultaneously satisfy their individual constraints. The notion of SE, models the problem of QoS provisioning in decentralized self-configuring networks. Here, radio devices are satisfied if they are able to provide the requested QoS. Within this framework, the concept of SE is formalized for both pure and mixed strategies considering finite sets of players and actions. In both cases, sufficient conditions for the existence and uniqueness of the SE are presented. When multiple SE exist, we introduce the idea of effort or cost of satisfaction and we propose a refinement of the SE, namely the efficient SE (ESE). At the ESE, all players adopt the action which requires the lowest effort for satisfaction. A learning method that allows radio devices to achieve a SE in pure strategies in finite time and requiring only one-bit feedback is also presented. Finally, a power control game in the interference channel is used to highlight the advantages of modeling QoS problems following the notion of SE rather than other equilibrium concepts, e.g., generalized Nash equilibrium.Comment: Article accepted for publication in IEEE Journal on Selected Topics in Signal Processing, special issue in Game Theory in Signal Processing. 16 pages, 6 figure

Mean-Field-Type Games in Engineering

A mean-field-type game is a game in which the instantaneous payoffs and/or the state dynamics functions involve not only the state and the action profile but also the joint distributions of state-action pairs. This article presents some engineering applications of mean-field-type games including road traffic networks, multi-level building evacuation, millimeter wave wireless communications, distributed power networks, virus spread over networks, virtual machine resource management in cloud networks, synchronization of oscillators, energy-efficient buildings, online meeting and mobile crowdsensing.Comment: 84 pages, 24 figures, 183 references. to appear in AIMS 201

Joint Channel Selection and Power Control in Infrastructureless Wireless Networks: A Multi-Player Multi-Armed Bandit Framework

This paper deals with the problem of efficient resource allocation in dynamic infrastructureless wireless networks. Assuming a reactive interference-limited scenario, each transmitter is allowed to select one frequency channel (from a common pool) together with a power level at each transmission trial; hence, for all transmitters, not only the fading gain, but also the number of interfering transmissions and their transmit powers are varying over time. Due to the absence of a central controller and time-varying network characteristics, it is highly inefficient for transmitters to acquire global channel and network knowledge. Therefore a reasonable assumption is that transmitters have no knowledge of fading gains, interference, and network topology. Each transmitting node selfishly aims at maximizing its average reward (or minimizing its average cost), which is a function of the action of that specific transmitter as well as those of all other transmitters. This scenario is modeled as a multi-player multi-armed adversarial bandit game, in which multiple players receive an a priori unknown reward with an arbitrarily time-varying distribution by sequentially pulling an arm, selected from a known and finite set of arms. Since players do not know the arm with the highest average reward in advance, they attempt to minimize their so-called regret, determined by the set of players' actions, while attempting to achieve equilibrium in some sense. To this end, we design in this paper two joint power level and channel selection strategies. We prove that the gap between the average reward achieved by our approaches and that based on the best fixed strategy converges to zero asymptotically. Moreover, the empirical joint frequencies of the game converge to the set of correlated equilibria. We further characterize this set for two special cases of our designed game

Energy-Efficient Power Control for Contention-Based Synchronization in OFDMA Systems with Discrete Powers and Limited Feedback

This work derives a distributed and iterative algorithm by which mobile terminals can selfishly control their transmit powers during the synchronization procedure specified by the IEEE 802.16m and the 3GPP-LTE standards for orthogonal frequency-division multiple-access technologies. The proposed solution aims at maximizing the energy efficiency of the network and is derived on the basis of a finite noncooperative game in which the players have discrete action sets of transmit powers. The set of Nash equilibria of the game is investigated, and a distributed power control algorithm is proposed to achieve synchronization in an energy-efficient manner under the assumption that the feedback from the base station is limited. Numerical results show that the proposed solution improves the energy efficiency as well as the timing estimation accuracy of the network compared to existing alternatives, while requiring a reasonable amount of information to be exchanged on the return channel

Distributed Learning Policies for Power Allocation in Multiple Access Channels

We analyze the problem of distributed power allocation for orthogonal multiple access channels by considering a continuous non-cooperative game whose strategy space represents the users' distribution of transmission power over the network's channels. When the channels are static, we find that this game admits an exact potential function and this allows us to show that it has a unique equilibrium almost surely. Furthermore, using the game's potential property, we derive a modified version of the replicator dynamics of evolutionary game theory which applies to this continuous game, and we show that if the network's users employ a distributed learning scheme based on these dynamics, then they converge to equilibrium exponentially quickly. On the other hand, a major challenge occurs if the channels do not remain static but fluctuate stochastically over time, following a stationary ergodic process. In that case, the associated ergodic game still admits a unique equilibrium, but the learning analysis becomes much more complicated because the replicator dynamics are no longer deterministic. Nonetheless, by employing results from the theory of stochastic approximation, we show that users still converge to the game's unique equilibrium. Our analysis hinges on a game-theoretical result which is of independent interest: in finite player games which admit a (possibly nonlinear) convex potential function, the replicator dynamics (suitably modified to account for nonlinear payoffs) converge to an eps-neighborhood of an equilibrium at time of order O(log(1/eps)).Comment: 11 pages, 8 figures. Revised manuscript structure and added more material and figures for the case of stochastically fluctuating channels. This version will appear in the IEEE Journal on Selected Areas in Communication, Special Issue on Game Theory in Wireless Communication