Finding mixed-strategy equilibria of continuous-action games without gradients using randomized policy networks

We study the problem of computing an approximate Nash equilibrium of continuous-action game without access to gradients. Such game access is common in reinforcement learning settings, where the environment is typically treated as a black box. To tackle this problem, we apply zeroth-order optimization techniques that combine smoothed gradient estimators with equilibrium-finding dynamics. We model players' strategies using artificial neural networks. In particular, we use randomized policy networks to model mixed strategies. These take noise in addition to an observation as input and can flexibly represent arbitrary observation-dependent, continuous-action distributions. Being able to model such mixed strategies is crucial for tackling continuous-action games that lack pure-strategy equilibria. We evaluate the performance of our method using an approximation of the Nash convergence metric from game theory, which measures how much players can benefit from unilaterally changing their strategy. We apply our method to continuous Colonel Blotto games, single-item and multi-item auctions, and a visibility game. The experiments show that our method can quickly find high-quality approximate equilibria. Furthermore, they show that the dimensionality of the input noise is crucial for performance. To our knowledge, this paper is the first to solve general continuous-action games with unrestricted mixed strategies and without any gradient information

Focality and Asymmetry in Multi-battle Contests

This article examines behavior in two-person constant-sum Colonel Blotto games in which each player maximizes the expected total value of the battlefields won. A lottery contest success function is employed in each battlefield. Recent experimental research on such games provides only partial support for Nash equilibrium behavior. We hypothesize that the salience of battlefields affects strategic behavior (the salient target hypothesis). We present a controlled test of this hypothesis – against Nash predictions – when the sources of salience come from certain asymmetries in either battlefield values or labels (as in Schelling (1960)). In both cases, subjects over-allocate the resource to the salient battlefields relative to the Nash prediction. However, the effect is stronger with salient values. In the absence of salience, we replicate previous results in the literature supporting the Nash prediction

Focality and Asymmetry in Multi-battle Contests

This article examines the influence of focality in Colonel Blotto games with a lottery contest success function (CSF), where the equilibrium is unique and in pure strategies. We hypothesise that the salience of battlefields affects strategic behaviour (the salient target hypothesis) and present a controlled test of this hypothesis against Nash predictions, checking the robustness of equilibrium play. When the sources of salience come from asymmetries in battlefield values or labels (as in Schelling, 1960), subjects over-allocate the resource to the salient battlefields relative to the Nash prediction. However, the effect is stronger with salient values. In the absence of salience, we find support for the Nash prediction

Colonel Blotto games with a head start

This paper studies Colonel Blotto games with two battlefields where one player has a head start in the form of additional troops on one of the battlefields. Such games arise naturally in marketing, electoral competition, and military conflict. Sion and Wolfe (1957) have shown that, if the strategy space is continuous, a mixed-strategy Nash equilibrium need not exist. Therefore, we consider a finite approximation. Using the iterated elimination of (weakly) dominated strategies, we identify an equilibrium for all parameter constellations and discuss its uniqueness properties. In equilibrium, resource decisions are typically not uniform but tend to concern units that roughly correspond in size to multiples of the head start. Moreover, competition takes the form of a hide-and-seek game, where the favorite tries to outguess the number of units that the underdog commits to the balanced battlefield. Somewhat unexpectedly, equilibrium payoffs of finite approximations of the Sion-Wolfe game accumulate around precisely three values. We also discuss the relation to the model with heterogeneous budgets but no head start

How strength asymmetries shape multi-sided conflicts

Governments and multilateral organisations often attempt to influence multi-sided violent conflicts by supporting or undermining one of the conflicting parties. We investigate the (intended and unintended) consequences of strengthening or weakening an agent in a multi-sided conflict. Using a conflict network based on Franke and Öztürk (J Public Econ 126:104–113, 2015), we study how changing the strength of otherwise symmetric agents creates knock-on effects throughout the network. Increasing or decreasing an agent’s strength has the same unintended consequences. Changes in the strength of an agent induce a relocation of conflict investments: Distant conflicts are carried out more fiercely. In line with previous results, asymmetry reduces aggregate conflict investments. In the case of bipartite networks, with two conflicting tacit groups with aligned interests, agents in the group of the (now) strong or weak agent face more intense conflicts. Furthermore, in conflicts where the (now strong or weak) agent is not involved, the probabilities of winning remain unchanged compared to the symmetric case.</p

Generalizations of the General Lotto and Colonel Blotto Games

In this paper, we generalize the General Lotto game (budget constraints satisfied in expectation) and the Colonel Blotto game (budget constraints hold with probability one) to allow for battlefield valuations that are heterogeneous across battlefields and asymmetric across players, and for the players to have asymmetric resource constraints. We completely characterize Nash equilibrium in the generalized version of the General Lotto game and then show how this characterization can be applied to identify equilibria in the Colonel Blotto version of the game. In both games, we find that there exist sets of non-pathological parameter configurations of positive Lebesgue measure with multiple payoff nonequivalent equilibria