On Randomized Fictitious Play for Approximating Saddle Points Over Convex Sets

Given two bounded convex sets X\subseteq\RR^m and Y\subseteq\RR^n, specified by membership oracles, and a continuous convex-concave function F:X\times Y\to\RR, we consider the problem of computing an \eps-approximate saddle point, that is, a pair $(x^*,y^*)\in X\times Y$ such that \sup_{y\in Y} F(x^*,y)\le \inf_{x\in X}F(x,y^*)+\eps. Grigoriadis and Khachiyan (1995) gave a simple randomized variant of fictitious play for computing an \eps-approximate saddle point for matrix games, that is, when $F$ is bilinear and the sets $X$ and $Y$ are simplices. In this paper, we extend their method to the general case. In particular, we show that, for functions of constant "width", an \eps-approximate saddle point can be computed using O^*(\frac{(n+m)}{\eps^2}\ln R) random samples from log-concave distributions over the convex sets $X$ and $Y$. It is assumed that $X$ and $Y$ have inscribed balls of radius $1/R$ and circumscribing balls of radius $R$. As a consequence, we obtain a simple randomized polynomial-time algorithm that computes such an approximation faster than known methods for problems with bounded width and when \eps \in (0,1) is a fixed, but arbitrarily small constant. Our main tool for achieving this result is the combination of the randomized fictitious play with the recently developed results on sampling from convex sets

Application of multiplicative weights update method in algorithmic game theory

In this thesis, we apply the Multiplicative Weights Update Method (MWUM) to the design of approximation algorithms for some optimization problems in game-theoretic settings. Lavi and Swamy {LS05,LS11} introduced a randomized mechanism for combinatorial auctions that uses an approximation algorithm for the underlying optimization problem, so-called social welfare maximization and converts the approximation algorithm to a randomized mechanism that is { truthful-in-expectation}, which means each player maximizes its expected utility by telling the truth. The mechanism is powerful (e.g., see {LS05,LS11,CEF10,HKV11} for applications), but unlikely to be efficient in practice, because it uses the Ellipsoid method. In chapter 2, we follow the general scheme suggested by Lavi and Swamy and replace the Ellipsoid method with MWUM. This results in a faster and simpler approximately truthful-in-expectation mechanism. We also extend their assumption regarding the existence of an exact solution for the LP-relaxation of social welfare maximization. We assume that there exists an approximation algorithm for the LP and establish a new randomized approximation mechanism. In chapter 3, we consider the problem of computing an approximate saddle point, or equivalently equilibrium, for a convex-concave functions F: X\times Y\to \RR, where $X$ and $Y$ are convex sets of arbitrary dimensions. Our main contribution is the design of a randomized algorithm for computing an \eps-approximation saddle point for $F$. Our algorithm is based on combining a technique developed by Grigoriadis and Khachiyan {GK95}, which is a randomized variant of Brown's fictitious play {B51}, with the recent results on random sampling from convex sets (see, e.g., {LV06,V05}). The algorithm finds an \eps-approximation saddle point in an expected number of O\left(\frac{\rho^2(n+m)}{\eps^{2}}\ln\frac{R}{\eps}\right) iterations, where in each iteration two points are sampled from log-concave distributions over strategy sets. It is assumed that $X$ and $Y$ have inscribed balls of radius $1/R$ and circumscribing balls of radius $R$ and $\rho=\max_{x\in X, y\in Y} |F(x,y)|$. In particular, the algorithm requires O^*\left(\frac{\rho^2(n+m)^6}{\eps^{2}}\ln{R}\right) calls to a membership oracle, where $O^*(\cdot)$ suppresses polylogarithmic factors that depend on $n$, $m$, and \eps.In dieser Doktorarbeit verwenden wir die Multiplicative Weights Update Method (MWUM) für den Entwurf von Approximationsalgorithmen für bestimmte Optimierungsprobleme im spieltheoretischen Umfeld. Lavi und Swamy {LS05,LS11} präsentierten einen randomisierten Mechanismus für kombinatorische Auktionen. Sie verwenden dazu einen Approximationsalgorithmus für die Lösung des zugrundeliegenden Optimierungsproblem, das so genannte Social Welfare Maximization Problem, und wandeln diesen zu einem randomisierten Mechanismus um, der im Erwartungsfall anreizkompatibel ist. Dies bedeutet jeder Spieler erreicht den maximalen Gewinn, wenn er sich ehrlich verhält. Der Mechanismus ist sehr mächtig (siehe {LS05,LS11,CEF10,HKV11} für Anwendungen); trotzdem ist es unwahrscheinlich, dass er in der Praxis effizient ist, da hier die Ellipsoidmethode verwendet wird. In Kapitel 2 folgen wir dem von Lavi und Swamy vorgeschlagenem Schema und ersetzen die Ellipsoidmethode durch MWUM. Das Ergebnis ist ein schnellerer, einfacherer und im Erwartungsfall anreizkompatibler Approximationsmechanismus. Wir erweitern ihre Annahme zur Existenz einer exakten Lösung der LP-Relaxierung für das Social Welfare Maximization Problem. Wir nehmen an, dass ein Approximationsalgorithmus für das LP existiert und beschreiben darauf basierend einen neuen randomisierten Approximationsmechanismus. In Kapitel 3 betrachten wir das Problem für konvexe und konkave Funktionen $F:X\times Y\rightarrow\mathbb{R}$, wobei $X$ und $Y$ konvexe Mengen von beliebiger Dimension sind, einen Sattelpunkt zu approximieren (oder gleichbedeutend ein Equilibrium). Unser Hauptbeitrag ist der Entwurf eines randomisierten Algorithmus zur Berechnung einer $\epsilon$-Näherung eines Sattelpunktes von $F$. Unser Algorithmus beruht auf der Kombination einer Technik entwickelt durch Grigoriadis und Khachiyan {GK95}, welche eine zufallsbasierte Variation von Browns Fictitious Play {B51} ist, mit kürzlich erschienenen Resultaten im Bereich der zufälligen Stichprobennahme aus konvexen Mengen (siehe {LV06,V05}). Der Algorithmus findet eine $\epsilon$-Näherung eines Sattelpunktes im Erwartungsfall in $O(\frac{\rho^{2}(n+m)^{6}}{\epsilon^{2}}\log\frac{R}{\epsilon})$ Rechenschritten, wobei in jedem Rechenschritt zwei Punkte zufällig gemäß einer log-konkaven Verteilungen über Strategiemengen gezogen werden. Hier nehmen wir an, dass $X$ und $Y$ einbeschriebene Kugeln mit Radius $1/R$ und umschreibende Kugeln von Radius R besitzen und $\rho=\max_{x\in X,y\in Y}|F(x,y)|$. Der Algorithmus benötigt dabei $O^{*}(\frac{\rho^{2}(n+m)^{6}}{\epsilon^{2}}\log R)$ Aufrufe eines Zugehörigkeitsorakels, hier versteckt $O^{*}(\cdot)$ polylogarithmische Faktoren, die von $n,m$ und $\epsilon$ abhängen

Frank-Wolfe Algorithms for Saddle Point Problems

We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solver over polytopes, thereby partially answering a 30 year-old conjecture. We also survey other convergence results and highlight gaps in the theoretical underpinnings of FW-style algorithms. Motivating applications without known efficient alternatives are explored through structured prediction with combinatorial penalties as well as games over matching polytopes involving an exponential number of constraints.Comment: Appears in: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017). 39 page

Spatial evolution of human dialects

The geographical pattern of human dialects is a result of history. Here, we formulate a simple spatial model of language change which shows that the final result of this historical evolution may, to some extent, be predictable. The model shows that the boundaries of language dialect regions are controlled by a length minimizing effect analogous to surface tension, mediated by variations in population density which can induce curvature, and by the shape of coastline or similar borders. The predictability of dialect regions arises because these effects will drive many complex, randomized early states toward one of a smaller number of stable final configurations. The model is able to reproduce observations and predictions of dialectologists. These include dialect continua, isogloss bundling, fanning, the wave-like spread of dialect features from cities, and the impact of human movement on the number of dialects that an area can support. The model also provides an analytical form for S\'{e}guy's Curve giving the relationship between geographical and linguistic distance, and a generalisation of the curve to account for the presence of a population centre. A simple modification allows us to analytically characterize the variation of language use by age in an area undergoing linguistic change

Topics arising from fictitious play dynamics

In this thesis, we present a few different topics arising in the study of the learning dynamics called fictitious play. We investigate the combinatorial properties of this dynamical system describing the strategy sequences of the players, and in particular deduce a combinatorial classification of zero-sum games with three strategies per player. We further obtain results about the limit sets and asymptotic payoff performance of fictitious play as a learning algorithm. In order to study coexistence of regular (periodic and quasi-periodic) and chaotic behaviour in fictitious play and a related continuous, piecewise affne flow on the threesphere, we look at its planar first return maps and investigate several model problems for such maps. We prove a non-recurrence result for non-self maps of regions in the plane, similar to Brouwer’s classical result for planar homeomorphisms. Finally, we consider a family of piecewise affne maps of the square, which is very similar to the first return maps of fictitious play, but simple enough for explicit calculations, and prove several results about its dynamics, particularly its invariant circles and regions

A survey of random processes with reinforcement

The models surveyed include generalized P\'{o}lya urns, reinforced random walks, interacting urn models, and continuous reinforced processes. Emphasis is on methods and results, with sketches provided of some proofs. Applications are discussed in statistics, biology, economics and a number of other areas.Comment: Published at http://dx.doi.org/10.1214/07-PS094 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org