152 research outputs found
On Randomized Fictitious Play for Approximating Saddle Points Over Convex Sets
Given two bounded convex sets X\subseteq\RR^m and Y\subseteq\RR^n, specified by membership oracles, and a continuous convex-concave function F:X\times Y\to\RR, we consider the problem of computing an \eps-approximate saddle point, that is, a pair such that \sup_{y\in Y} F(x^*,y)\le \inf_{x\in X}F(x,y^*)+\eps. Grigoriadis and Khachiyan (1995) gave a simple randomized variant of fictitious play for computing an \eps-approximate saddle point for matrix games, that is, when is bilinear and the sets and are simplices. In this paper, we extend their method to the general case. In particular, we show that, for functions of constant "width", an \eps-approximate saddle point can be computed using O^*(\frac{(n+m)}{\eps^2}\ln R) random samples from log-concave distributions over the convex sets and . It is assumed that and have inscribed balls of radius and circumscribing balls of radius . As a consequence, we obtain a simple randomized polynomial-time algorithm that computes such an approximation faster than known methods for problems with bounded width and when \eps \in (0,1) is a fixed, but arbitrarily small constant. Our main tool for achieving this result is the combination of the randomized fictitious play with the recently developed results on sampling from convex sets
Application of multiplicative weights update method in algorithmic game theory
In this thesis, we apply the Multiplicative Weights Update Method (MWUM) to the design of approximation algorithms for some optimization problems in game-theoretic settings. Lavi and Swamy {LS05,LS11} introduced a randomized mechanism for combinatorial auctions that uses an approximation algorithm for the underlying optimization problem, so-called social welfare maximization and converts the approximation algorithm to a randomized mechanism that is { truthful-in-expectation}, which means each player maximizes its expected utility by telling the truth. The mechanism is powerful (e.g., see {LS05,LS11,CEF10,HKV11} for applications), but unlikely to be efficient in practice, because it uses the Ellipsoid method. In chapter 2, we follow the general scheme suggested by Lavi and Swamy and replace the Ellipsoid method with MWUM. This results in a faster and simpler approximately truthful-in-expectation mechanism. We also extend their assumption regarding the existence of an exact solution for the LP-relaxation of social welfare maximization. We assume that there exists an approximation algorithm for the LP and establish a new randomized approximation mechanism. In chapter 3, we consider the problem of computing an approximate saddle point, or equivalently equilibrium, for a convex-concave functions F: X\times Y\to \RR, where and are convex sets of arbitrary dimensions. Our main contribution is the design of a randomized algorithm for computing an \eps-approximation saddle point for . Our algorithm is based on combining a technique developed by Grigoriadis and Khachiyan {GK95}, which is a randomized variant of Brown's fictitious play {B51}, with the recent results on random sampling from convex sets (see, e.g., {LV06,V05}). The algorithm finds an \eps-approximation saddle point in an expected number of O\left(\frac{\rho^2(n+m)}{\eps^{2}}\ln\frac{R}{\eps}\right) iterations, where in each iteration two points are sampled from log-concave distributions over strategy sets. It is assumed that and have inscribed balls of radius and circumscribing balls of radius and . In particular, the algorithm requires O^*\left(\frac{\rho^2(n+m)^6}{\eps^{2}}\ln{R}\right) calls to a membership oracle, where suppresses polylogarithmic factors that depend on , , and \eps.In dieser Doktorarbeit verwenden wir die Multiplicative Weights Update Method (MWUM) fĂŒr den Entwurf von Approximationsalgorithmen fĂŒr bestimmte Optimierungsprobleme im spieltheoretischen Umfeld. Lavi und Swamy {LS05,LS11} prĂ€sentierten einen randomisierten Mechanismus fĂŒr kombinatorische Auktionen. Sie verwenden dazu einen Approximationsalgorithmus fĂŒr die Lösung des zugrundeliegenden Optimierungsproblem, das so genannte Social Welfare Maximization Problem, und wandeln diesen zu einem randomisierten Mechanismus um, der im Erwartungsfall anreizkompatibel ist. Dies bedeutet jeder Spieler erreicht den maximalen Gewinn, wenn er sich ehrlich verhĂ€lt. Der Mechanismus ist sehr mĂ€chtig (siehe {LS05,LS11,CEF10,HKV11} fĂŒr Anwendungen); trotzdem ist es unwahrscheinlich, dass er in der Praxis effizient ist, da hier die Ellipsoidmethode verwendet wird.
In Kapitel 2 folgen wir dem von Lavi und Swamy vorgeschlagenem Schema und ersetzen die Ellipsoidmethode durch MWUM. Das Ergebnis ist ein schnellerer, einfacherer und im Erwartungsfall anreizkompatibler Approximationsmechanismus.
Wir erweitern ihre Annahme zur Existenz einer exakten Lösung der LP-Relaxierung fĂŒr das Social Welfare Maximization Problem. Wir nehmen an, dass ein Approximationsalgorithmus fĂŒr das LP existiert und beschreiben darauf basierend einen neuen randomisierten Approximationsmechanismus.
In Kapitel 3 betrachten wir das Problem fĂŒr konvexe und konkave Funktionen , wobei und konvexe Mengen von beliebiger Dimension sind, einen Sattelpunkt zu approximieren (oder gleichbedeutend ein Equilibrium). Unser Hauptbeitrag ist der Entwurf eines randomisierten Algorithmus zur Berechnung einer -NĂ€herung eines Sattelpunktes von . Unser Algorithmus beruht auf der Kombination einer Technik entwickelt durch Grigoriadis und Khachiyan {GK95}, welche eine zufallsbasierte Variation von Browns Fictitious Play {B51} ist, mit kĂŒrzlich erschienenen Resultaten im Bereich der zufĂ€lligen Stichprobennahme aus konvexen Mengen (siehe {LV06,V05}). Der Algorithmus findet eine -NĂ€herung eines Sattelpunktes im Erwartungsfall in Rechenschritten, wobei in jedem Rechenschritt zwei Punkte zufĂ€llig gemÀà einer log-konkaven Verteilungen ĂŒber Strategiemengen gezogen werden. Hier nehmen wir an, dass und einbeschriebene Kugeln mit Radius und umschreibende Kugeln von Radius R besitzen und . Der Algorithmus benötigt dabei Aufrufe eines Zugehörigkeitsorakels, hier versteckt polylogarithmische Faktoren, die von und abhĂ€ngen
Frank-Wolfe Algorithms for Saddle Point Problems
We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained
smooth convex-concave saddle point (SP) problems. Remarkably, the method only
requires access to linear minimization oracles. Leveraging recent advances in
FW optimization, we provide the first proof of convergence of a FW-type saddle
point solver over polytopes, thereby partially answering a 30 year-old
conjecture. We also survey other convergence results and highlight gaps in the
theoretical underpinnings of FW-style algorithms. Motivating applications
without known efficient alternatives are explored through structured prediction
with combinatorial penalties as well as games over matching polytopes involving
an exponential number of constraints.Comment: Appears in: Proceedings of the 20th International Conference on
Artificial Intelligence and Statistics (AISTATS 2017). 39 page
Model-Free Online Learning in Unknown Sequential Decision Making Problems and Games
Regret minimization has proved to be a versatile tool for tree-form
sequential decision making and extensive-form games. In large two-player
zero-sum imperfect-information games, modern extensions of counterfactual
regret minimization (CFR) are currently the practical state of the art for
computing a Nash equilibrium. Most regret-minimization algorithms for tree-form
sequential decision making, including CFR, require (i) an exact model of the
player's decision nodes, observation nodes, and how they are linked, and (ii)
full knowledge, at all times t, about the payoffs -- even in parts of the
decision space that are not encountered at time t. Recently, there has been
growing interest towards relaxing some of those restrictions and making regret
minimization applicable to settings for which reinforcement learning methods
have traditionally been used -- for example, those in which only black-box
access to the environment is available. We give the first, to our knowledge,
regret-minimization algorithm that guarantees sublinear regret with high
probability even when requirement (i) -- and thus also (ii) -- is dropped. We
formalize an online learning setting in which the strategy space is not known
to the agent and gets revealed incrementally whenever the agent encounters new
decision points. We give an efficient algorithm that achieves
regret with high probability for that setting, even when the agent faces an
adversarial environment. Our experiments show it significantly outperforms the
prior algorithms for the problem, which do not have such guarantees. It can be
used in any application for which regret minimization is useful: approximating
Nash equilibrium or quantal response equilibrium, approximating coarse
correlated equilibrium in multi-player games, learning a best response,
learning safe opponent exploitation, and online play against an unknown
opponent/environment.Comment: Full version. The body of the paper appeared in the proceedings of
the AAAI 2021 conferenc
Spatial evolution of human dialects
The geographical pattern of human dialects is a result of history. Here, we
formulate a simple spatial model of language change which shows that the final
result of this historical evolution may, to some extent, be predictable. The
model shows that the boundaries of language dialect regions are controlled by a
length minimizing effect analogous to surface tension, mediated by variations
in population density which can induce curvature, and by the shape of coastline
or similar borders. The predictability of dialect regions arises because these
effects will drive many complex, randomized early states toward one of a
smaller number of stable final configurations. The model is able to reproduce
observations and predictions of dialectologists. These include dialect
continua, isogloss bundling, fanning, the wave-like spread of dialect features
from cities, and the impact of human movement on the number of dialects that an
area can support. The model also provides an analytical form for S\'{e}guy's
Curve giving the relationship between geographical and linguistic distance, and
a generalisation of the curve to account for the presence of a population
centre. A simple modification allows us to analytically characterize the
variation of language use by age in an area undergoing linguistic change
Topics arising from fictitious play dynamics
In this thesis, we present a few different topics arising in the study of the learning dynamics
called fictitious play. We investigate the combinatorial properties of this dynamical system
describing the strategy sequences of the players, and in particular deduce a combinatorial
classification of zero-sum games with three strategies per player. We further obtain results
about the limit sets and asymptotic payoff performance of fictitious play as a learning
algorithm.
In order to study coexistence of regular (periodic and quasi-periodic) and chaotic
behaviour in fictitious play and a related continuous, piecewise affne flow on the threesphere,
we look at its planar first return maps and investigate several model problems for
such maps. We prove a non-recurrence result for non-self maps of regions in the plane,
similar to Brouwerâs classical result for planar homeomorphisms. Finally, we consider a
family of piecewise affne maps of the square, which is very similar to the first return maps
of fictitious play, but simple enough for explicit calculations, and prove several results about
its dynamics, particularly its invariant circles and regions
A survey of random processes with reinforcement
The models surveyed include generalized P\'{o}lya urns, reinforced random
walks, interacting urn models, and continuous reinforced processes. Emphasis is
on methods and results, with sketches provided of some proofs. Applications are
discussed in statistics, biology, economics and a number of other areas.Comment: Published at http://dx.doi.org/10.1214/07-PS094 in the Probability
Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Recommended from our members
New Discretization Methods for the Numerical Approximation of PDEs
The construction and mathematical analysis of numerical methods for PDEs is a fundamental area of modern applied mathematics. Among the various techniques that have been proposed in the past, some â in particular, finite element methods, â have been exceptionally successful in a range of applications. There are however a number of important challenges that remain, including the optimal adaptive finite element approximation of solutions to transport-dominated diffusion problems, the efficient numerical approximation of parametrized families of PDEs, and the efficient numerical approximation of high-dimensional partial differential equations (that arise from stochastic analysis and statistical physics, for example, in the form of a backward Kolmogorov equation, which, unlike its formal adjoint, the forward Kolmogorov equation, is not in divergence form, and therefore not directly amenable to finite element approximation, even when the spatial dimension is low). In recent years several original and conceptionally new ideas have emerged in order to tackle these open problems.
The goal of this workshop was to discuss and compare a number of novel approaches, to study their potential and applicability, and to formulate the strategic goals and directions of research in this field for the next five years
- âŠ