Approximating Mixed Nash Equilibria using Smooth Fictitious Play in Simultaneous Auctions

We investigate equilibrium strategies for bidding agents that participate in multiple, simultaneous second-price auctions with perfect substitutes. For this setting, previous research has shown that it is a best response for a bidder to participate in as many such auctions as there are available, provided that other bidders only participate in a single auction. In contrast, in this paper we consider equilibrium behaviour where all bidders participate in multiple auctions. For this new setting we consider mixed-strategy Nash equilibria where bidders can bid high in one auction and low in all others. By discretising the bid space, we are able to use smooth fictitious play to compute approximate solutions. Specifically, we find that the results do indeed converge to $\epsilon$-Nash mixed equilibria and, therefore, we are able to locate equilibrium strategies in such complex games where no known solutions previously existed

Complexity Theory, Game Theory, and Economics: The Barbados Lectures

This document collects the lecture notes from my mini-course "Complexity Theory, Game Theory, and Economics," taught at the Bellairs Research Institute of McGill University, Holetown, Barbados, February 19--23, 2017, as the 29th McGill Invitational Workshop on Computational Complexity. The goal of this mini-course is twofold: (i) to explain how complexity theory has helped illuminate several barriers in economics and game theory; and (ii) to illustrate how game-theoretic questions have led to new and interesting complexity theory, including recent several breakthroughs. It consists of two five-lecture sequences: the Solar Lectures, focusing on the communication and computational complexity of computing equilibria; and the Lunar Lectures, focusing on applications of complexity theory in game theory and economics. No background in game theory is assumed.Comment: Revised v2 from December 2019 corrects some errors in and adds some recent citations to v1 Revised v3 corrects a few typos in v

On the Convergence of Learning Algorithms in Bayesian Auction Games

Equilibrium problems in Bayesian auction games can be described as systems of differential equations. Depending on the model assumptions, these equations might be such that we do not have a rigorous mathematical solution theory. The lack of analytical or numerical techniques with guaranteed convergence for the equilibrium problem has plagued the field and limited equilibrium analysis to rather simple auction models such as single-object auctions. Recent advances in equilibrium learning led to algorithms that find equilibrium under a wide variety of model assumptions. We analyze first- and second-price auctions where simple learning algorithms converge to an equilibrium. The equilibrium problem in auctions is equivalent to solving an infinite-dimensional variational inequality (VI). Monotonicity and the Minty condition are the central sufficient conditions for learning algorithms to converge to an equilibrium in such VIs. We show that neither monotonicity nor pseudo- or quasi-monotonicity holds for the respective VIs. The second-price auction's equilibrium is a Minty-type solution, but the first-price auction is not. However, the Bayes--Nash equilibrium is the unique solution to the VI within the class of uniformly increasing bid functions, which ensures that gradient-based algorithms attain the {equilibrium} in case of convergence, as also observed in numerical experiments

Reinforcement Learning from Self-Play in Imperfect-Information Games

This thesis investigates artificial agents learning to make strategic decisions in imperfect-information games. In particular, we introduce a novel approach to reinforcement learning from self-play. We introduce Smooth UCT, which combines the game-theoretic notion of fictitious play with Monte Carlo Tree Search (MCTS). Smooth UCT outperformed a classic MCTS method in several imperfect-information poker games and won three silver medals in the 2014 Annual Computer Poker Competition. We develop Extensive-Form Fictitious Play (XFP) that is entirely implemented in sequential strategies, thus extending this prominent game-theoretic model of learning to sequential games. XFP provides a principled foundation for self-play reinforcement learning in imperfect-information games. We introduce Fictitious Self-Play (FSP), a class of sample-based reinforcement learning algorithms that approximate XFP. We instantiate FSP with neuralnetwork function approximation and deep learning techniques, producing Neural FSP (NFSP). We demonstrate that (approximate) Nash equilibria and their representations (abstractions) can be learned using NFSP end to end, i.e. interfacing with the raw inputs and outputs of the domain. NFSP approached the performance of state-of-the-art, superhuman algorithms in Limit Texas Hold’em - an imperfect-information game at the absolute limit of tractability using massive computational resources. This is the first time that any reinforcement learning algorithm, learning solely from game outcomes without prior domain knowledge, achieved such a feat

Mechanism Design and Analysis Using Simulation-Based Game Models.

As agent technology matures, it becomes easier to envision electronic marketplaces teeming with autonomous agents. Since agents are explicitly programmed to (nearly) optimally compete in these marketplaces, and markets themselves are designed with specific objectives in mind, tools are necessary for systematic analyses of strategic interactions among autonomous agents. While traditional game-theoretic approaches to the analysis of multi-agent systems can provide much insight, they are often inadequate, as they rely heavily on analytic tractability of the problem at hand; however, even mildly realistic models of electronic marketplaces contain enough complexity to render a fully analytic approach hopeless. To address questions not amenable to traditional theoretical approaches, I develop methods that allow systematic computational analysis of game-theoretic models in which the players' payoff functions are represented using simulations (i.e., simulation-based games). I develop a globally convergent algorithm for Nash equilibrium approximation in infinite simulation-based games, which I instantiate in the context of infinite games of incomplete information. Additionally, I use statistical learning techniques to improve the quality of Nash equilibrium approximation based on data collected from a game simulator. I also derive probabilistic confidence bounds and present convergence results about solutions of finite games modeled using simulations. The former allow an analyst to make statistically-founded statements about results based on game-theoretic simulations, while the latter provide formal justification for approximating game-theoretic solutions using simulation experiments. To address the broader mechanism design problem, I introduce an iterative algorithm for search in the design space, which requires a game solver as a subroutine. As a result, I enable computational mechanism design using simulation-based models of games by availing the designer of a set of solution tools geared specifically towards games modeled using simulations. I apply the developed computational techniques to analyze strategic procurement and answer design questions in a supply-chain simulation, as well as to analyze dynamic bidding strategies in sponsored search auctions. Indeed, the techniques I develop have broad potential applicability beyond electronic marketplaces: they are geared towards any system that features competing strategic players who respond to incentives in a way that can be reasonably predicted via a game-theoretic analysis.Ph.D.Computer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/60786/1/yvorobey_1.pd

Improving Behavior of Computer Game Bots Using Fictitious Play

In modern computer games, "bots" -intelligent realistic agents play a prominent role in the popularity of a game in the market. Typically, bots are modeled using finite-state machine and then programmed via simple conditional statements which are hard-coded in bots logic. Since these bots have become quite predictable to an experienced games player, a player might lose interest in the game. We propose the use of a game theoretic based learning rule called fictitious play for improving behavior of these computer game bots which will make them less predictable and hence, more a enjoyable game

DECENTRALIZED ALGORITHMS FOR NASH EQUILIBRIUM PROBLEMS – APPLICATIONS TO MULTI-AGENT NETWORK INTERDICTION GAMES AND BEYOND

Nash equilibrium problems (NEPs) have gained popularity in recent years in the engineering community due to their ready applicability to a wide variety of practical problems ranging from communication network design to power market analysis. There are strong links between the tools used to analyze NEPs and the classical techniques of nonlinear and combinatorial optimization. However, there remain significant challenges in both the theoretical and algorithmic analysis of NEPs. This dissertation studies certain special classes of NEPs, with the overall purpose of analyzing theoretical properties such as existence and uniqueness, while at the same time proposing decentralized algorithms that provably converge to solutions. The subclasses are motivated by relevant application examples