Fictitious Play Outperforms Counterfactual Regret Minimization

We compare the performance of two popular algorithms, fictitious play and counterfactual regret minimization, in approximating Nash equilibrium in multiplayer games. Despite recent success of counterfactual regret minimization in multiplayer poker and conjectures of its superiority, we show that fictitious play leads to improved Nash equilibrium approximation over a variety of game classes and sizes.Comment: Fixed a bug in the 5-player CFR implementation from prior version and reran the 5-player experiment

On the Convergence of Model Free Learning in Mean Field Games

Learning by experience in Multi-Agent Systems (MAS) is a difficult and exciting task, due to the lack of stationarity of the environment, whose dynamics evolves as the population learns. In order to design scalable algorithms for systems with a large population of interacting agents (e.g. swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite. Recently, a very active burgeoning field studies the effects of diverse reinforcement learning algorithms for agents with no prior information on a stationary Mean Field Game (MFG) and learn their policy through repeated experience. We adopt a high perspective on this problem and analyze in full generality the convergence of a fictitious iterative scheme using any single agent learning algorithm at each step. We quantify the quality of the computed approximate Nash equilibrium, in terms of the accumulated errors arising at each learning iteration step. Notably, we show for the first time convergence of model free learning algorithms towards non-stationary MFG equilibria, relying only on classical assumptions on the MFG dynamics. We illustrate our theoretical results with a numerical experiment in a continuous action-space environment, where the approximate best response of the iterative fictitious play scheme is computed with a deep RL algorithm

The matching law and melioration learning: From individual decision-making to social interactions

Das Thema dieser Dissertation ist die Anwendung des „Matching Law” als Verhaltensannahme bei der Erklärung sozialer Phänomene. Das „Matching Law” ist ein Modell der behavioristischen Lerntheorie und sagt aus, dass die relative Häufigkeit der Wahl einer Handlung mit der relativen Häufigkeit der Belohnung dieser Handlung übereinstimmt. In der Dissertation werden verschiedene Probleme in Bezug auf die soziologische Anwendung des „Matching Law” erörtert. Aufbauend auf diesen Erkenntnissen wird das Entsprechungsgesetz in die ökonomische Entscheidungstheorie integriert und mit bestehenden Verhaltensprognosen theoretisch verglichen. Anschließend wird das Entsprechungsgesetz auf mehrere soziale Situationen angewandt. Dabei kommt ein Lernmodell zum Einsatz, welches als „Melioration Learning” bezeichnet wird und unter bestimmten Bedingungen zum Entsprechungsgesetz führt. Mit Hilfe dieses Lernmodells und agentenbasierter Simulationen werden Hypothesen zu sozialem Verhalten hergeleitet. Zunächst werden einfache Situationen mit nur zwei interagierenden Akteuren betrachtet. Dabei lassen sich durch das Entsprechungsgesetz einige Lösungskonzepte der Spieltheorie replizieren, obwohl weniger Annahmen bezüglich der kognitiven Fähigkeiten der Akteure und der verfügbaren Informationen gesetzt werden. Außerdem werden Interaktionen zwischen beliebig vielen Akteuren untersucht. Erstens lässt sich die Entstehung sozialer Konventionen über das Entsprechungsgesetz erklären. Zweitens wird dargestellt, dass die Akteure lernen, in einem Freiwilligendilemma oder einem Mehrpersonen-Gefangenendilemma zu kooperieren

Learning, evolution and price dispersion

Defence date: 4 March 1996Examining board: Prof. Ken Binmore, University College London ; Prof. Alan Kirman, EUI, Supervisor ; Prof. Mark Salmon, EUI, Co-supervisor ; Prof. John Sutton, London School of Economics ; Prof. Lerry Salmuelson, University of WisconsinFirst made available online: 5 September 201

Testing the Mere Exposure Effect in Videogaming

Due to proliferation of media and platforms it is becoming increasingly difficult for marketers to reach and engage consumers using traditional forms of mass media such as advertising. Marketers are turning to alternate forms of communication, such as brand placement in videogames as the games industry continues to grow. To date academic research appears inconclusive in terms of validating the use of videogames as a promotional tool. Moreover, there is a lack of empirical evidence concerning the effects on consumers and brands of marketing messages in the videogame environment. This aim of this study was to investigate whether exposure to brand placement affects unknown brand likeability as a result of mere exposure for game players and game watchers in videogames. The study adopted a quasi-experiment between group design, with a Control, Watch Group and Play Group (300 participants in total) and a post exposure questionnaire. Results suggest some support a mere exposure effect which is that a frequently presented brand placement in a videogame can have a positive effect on players and watchers’ brand attitudes, although they do not recall the brand. This is the first empirical study to investigate brand placement and mere exposure effects in videogames. Theoretically, the study contributes to knowledge concerning brand placement processing in videogames and builds on the existing paradigms of MEE, low-involvement processing, implicit and explicit processing and brand attitude formation. For game developers and brand owners, the study has implications for marketing communications strategy, and graphic design elements for the placements, design of videogames and the most effective position for placements in a game

MAGRITTE: a new multidimensional accelerated general-purpose radiative transfer code

Magritte is a new deterministic radiative transfer code. It is a ray-tracing code that computes the radiation field by solving the radiative transfer equation along a fixed set of rays for each grid cell. Its ray-tracing algorithm is independent of the type of input grid and thus can handle smoothed-particle hydrodynamics (SPH) particles, structured as well as unstructured grids. The radiative transfer solver is highly parallelized and optimized to have well scaling performance on several computer architectures. Magritte also contains separate dedicated modules for chemistry and thermal balance. These enable it to self-consistently model the interdependence between the radiation field and the local thermal and chemical states. The source code for Magritte will be made publically available at github.com/Magritte-code

Out-of-equilibrium economic dynamics and persistent polarisation

Most of economics is equilibrium economics of one sort or another. The study of outof- equilibrium economics has largely been neglected. This thesis, engaging with ideas and techniques from complexity science, develops frameworks and tools for out-of-equilibrium modelling. We initially focus our attention on models of exchange before examining methods of agent-based modelling. Finally we look at a set of models for social dynamics with nontrivial micro-macro interrelationships. Chapter 2 introduces complexity science and relevant economic concepts. In particular we examine the idea of complex adaptive systems, the application of complexity to economics, some key ideas from microeconomics, agent-based modelling and models of segregation and/or polarisation. Chapter 3 develops an out-of-equilibrium, fully decentralised model of bilateral exchange. Initially we study the limiting properties of our out-of-equilibrium dynamic, characterising the conditions required for convergence to pairwise and Pareto optimal allocation sets. We illustrate problems that can arise for a rigid version of the model and show how even a small amount of experimentation can overcome these. We investigate the model numerically characterising the speed of convergence and changes in ex post wealth. In chapter 4 we now explicitly model the trading structure on a network. We derive analytical results for this general network case. We investigate the e�ect of network structure on outcomes numerically and contrast the results with the fully connected case of chapter 3. We look at extensions of the model including a version with an endogenous network structure and a versions where agents can learn to accept a `worthless' but widely available good in exchanges. Chapter 5 outlines and demonstrates a new approach to agent-based modelling which draws on a number techniques from contemporary software engineering. We develop a prototype framework to illustrate how the ideas might be applied in practice in order to address methodological gaps in many current approaches. We develop example agent-based models and contrast the approach with existing agent-based modelling approaches and the kind of purpose built models which were used for the numerical results in chapters 3 and 4. Chapter 6 develops a new set of models for thinking about a wide range of social dynamics issues including human capital acquisition and migration. We analyse the models initially from a Nash equilibrium perspective. Both continuum and �nite versions of the model are developed and related. Using the criterion of stochastic stability we think about the long run behaviour of a version of the model. We introduce agent heterogeneity into the model. We conclude with a fully dynamic version of the model (using techniques from chapter 5) which looks at endogenous segregation