Convergence in Finite Cournot Oligopoly with Social and Individual Learning

Convergence to Nash equilibrium in Cournot oligopoly is a problem that recurrently arises as a subject of study in economics. The development of evolutionary game theory has provided an equilibrium concept more directly connected with adjustment dynamics and the evolutionary stability of the equilibria of the Cournot game has been studied by several articles. Several articles show that the Walrasian equilibrium is the stable evolutionary solution of the Cournot game. Vriend (2000) proposes to use genetic algorithm for studying learning dynamics in this game and obtains convergence to Cournot equilibrium with individual learning. We show in this article how social learning gives rise to Walras equilibrium and why, in a general setup, individual learning can effectively yield convergence to Cournot instead of Walras equilibrium. We illustrate these general results by computational experiments.Cournot oligopoly; Learning; Evolution; Selection; Evolutionary stability; Nash equilibrium; Genetic algorithms

The separation of economic versus EA parameters in EA-learning.

Agent-based computational economics (ACE) combines elements from economics and computer science. In this paper, we focus on the relation between the evolutionary technique that is used and the economic problem that is modeled. Current economic simulations often derive parameter settings for the genetic algorithm directly from the values of the economic model parameters. In this paper we show that this practice may hinder the performance of the GA and thereby hinder agent learning. More specifically, we show that economic model parameters and evolutionary algorithm parameters should be treated separately by comparing two widely used approaches to population learning with respect to their convergence properties and robustnes

Coevolutionary Genetic Algorithms for Establishing Nash Equilibrium in Symmetric Cournot Games

We use co-evolutionary genetic algorithms to model the players' learning process in several Cournot models, and evaluate them in terms of their convergence to the Nash Equilibrium. The \social-learning" versions of the two co-evolutionary algorithms we introduce, establish Nash Equilibrium in those models, in contrast to the \individual learning" versions which, as we see here, do not imply the convergence of the players' strategies to the Nash outcome. When players use \canonical co-evolutionary genetic algorithms" as learning algorithms, the process of the game is an ergodic Markov Chain, and therefore we analyze simulation results using the relevant methodology, to find that in the \social" case, states leading to NE play are highly frequent at the stationary distribution of the chain, in contrast to the \individual learning" case, when NE is not reached at all in our simulations; to ftnd that the expected Hamming distance of the states at the limiting distribution from the \NE state" is significantly smaller in the \social" than in the \individual learning case"; to estimate the expected time that the \social" algorithms need to get to the \NE state" and verify their robustness and finally to show that a large fraction of the games played are indeed at the Nash Equilibrium.Genetic Algorithms, Cournot oligopoly, Evolutionary Game Theory, Nash Equilibrium

Determination of sequential best replies in n-player games by Genetic Algorithms

An iterative algorithm for establishing the Nash Equilibrium in pure strategies (NE) is proposed and tested in Cournot Game models. The algorithm is based on the convergence of sequential best responses and the utilization of a genetic algorithm for determining each player's best response to a given strategy profile of its opponents. An extra outer loop is used, to address the problem of finite accuracy, which is inherent in genetic algorithms, since the set of feasible values in such an algorithm is finite. The algorithm is tested in five Cournot models, three of which have convergent best replies sequence, one with divergent sequential best replies and one with \local NE traps"(Son and Baldick 2004), where classical local search algorithms fail to identify the Nash Equilibrium. After a series of simulations, we conclude that the algorithm proposed converges to the Nash Equilibrium, with any level of accuracy needed, in all but the case where the sequential best replies process diverges.Genetic Algorithms, Cournot oligopoly, Best Response, Nash Equilibrium

Economic Modeling Using Evolutionary Algorithms: The Effect of a Binary Encoding of Strategies

We are concerned with evolutionary algorithms that are employed for economic modeling purposes. We focus in particular on evolutionary algorithms that use a binary encoding of strategies. These algorithms, commonly referred to as genetic algorithms, are popular in agent-based computational economics research. In many studies, however, there is no clear reason for the use of a binary encoding of strategies. We therefore examine to what extent the use of such an encoding may influence the results produced by an evolutionary algorithm. It turns out that the use of a binary encoding can have quite significant effects. Since these effects do not have a meaningful economic interpretation, they should be regarded as artifacts. Our findings indicate that in general the use of a binary encoding is undesirable. They also highlight the importance of employing evolutionary algorithms with a sensible economic interpretation

A Theoretical Analysis of Cooperative Behavior in Multi-Agent Q-learning

A number of experimental studies have investigated whether cooperative behavior may emerge in multi-agent Q-learning. In some studies cooperative behavior did emerge, in others it did not. This report provides a theoretical analysis of this issue. The analysis focuses on multi-agent Q-learning in iterated prisoner’s dilemmas. It is shown that under certain assumptions cooperative behavior may emerge when multi-agent Q-learning is applied in an iterated prisoner’s dilemma. An important consequence of the analysis is that multi-agent Q-learning may result in non-Nash behavior. It is found experimentally that the theoretical results derived in this report are quite robust to violations of the underlying assumptions.Cooperation;Multi-Agent Q-Learning;Multi-Agent Reinforcement Learning;Nash Equilibrium;Prisoner’s Dilemma

Coevolutionary Genetic Algorithms for Establishing Nash Equilibrium in Symmetric Cournot Games

We use co-evolutionary genetic algorithms to model the players' learning process in several Cournot models, and evaluate them in terms of their convergence to the Nash Equilibrium. The "social-learning" versions of the two co-evolutionary algorithms we introduce, establish Nash Equilibrium in those models, in contrast to the "individual learning" versions which, as we see here, do not imply the convergence of the players' strategies to the Nash outcome. When players use "canonical co-evolutionary genetic algorithms" as learning algorithms, the process of the game is an ergodic Markov Chain, and therefore we analyze simulation results using both the relevant methodology and more general statistical tests, to find that in the "social" case, states leading to NE play are highly frequent at the stationary distribution of the chain, in contrast to the "individual learning" case, when NE is not reached at all in our simulations; to find that the expected Hamming distance of the states at the limiting distribution from the "NE state" is significantly smaller in the "social" than in the "individual learning case"; to estimate the expected time that the "social" algorithms need to get to the "NE state" and verify their robustness and finally to show that a large fraction of the games played are indeed at the Nash Equilibrium.Comment: 18 pages, 4 figure

Learning to Collude Tacitly on Production Levels by Oligopolistic Agents

Classical oligopoly theory has strong analytical foundations but is weak in capturing the operating environment of oligopolists and the available knowledge they have for making decisions, areas in which the management literature is relevant. We use agent-based models to simulate the impact on firm profitability of policies that oligopolists can pursue when setting production levels. We develop an approach to analyzing simulation results that makes use of nonparametric statistical tests, taking advantage of the large amounts of data generated by simulations, and avoiding the assumption of normality that does not necessarily hold. Our results show that in a quantity game, a simple exploration rule, which we call Probe and Adjust, can find either the Cournot equilibrium or the monopoly solution depending on the measure of success chosen by the firms. These results shed light on how tacit collusion can develop within an oligopoly

A Theoretical Analysis of Cooperative Behavior in Multi-Agent Q-learning

A number of experimental studies have investigated whether cooperative behavior may emerge in multi-agent Q-learning. In some studies cooperative behavior did emerge, in others it did not. This report provides a theoretical analysis of this issue. The analysis focuses on multi-agent Q-learning in iterated prisoner’s dilemmas. It is shown that under certain assumptions cooperative behavior may emerge when multi-agent Q-learning is applied in an iterated prisoner’s dilemma. An important consequence of the analysis is that multi-agent Q-learning may result in non-Nash behavior. It is found experimentally that the theoretical results derived in this report are quite robust to violations of the underlying assumptions