Learning by Doing vs. Learning from Others in a Principal-Agent Model

We introduce learning in a principal-agent model of stochastic output sharing under moral hazard. Without knowing the agents' preferences and technology the principal tries to learn the optimal agency contract. We implement two learning paradigms - social (learning from others) and individual (learning by doing). We use a social evolutionary learning algorithm (SEL) to represent social learning. Within the individual learning paradigm, we investigate the performance of reinforcement learning (RL), experience-weighted attraction learning (EWA), and individual evolutionary learning (IEL). Overall, our results show that learning in the principal-agent environment is very difficult. This is due to three main reasons: (1) the stochastic environment, (2) a discontinuity in the payoff space in a neighborhood of the optimal contract due to the participation constraint and (3) incorrect evaluation of foregone payoffs in the sequential game principal-agent setting. The first two factors apply to all learning algorithms we study while the third is the main contributor for the failure of the EWA and IEL models. Social learning (SEL), especially combined with selective replication, is much more successful in achieving convergence to the optimal contract than the canonical versions of individual learning from the literature. A modified version of the IEL algorithm using realized payoff evaluation performs better than the other individual learning models; however, it still falls short of the social learning's ability to converge to the optimal contract.learning, principal-agent model, moral hazard

Learning in a model of economic growth and development

We study a model of economic growth and development with a threshold externality. The model has one steady state with a low and stagnant level of income per capita and another steady state with a high and growing level of income per capita. Both of these steady states are locally stable under the perfect foresight assumption. We introduce learning into this environment. Learning acts as an equilibrium selection criterion and provides an interesting transition dynamic between steady states. We find that for sufficiently low initial values of human capital-values that would tend to characterize preindustrial economies-the system under learning spends a long period of time (an epoch) in the neighborhood of the low income steady state before finally transitioning to a neighborhood of the high income steady state. We urge that this type of transition dynamic provides a good characterization of the economic growth and development patterns that have been observed across countries.Economic development ; Economics

Social learning and monetary policy rules

We analyze the effects of social learning in a widely-studied monetary policy context. Social learning might be viewed as more descriptive of actual learning behavior in complex market economies. Ideas about how best to forecast the economy's state vector are initially heterogeneous. Agents can copy better forecasting techniques and discard those techniques which are less successful. We seek to understand whether the economy will converge to a rational expectations equilibrium under this more realistic learning dynamic. A key result from the literature in the version of the model we study is that the Taylor Principle governs both the uniqueness and the expectational stability of the rational expectations equilibrium when all agents learn homogeneously using recursive algorithms. We find that the Taylor Principle is not necessary for convergence in a social learning context. We also contribute to the use of genetic algorithm learning in stochastic environments.

Learning to alternate

The Individual Evolutionary Learning (IEL) model explains human subjects’ behavior in a wide range of repeated games which have unique Nash equilibria. Using a variation of ‘better response’ strategies, IEL agents quickly learn to play Nash equilibrium strategies and their dynamic behavior is like that of humans subjects. In this paper we study whether IEL can also explain behavior in games with gains from coordination. We focus on the simplest such game: the 2 person repeated Battle of Sexes game. In laboratory experiments, two patterns of behavior often emerge: players either converge rapidly to one of the stage game Nash equilibria and stay there or learn to coordinate their actions and alternate between the two Nash equilibria every other round. We show that IEL explains this behavior if the human subjects are truly in the dark and do not know or believe they know their opponent’s payoffs. To explain the behavior when agents are not in the dark, we need to modify the basic IEL model and allow some agents to begin with a good idea about how to play. We show that if the proportion of inspired agents with good ideas is chosen judiciously, the behavior of IEL agents looks remarkably similar to that of human subjects in laboratory experiments

Learning Benevolent Leadership in a Heterogenous Agents Economy

This paper studies the potential commitment value of cheap talkinflation announcements in an agent-based dynamic extension of theKydland-Prescott model. In every period, the policy maker makesa non-binding inflation announcement before setting the actualinflation rate. It updates its decisions using individual evolutionarylearning. The private agents can choose between two differentforecasting strategies: They can either set their forecast equal tothe announcement or compute it, at a cost, using an adaptive learningscheme. They switch between these two strategies as a function ofinformation about the associated payoffs they obtain throughword-of-mouth, choosing always the currently most favorable one.Weshow that the policy maker is able to sustain a situation with apositive but fluctuating fraction of believers. This equilibrium isPareto superior to the outcome predicted by standard theory. Theinfluence of changes in key parameters and the impact of transmissionof information among nonbelievers on the dynamics are studied.time inconsistency; bounded rationality; forecast and agentheterogeneity; cheap talk; evolutionary learning

Are sunspots learnable? An experimental investigation in a simple macroeconomic model

We conduct experiments with human subjects in a model with a positive production externality in which productivity is a nondecreasing function of the average level of employment of other firms. The model has three steady states and a sunspot equilibrium that fluctuates between the high and low steady states. Steady states are payoff ranked: low values give lower profits than higher values. We investigate whether subjects can learn a sunspot equilibrium. We observe coordination on the extrinsic announcements in our experimental economies. Cases of apparent convergence to the low and high steady states are also observed.PostprintPeer reviewe

Individual evolutionary learning with many agents

Individual Evolutionary Learning (IEL) is a learning model based on the evolution of a population of strategies of an individual agent. In prior work, IEL has been shown to be consistent with the behavior of human subjects in games with a small number of agents. In this paper, we examine the performance of IEL in games with many agents. We find IEL to be robust to this type of scaling. With the appropriate linear adjustment of the mechanism parameter, the convergence behavior of IEL in games induced by Groves–Ledyard mechanisms in quadratic environments is independent of the number of participants

Efficiency of continuous double auctions under individual evolutionary learning with full or limited information

In this paper we explore how specific aspects of market transparency and agents’ behavior affect the efficiency of the market outcome. In particular, we are interested whether learning behavior with and without information about actions of other participants improves market efficiency. We consider a simple market for a homogeneous good populated by buyers and sellers. The valuations of the buyers and the costs of the sellers are given exogenously. Agents are involved in consecutive trading sessions, which are organized as a continuous double auction with order book. Using Individual Evolutionary Learning agents submit price bids and offers, trying to learn the most profitable strategy by looking at their realized and counterfactual or “foregone” payoffs. We find that learning outcomes heavily depend on information treatments. Under full information about actions of others, agents’ orders tend to be similar, while under limited information agents tend to submit their valuations/ costs. This behavioral outcome results in higher price volatility for the latter treatment. We also find that learning improves allocative efficiency when compared to outcomes Zero-Intelligent traders

Learning to alternate

The Individual Evolutionary Learning (IEL) model explains human subjects’ behavior in a wide range of repeated games which have unique Nash equilibria. Using a variation of ‘better response’ strategies, IEL agents quickly learn to play Nash equilibrium strategies and their dynamic behavior is like that of humans subjects. In this paper we study whether IEL can also explain behavior in games with gains from coordination. We focus on the simplest such game: the 2 person repeated Battle of Sexes game. In laboratory experiments, two patterns of behavior often emerge: players either converge rapidly to one of the stage game Nash equilibria and stay there or learn to coordinate their actions and alternate between the two Nash equilibria every other round. We show that IEL explains this behavior if the human subjects are truly in the dark and do not know or believe they know their opponent’s payoffs. To explain the behavior when agents are not in the dark, we need to modify the basic IEL model and allow some agents to begin with a good idea about how to play. We show that if the proportion of inspired agents with good ideas is chosen judiciously, the behavior of IEL agents looks remarkably similar to that of human subjects in laboratory experiments

An Initial Implementation of the Turing Tournament to Learning in

Abstract We report on a design of a Turing tournament and its initial implementation to learning in repeated 2-person games. The principal objectives of the tournament, named after the original Turing Test, are (1) to find learning algorithms (emulators) that most closely simulate human behavior, (2) to find algorithms (detectors) that most accurately distinguish between humans and machines, and (3) to provide a demonstration of how to implement this methodology for evaluating models of human behavior. In order to test our concept, we developed the software and implemented a number of learning models well known in the literature and developed a few detectors. This initial implementation found significant differences in data generated by these learning models and humans, with the greatest ones in coordination games. Finally, we investigate the stability of our result with respect to different evaluation approaches