Agent-Based Models and Human Subject Experiments

This paper considers the relationship between agent-based modeling and economic decision-making experiments with human subjects. Both approaches exploit controlled ``laboratory'' conditions as a means of isolating the sources of aggregate phenomena. Research findings from laboratory studies of human subject behavior have inspired studies using artificial agents in ``computational laboratories'' and vice versa. In certain cases, both methods have been used to examine the same phenomenon. The focus of this paper is on the empirical validity of agent-based modeling approaches in terms of explaining data from human subject experiments. We also point out synergies between the two methodologies that have been exploited as well as promising new possibilities.agent-based models, human subject experiments, zero- intelligence agents, learning, evolutionary algorithms

Learning from failure

We study decentralized learning in organizations. Decentralization is captured through a symmetry constraint on agents’ strategies. Among such attainable strategies, we solve for optimal and equilibrium strategies. We model the organization as a repeated game with imperfectly observable actions. A fixed but unknown subset of action profiles are successes and all other action profiles are failures. The game is played until either there is a success or the time horizon is reached. For any time horizon, including infinity, we demonstrate existence of optimal attainable strategies and show that they are Nash equilibria. For some time horizons, we can solve explicitly for the optimal attainable strategies and show uniqueness. The solution connects the learning behavior of agents to the fundamentals that characterize the organization: Agents in the organization respond more slowly to failure as the future becomes more important, the size of the organization increases and the probability of success decreases.Game theory

Learning in Repeated Games: Human Versus Machine

While Artificial Intelligence has successfully outperformed humans in complex combinatorial games (such as chess and checkers), humans have retained their supremacy in social interactions that require intuition and adaptation, such as cooperation and coordination games. Despite significant advances in learning algorithms, most algorithms adapt at times scales which are not relevant for interactions with humans, and therefore the advances in AI on this front have remained of a more theoretical nature. This has also hindered the experimental evaluation of how these algorithms perform against humans, as the length of experiments needed to evaluate them is beyond what humans are reasonably expected to endure (max 100 repetitions). This scenario is rapidly changing, as recent algorithms are able to converge to their functional regimes in shorter time-scales. Additionally, this shift opens up possibilities for experimental investigation: where do humans stand compared with these new algorithms? We evaluate humans experimentally against a representative element of these fast-converging algorithms. Our results indicate that the performance of at least one of these algorithms is comparable to, and even exceeds, the performance of people

Experience-weighted Attraction Learning in Normal Form Games

In ‘experience-weighted attraction’ (EWA) learning, strategies have attractions that reflect initial predispositions, are updated based on payoff experience, and determine choice probabilities according to some rule (e.g., logit). A key feature is a parameter δ that weights the strength of hypothetical reinforcement of strategies that were not chosen according to the payoff they would have yielded, relative to reinforcement of chosen strategies according to received payoffs. The other key features are two discount rates, φ and ρ, which separately discount previous attractions, and an experience weight. EWA includes reinforcement learning and weighted fictitious play (belief learning) as special cases, and hybridizes their key elements. When δ= 0 and ρ= 0, cumulative choice reinforcement results. When δ= 1 and ρ=φ, levels of reinforcement of strategies are exactly the same as expected payoffs given weighted fictitious play beliefs. Using three sets of experimental data, parameter estimates of the model were calibrated on part of the data and used to predict a holdout sample. Estimates of δ are generally around .50, φ around .8 − 1, and ρ varies from 0 to φ. Reinforcement and belief-learning special cases are generally rejected in favor of EWA, though belief models do better in some constant-sum games. EWA is able to combine the best features of previous approaches, allowing attractions to begin and grow flexibly as choice reinforcement does, but reinforcing unchosen strategies substantially as belief-based models implicitly do

Modelling the effect of learning and evolving rules on the use of common-pool resources

The extend to which common-pool resources are used and managed sustainably depends highly on incentives. Incentives influence the behaviour of individuals with respect to natural resource management and are determined by institutional arrangements comprising of formal and informal rules and markets. Changes in institutional arrangements will affect individual incentives and will therefore have an impact on resource use. In order to model the connections between institutional arrangements and the sustainable use of common-pool resources we must take into consideration the behaviour of individuals. Game-theoretical models appear to be an adequate modelling technique with which to assess the behaviour of individuals as well as the development of institutions with regards to common-pool resource regimes. The implementation of a game-theoretical framework in the form of an agent-based model appears to be a particularly appropriate tool with which to assess common-pool resource use regimes as such models enable the behaviour of different agents to be modelled as strategies. Traditionally with agent-based models, the strategies that agents pursue are given, with their expression endogenously determined by the set of rules which govern their behaviour. In this paper I focus on the implementation of mechanisms that also allow for rules to adapt endogenously. Such an approach will be applied to common-pool resource use in order to analyse the effect of rule changes.Institutional arrangements, agent-based modelling, learning, evolving rules

Learning to Respond: The Use of Heuristics in Dynamic Games

While many learning models have been proposed in the game theoretic literature to track individuals’ behavior, surprisingly little research has focused on how well these models describe human adaptation in changing dynamic environments. Analysis of human behavior demonstrates that people are often remarkably responsive to changes in their environment, on time scales ranging from millennia (evolution) to milliseconds (reflex). The goal of this paper is to evaluate several prominent learning models in light of a laboratory experiment on responsiveness in a lowinformation dynamic game subject to changes in its underlying structure. While history-dependent reinforcement learning models track convergence of play well in repeated games, it is shown that they are ill suited to these environments, in which sastisficing models accurately predict behavior. A further objective is to determine which heuristics, or “rules of thumb,” when incorporated into learning models, are responsible for accurately capturing responsiveness. Reference points and a particular type of experimentation are found to be important in both describing and predicting play.learning, limited information, dynamic games