Simulating Evolutionary Games: A Python-Based Introduction

This paper is an introduction to agent-based simulation using the Python programming language. The core objective of the paper is to enable students, teachers, and researchers immediately to begin social-science simulation projects in a general purpose programming language. This objective is facilitated by design features of the Python programming language, which we very briefly discuss. The paper has a 'tutorial' component, in that it is enablement-focused and therefore strongly application-oriented. As our illustrative application, we choose a classic agent-based simulation model: the evolutionary iterated prisoner's dilemma. We show how to simulate the iterated prisoner's dilemma with code that is simple and readable yet flexible and easily extensible. Despite the simplicity of the code, it constitutes a useful and easily extended simulation toolkit. We offer three examples of this extensibility: we explore the classic result that topology matters for evolutionary outcomes, we show how player type evolution is affected by payoff cardinality, and we show that strategy evaluation procedures can affect strategy persistence. Social science students and instructors should find that this paper provides adequate background to immediately begin their own simulation projects. Social science researchers will additionally be able to compare the simplicity, readability, and extensibility of the Python code with comparable simulations in other languages.Agent-Based Simulation, Python, Prisoner's Dilemma

Computing Nash equilibria and evolutionarily stable states of evolutionary games

Stability analysis is an important research direction in evolutionary game theory. Evolutionarily stable states have a close relationship with Nash equilibria of repeated games, which are characterized by the folk theorem. When applying the folk theorem, one needs to compute the minimax profile of the game in order to find Nash equilibria. Computing the minimax profile is an NP-hard problem. In this paper we investigate a new methodology to compute evolutionary stable states based on the level-k equilibrium, a new refinement of Nash equilibrium in repeated games. A level-k equilibrium is implemented by a group of players who adopt reactive strategies and who have no incentive to deviate from their strategies simultaneously. Computing the level-k equilibria is tractable because the minimax payoffs and strategies are not needed. As an application, this paper develops a tractable algorithm to compute the evolutionarily stable states and the Pareto front of n-player symmetric games. Three games, including the iterated prisoner’s dilemma, are analyzed by means of the proposed methodology

Analysis of game playing agents with fingerprints

Evolutionary computation (EC) can create a vast number of strategies for playing simple games in a short time. Analysis of these strategies is typically more time-consuming than their production. As a result, analysis of strategies produced by an EC system is often lacking or restricted to the extraction of superficial summary Statistics and Probability; This thesis presents a technique for extracting a functional signature from evolved agents that play games. This signature can be used as a visualization of agent behavior in games with two moves and also provides a numerical target for clustering and other forms of automatic analysis. The fingerprint can be used to induce a similarity measure on the space of game strategies. This thesis develops fingerprints in the context of the iterated prisoner\u27s dilemma; we note that they can be computed for any two player simultaneous game with a finite set of moves. When using a clustering algorithm, the results are strongly influenced by the choice of the measure used to find the distance between or to compare the similarity of the data being clustered. The Euclidean metric, for example, rates a convex polytope as the most compact type of object and builds clusters that are contained in compact polytopes. Presented here is a general method, called multi-clustering, that compensates for the intrinsic shape of a metric or similarity measure. The method is tested on synthetic data sets that are natural for the Euclidean metric and on data sets designed to defeat k-means clustering with the Euclidean metric. Multi-clustering successfully discovers the designed cluster structure of all the synthetic data sets used with a minimum of parameter tuning. We then use multi-clustering and filtration on fingerprint data. Cellular representation is the practice of evolving a set of instructions for constructing a desired structure. This thesis presents a cellular encoding for finite state machines and specializes it to play the iterated prisoner\u27s dilemma. The impact on the character and behavior of finite state agents of using the cellular representation is investigated. For the cellular representation resented a statistically significant drop in the level of cooperation is found. Other differences in the character of the automaton generated with a direct and cellular representation are reported

Influence of Supply Chain Network Topology on the Evolution of Firm Strategies

This study investigates the influence of the topological structure of a supply chain network (SCN) on the evolution of cooperative and defective strategies adopted by the individual firms. First, a range of topologies representative of SCNs was generated using a fitness-based network growth model, which enabled cross comparisons by parameterising the network topologies with the power law exponent of their respective degree distributions. Then, the inter-firm links in each SCN were considered as repeated strategic interactions and were modelled by the Prisoner’s Dilemma game to represent the self-interested nature of the individual firms. This model is considered an agent-based model, where the agents are bound to their local neighbourhood by the network topology. A novel strategy update rule was then introduced to mimic the behaviour of firms. In particular, the heterogeneously distributed nature of the firm rationality was considered when they update their strategies at the end of each game round. Additionally, the payoff comparison against the neighbours was modelled to be strategy specific as opposed to accumulated payoff comparison analysis adopted in past work. It was found that the SCN topology, the level of rationality of firms and the relative strategy payoff differences are all essential elements in the evolution of cooperation. In summary, a tipping point was found in terms of the power law exponent of the SCN degree distribution, for achieving the highest number of co- operators. When the connection distribution of an SCN is highly unbalanced (such as in hub and spoke topologies) or well balanced (such as in random topologies), more difficult it is to achieve higher levels of co-operation among the firms. It was concluded that the scale-free topologies provide the best balance of hubs firms and lesser connected firms. Therefore, scale-free topologies are capable of achieving the highest proportion of co- operators in the firm population compared to other network topologies

Experimental and theoretical investigations of the emergence and sustenance of prosocial behavior in groups

Das Ziel der vorliegenden Arbeit war es, Bedingungen, unter denen prosoziales Verhalten entsteht, zu untersuchen. Hierzu wurden Vorhersagen und Annahmen der evolutionären Spieltheorie auf menschliches Kooperationsverhalten angewendet. Kooperatives Verhalten wird als evolutionäres Rätsel betrachtet, da natürliche Selektion betrügerisches Verhalten im Laufe der Zeit eigentlich begünstigen sollte. Nichtsdestotrotz ist Kooperation überall in der Natur vorzufinden. Per Definition ist kooperatives Verhalten kostenverursachend für den Handelnden und bietet gleichzeitig Vorteile oder Gewinne für andere Personen. Betrug oder Defektion dagegen verursachen keine Kosten, aber die bereitgestellten Vorteile anderer können dennoch genutzt werden. Infolgedessen ist Kooperation ohne die Unterstützung von Mechanismen, die die Gefahr der Ausbeutung reduzieren, keine evolutionär stabile Strategie. In der vorliegenden Arbeit wurden nun folgende Aspekte untersucht: (i) reziprokes Verhalten in multiplen paarweisen Interaktionen und (ii) die Auswirkung von sozialen Strukturen auf dyadische Beziehungen im Gefangenendilemma; und (iii) die Verwendung von Bestrafung unter Berücksichtigung der Möglichkeit von Konflikteskalation im öffentlichen‐Güter‐Spiel. In Kapitel 1 wurde untersucht, ob und in welcher Weise sich unterschiedliche Anzahlen von Interaktionspartnern auf kooperatives Verhalten im wiederholten Gefangenendilemma (kurz IPD) auswirken. Gemäß den Annahmen der direkten Reziprozität zeigte sich, dass die Versuchspersonen im traditionellen IPD mit unbekanntem Endpunkt mehrheitlich kooperierten. Insgesamt entsprach das Verhalten reaktiven Strategien ähnlich zu großzügigem Tit‐For‐Tat . Wenn die Versuchspersonen mit mehreren Partnern in drei IPDs gleichzeitig interagierten, sankt die durchschnittliche Kooperativität allerdings signifikant ab. Weiterführende Analysen zeigten, dass diese Versuchspersonen nur eine kooperative Beziehung ähnlich der Beziehung aus dem ein‐Partner IPD etablieren konnten, dass aber keine Kooperation in einer zweiten Beziehung aufgebaut werden konnten (das Kooperationslevel der dritten Beziehung lag zwischen diesen beiden). Diese Resultate widersprechen der traditionellen Annahme der evolutionären Spieltheorie, die eine Unabhängigkeit von Spielen annimmt, da eine erhöhte Versuchung in einigen Beziehungen zu bestehen scheint, wenn man mit drei anstelle von nur einem Sozialpartner interagiert. All dies deutet daraufhin, dass Modelle explizit den Effekt von unterschiedlichen Anzahlen von Partnern mitaufnehmen sollten, um so dem differenzierenden Verhalten eines Individuums gerecht zu werden. Ein Anfang stellt hier die Erforschung von Kooperation in heterogenen Netzwerken dar. Die Auswirkung von sozialen Strukturen auf Kooperation wurde in Kapitel 2 betrachtet. Beziehungen können durch eine zugrundeliegende Netzwerkstruktur charakterisiert werden. Bisher wurde diese Gegebenheit in theoretischen Überlegungen zumeist ignoriert und erst kürzlich fanden Netzwerkstrukturen Berücksichtigung in Modellen. Empirische Erkenntnisse zu diesen Modellen gab es bisher kaum, so dass sich dieses Kapitel genau dieser Lücke widmete. Hier interagierten die Versuchspersonen in mehreren, unabhängigen IPDs entweder innerhalb eines statischen oder eines dynamischen Netzwerkes. In Letzterem hatten die Versuchspersonen die Möglichkeit ihre sozialen Verbindungen nach jeder Gefangenendilemma‐Runde zu verändern. In Übereinstimmung mit theoretischen Modellen war die Kooperation in den dynamischen Netzwerken höher als in den statischen. Darüber hinaus veränderten die Versuchspersonen der dynamischen Netzwerke ihr soziales Umfeld durch ein bevorzugtes Beenden von Beziehungen zu Defektoren. Hierdurch fand eine Sortierung innerhalb des Netzwerkes statt und es bildeten sich kooperative Cliquen. Diese Selbstorganisation ist bemerkenswert, weil sie zusätzlich zum Effekt der direkten Reziprozität auftrat und weil die Versuchspersonen die Cliquenbildung auf Netzwerkebene nicht wahrnehmen konnten. Zusammenfassend zeigen diese Resultate die hohe Bedeutung von dynamischen sozialen Netzwerken auf und belegen, dass Strukturen höherer Ordnung neben dem Verhalten auf Individuumsebene entstehen können, welche dann wiederum in Wechselwirkung zum Selektionsdruck stehen können. In Kapitel 3 wurde der Einfluss von kostenverursachender Bestrafung, die potentiell zwischen Versuchspersonen eskalieren kann, auf kooperatives Verhalten untersucht. Vierergruppen spielten das öffentliche‐Güter‐Spiel mit fünf aufeinanderfolgenden Bestrafungsrunden. In der Regel sind Bestrafer aufgrund des Experimentalaufbaus vor Vergeltung geschützt, hier war dies jedoch nicht der Fall. Tatsächlich entwickelten sich Sequenzen von kostenverursachender Bestrafung zwischen Versuchspersonen, sogenannte Vendetten. Sie traten besonders häufig auf, wenn die Bestrafung als ungerecht oder als beliebig eingestuft wurde. Diese Resultate stehen im Widerspruch zu theoretischen Modellen, in denen Vendetten nicht evolvieren, da sie zu kostenintensiv sind und Defektion die bessere Verhaltensalternative darstellt. Nichtsdestotrotz stieg die Kooperation im Laufe der Zeit an. Dies ist vermutlich darauf zurückzuführen, dass die Bestrafung der ersten Runde noch auf defektierende Gruppenmitglieder gerichtet war, welches ihre Motivation in das öffentliche Gut einzuzahlen letztlich erhöhte. Ferner schienen einige Versuchspersonen sogar den möglichen Ausbruch von kostenintensiven Vendetten zu antizipierten und verzögerten ihre Bestrafung bis zum letztmöglichen Zeitpunkt. Diese Resultate zeigen auf, dass Modelle einen wichtigen Aspekt bisher vermissen lassen, da sowohl Tiere als auch Menschen häufig Vergeltung üben und sich gerade in menschlichen Gesellschaften Vendetten finden lassen. Spekulativ ist anzunehmen, dass Equity und Reputation gerade solche Aspekte dar stellen. Zusammenfassend konnten mit dieser Arbeit Bedingungen identifizieren werden, unter denen Kooperation zwischen nicht‐verwandten Personen entsteht und unter denen Kooperation niedrig ausfällt. Auf der einen Seite konnten neuere Modelle zu dynamischen sozialen Netzwerken empirisch untermauert werden ‐ auf der anderen Seite wurden Schwachpunkte in anderen Modellen ausgewiesen. Zusätzlich trug diese Arbeit weitere Erkenntnisse zum Verständnis der kostenverursachenden Bestrafung und der direkten Reziprozität beim Menschen bei

Inferring to C or not to C: Evolutionary games with Bayesian inferential strategies

Strategies for sustaining cooperation and preventing exploitation by selfish agents in repeated games have mostly been restricted to Markovian strategies where the response of an agent depends on the actions in the previous round. Such strategies are characterized by lack of learning. However, learning from accumulated evidence over time and using the evidence to dynamically update our response is a key feature of living organisms. Bayesian inference provides a framework for such evidence-based learning mechanisms. It is therefore imperative to understand how strategies based on Bayesian learning fare in repeated games with Markovian strategies. Here, we consider a scenario where the Bayesian player uses the accumulated evidence of the opponent's actions over several rounds to continuously update her belief about the reactive opponent's strategy. The Bayesian player can then act on her inferred belief in different ways. By studying repeated Prisoner's dilemma games with such Bayesian inferential strategies, both in infinite and finite populations, we identify the conditions under which such strategies can be evolutionarily stable. We find that a Bayesian strategy that is less altruistic than the inferred belief about the opponent's strategy can outperform a larger set of reactive strategies, whereas one that is more generous than the inferred belief is more successful when the benefit-to-cost ratio of mutual cooperation is high. Our analysis reveals how learning the opponent's strategy through Bayesian inference, as opposed to utility maximization, can be beneficial in the long run, in preventing exploitation and eventual invasion by reactive strategies.Comment: 13 pages, 9 figure

A complexidade da cooperação climática internacional

Although there are several collective efforts to address the problem of climate change, the main initiatives, such as the Kyoto Protocol and the Paris Agreement, have not shown satisfactory results so far. The difficulty in engaging states into effective coordinated cooperative practices can be explained as a consequence of neoclassical rationality, given that the characterization of states as rationality-endowed entities bound them to situations like the Prisoners' Dilemma (PD) game and its related collective action dilemmas. There are models that provide ways to circumvent PD and foster cooperation among selfish rational agents, such as the application of strategies based on reciprocity (Tit-for-Tat) in iterated games. However, these approaches do not avoid the short-sighted neoclassical rationality that lies at the root of the problem. Thus, in order to develop more productive approaches to the development of global climate change policies, I present a characterization of the international political system as a complex adaptive system (CAS) and argue that this perspective, along with models based on evolutionary games rather than iterated games, provide a more promising approach.Embora existam vários esforços coletivos para enfrentar o problema das mudanças climáticas, as principais iniciativas, como o Protocolo de Quioto e o Acordo de Paris, não têm apresentado resultados satisfatórios até o momento. A dificuldade em envolver os Estados em práticas cooperativas coordenadas efetivas pode ser explicada como consequência da racionalidade neoclássica, uma vez que a caracterização dos Estados como entidades dotadas de racionalidade os vincul a situações como o jogo do Dilema do Prisioneiro (DP), bem como os dilemas da ação coletiva relacionados a esse jogo. Existem modelos que fornecem maneiras de contornar o PD e promover a cooperação entre agentes racionais egoístas, como por exemplo a aplicação de estratégias baseadas na reciprocidade (Tit-for-Tat) em jogos iterados. No entanto, essas abordagens não evitam a racionalidade neoclássica de curto prazo, que está na raiz do problema. Assim, para desenvolver abordagens mais produtivas para o desenvolvimento de políticas globais para lidar com a mudança climática, apresento uma caracterização do sistema político internacional como um sistema adaptativo complexo (CAS) e argumento que essa perspectiva, acompanhada de modelos baseados em jogos evolutivos em vez de jogos iterados, fornece uma abordagem mais promissora

EVALUATING AND EXTENDING THE CONCEPT OF WISDOM OF CROWDS IN THE CONTEXT OF PROBLEM SOLVING

James Surowiecki in his book on the wisdom of crowds [Jame04] wrote about the decisions made based on the aggregation of information in groups. Knowing the many case studies and anecdotes which show the success of wisdom of crowds, he argues that under certain circumstances the wisdom of crowds is often better than that of any single member in the group. This paper provides a new way of problem solving– using the wisdom of crowds (collective wisdom) to handle continuous decision making problems, especially in a complex and rapidly changing world. By extending the concept of Wisdom of Crowds, the method of using collective wisdom is applied to various fields, from Prisoner‘s Dilemma to simplified stock market. Simulations are built to evaluate this new problem solving method and different aggregation strategies are suggested based on different environments

Game theoretic modeling and analysis : A co-evolutionary, agent-based approach

Ph.DDOCTOR OF PHILOSOPH