An Experimental Investigation of Fairness and Reciprocal Behavior in a Triangular Principal'-Multiagent Relationship.

A laboratory investigation of a simple agency model that allow to study how the principal's fairness affects the attitude towards cooperation between two interdependent agents performing a simple production task.principal-agent theory; prisoner's dilemma; reciprocity; fairness; experimental economics

THE EFFECTS OF FEEDBACK ON COOPERATION IN THE PRISONER’S DILEMMA GAME SIMULATING A CLOSED MARKET SCENARIO

This study explores the effects of feedback on cooperation in the iterated prisoner’s dilemma game (PDG). Four sources of feedback were identified: peer, buyer, market and cultural feedback. Peer and buyer feedback were intrinsic to the PDG, for they were analyzed, but not manipulated. Market and cultural feedback comprised independent variables and their effects were measured on players’ and group cooperation (dependent variables). Twenty-seven participants played a PDG, divided in 9 groups of 3 players each. Cooperation was measured as rates of individual players’ cooperative X choices, and as aggregate products within groups. At the molecular (moment-to-moment) level, there was a significant within-subjects main effect of the market feedback F(1, 28) = 6.50, p = .02, ?p2 = .19. At the molar level, there was no significant effect of the market feedback, nor of the cultural feedback. It was not possible to establish a metacontingency between recurrent group cooperation and positive contingent group consequences. Players displayed sub-optimal choice behavior, seeking to maximize relative earnings within their group (defecting) over absolute earnings (cooperating). These results are discussed in light of how the source of feedback may sustain cooperation or defection in the PDG, and their implications in organizational settings. Reinforcing cooperative behaviors can be key to the maintenance and development of any organization, for informative performance feedback may not suffice. This study contributes to the understanding of economic decisional behavior in groups from a cultural selectionist perspective.Keywords: choice, cooperation, feedback, metacontingency, prisoner’s dilemma gameThis study explores the effects of feedback on cooperation in the iterated prisoner’s dilemma game (PDG). Four sources of feedback were identified: peer, buyer, market and cultural feedback. Peer and buyer feedback were intrinsic to the PDG, for they were analyzed, but not manipulated. Market and cultural feedback comprised independent variables and their effects were measured on players’ and group cooperation (dependent variables). Twenty-seven participants played a PDG, divided in 9 groups of 3 players each. Cooperation was measured as rates of individual players’ cooperative X choices, and as aggregate products within groups. At the molecular (moment-to-moment) level, there was a significant within-subjects main effect of the market feedback F(1, 28) = 6.50, p = .02, ?p2 = .19. At the molar level, there was no significant effect of the market feedback, nor of the cultural feedback. It was not possible to establish a metacontingency between recurrent group cooperation and positive contingent group consequences. Players displayed sub-optimal choice behavior, seeking to maximize relative earnings within their group (defecting) over absolute earnings (cooperating). These results are discussed in light of how the source of feedback may sustain cooperation or defection in the PDG, and their implications in organizational settings. Reinforcing cooperative behaviors can be key to the maintenance and development of any organization, for informative performance feedback may not suffice. This study contributes to the understanding of economic decisional behavior in groups from a cultural selectionist perspective.Keywords: choice, cooperation, feedback, metacontingency, prisoner’s dilemma gam

Stylised Facts and the Contribution of Simulation to the Economic Analysis of Budgeting

The application of computer simulation as a research method raises two important questions: (1) Does simulation really offer added value over established methods? (2) How can the danger of arbitrariness caused by the extended modelling possibilities be minimised? We present the concept of stylised facts as a methodological basis for approaching these questions systematically. In particular, stylised facts provide a point of reference for a comparative analysis of models intended to explain an observable phenomenon. This is shown with reference to a recent discussion in the "economic analysis of accounting" literature where established methods, i.e. game theory, as well as computer simulations are used: the susceptibility of the "Groves mechanism" to collusion. Initially, we identify six stylised facts on the stability of collusion in empirical studies. These facts serve as a basis for the subsequent comparison of four theoretical models with reference to the above questions: (1) We find that the simulation models of Krapp and Deliano offer added value in comparison to the game theoretical models. They can be related to more stylised facts, achieve a better reproduction and exhibit far greater potential for incorporating yet unaddressed stylised facts. (2) Considered in the light of the stylised facts to which the models can be related, Deliano's simulation model exhibits considerable arbitrariness in model design and lacks information on its robustness. In contrast, Krapp demonstrates that this problem is not inherent to the method. His simulation model methodically extends its game theoretical predecessors, leaving little room for arbitrary model design or questionable parameter calibration. All in all, the stylisedfactsconcept proved to be very useful in dealing with the questions simulation researchers are confronted with. Moreover, a "research landscape" emerges from the derived stylised facts pinpointing issues yet to be addressed.Computer Simulation, Stylised Facts, Methodology, Groves Mechanism, Collusion, Game Theory

An experimental investigation of fairness and reciprocal behavior in a simple principal-multiagent relationship

Issues of fairness within the agency framework have traditionally been investigated - both theoretically and experimentally - within two alternative approaches: a "vertical", hierarchical framework (studies of fairness in the agency relationship between one principal and one single agent) and a "horizontal", agent-to-agent framework (studies of reciprocity in peers' interactions under alternative incentive schemes). We explore in the laboratory a game which integrates vertical and horizontal relationships and allows to investigate how principal's fairness affects cooperation between two interdependent agents performing a simple production game. We set a 2-stage game where, in the second stage, agents play a prisoner's dilemma game and, in the first stage, the principal can withdraw any share of the output generated by the agents in the second stage. Despite theory predicting that no fairness should be observed by the principal and no cooperation should be observed by agents, our experimental findings show that agents are to some extent sensitive to principal's fairness. When the principal plays unfair (fair) agents are frequently observed to jointly defect (cooperate). Thus, fairness considerations on principal's actions may act as a coordination device for the agents and reciprocal behavior may, as a result, affect their propensity towards cooperation or defection in the game

Analysis of game playing agents with fingerprints

Evolutionary computation (EC) can create a vast number of strategies for playing simple games in a short time. Analysis of these strategies is typically more time-consuming than their production. As a result, analysis of strategies produced by an EC system is often lacking or restricted to the extraction of superficial summary Statistics and Probability; This thesis presents a technique for extracting a functional signature from evolved agents that play games. This signature can be used as a visualization of agent behavior in games with two moves and also provides a numerical target for clustering and other forms of automatic analysis. The fingerprint can be used to induce a similarity measure on the space of game strategies. This thesis develops fingerprints in the context of the iterated prisoner\u27s dilemma; we note that they can be computed for any two player simultaneous game with a finite set of moves. When using a clustering algorithm, the results are strongly influenced by the choice of the measure used to find the distance between or to compare the similarity of the data being clustered. The Euclidean metric, for example, rates a convex polytope as the most compact type of object and builds clusters that are contained in compact polytopes. Presented here is a general method, called multi-clustering, that compensates for the intrinsic shape of a metric or similarity measure. The method is tested on synthetic data sets that are natural for the Euclidean metric and on data sets designed to defeat k-means clustering with the Euclidean metric. Multi-clustering successfully discovers the designed cluster structure of all the synthetic data sets used with a minimum of parameter tuning. We then use multi-clustering and filtration on fingerprint data. Cellular representation is the practice of evolving a set of instructions for constructing a desired structure. This thesis presents a cellular encoding for finite state machines and specializes it to play the iterated prisoner\u27s dilemma. The impact on the character and behavior of finite state agents of using the cellular representation is investigated. For the cellular representation resented a statistically significant drop in the level of cooperation is found. Other differences in the character of the automaton generated with a direct and cellular representation are reported

Individual versus group success in round robin tournament of an ability-based two-player game

This thesis studies the development of cooperation and measures of success among self-interested agents in a defined ability-based two-player asymmetric game that is structured as a round-robin tournament. Our research is motivated by the notion that in many systems cooperative behaviour depends on some parameters that are usually not considered in existing research. These include: the balance between individual activity and interaction with others; the impact of agents' ability levels; and the need to maintain balance between individual and group performance. In this thesis, we examine all these issues by using a defined game-theoretic modelling and simulation framework. Our simulation experiments on six agent group compositions establish some patterns of how an agent's ability and strategy impact its individual and overall group performance. The results demonstrate that the design framework supports methodical comparative studies of strategy profiles with respect to specific individual and group performance measures.cooperative behaviourtwo-player asymmetric gametheoretic modelling and simulation framewor

Foresighted policy gradient reinforcement learning: solving large-scale social dilemmas with rational altruistic punishment

Many important and difficult problems can be modeled as “social dilemmas”, like Hardin's Tragedy of the Commons or the classic iterated Prisoner's Dilemma. It is well known that in these problems, it can be rational for self-interested agents to promote and sustain cooperation by altruistically dispensing costly punishment to other agents, thus maximizing their own long-term reward. However, self-interested agents using most current multi-agent reinforcement learning algorithms will not sustain cooperation in social dilemmas: the algorithms do not sufficiently capture the consequences on the agent's reward of the interactions that it has with other agents. Recent more foresighted algorithms specifically account for such expected consequences, and have been shown to work well for the small-scale Prisoner's Dilemma. However, this approach quickly becomes intractable for larger social dilemmas. Here, we advance on this work and develop a “teach/learn” stateless foresighted policy gradient reinforcement learning algorithm that applies to Social Dilemma's with negative, unilateral side-payments, in the from of costly punishment. In this setting, the algorithm allows agents to learn the most rewarding actions to take with respect to both the dilemma (Cooperate/Defect) and the “teaching” of other agent's behavior through the dispensing of punishment. Unlike other algorithms, we show that this approach scales well to large settings like the Tragedy of the Commons. We show for a variety of settings that large groups of self-interested agents using this algorithm will robustly find and sustain cooperation in social dilemmas where adaptive agents can punish the behavior of other similarly adaptive agents

An Analysis of the Traveler\u27s Dilemma with Experimental Evidence

Game theory studies how people should respond in strategic situations and is naturally used for predictive purposes. The optimal strategy predictions yielded by game theoretic reasoning can be surprising when they conflict with preconceived notions of how to play, i.e. the common sense strategy. Consequently, the game theoretically optimal strategy may be a poor predictor of how individuals actually behave in real-life strategic situations. In order to accurately model such situations for predictive purposes it is important to know the limitations of the current game theoretic tools. In the traveler\u27s dilemma, Kaushik Basu presents a parable to illustrate how game theoretic reasoning and intuition can be at odds. The parable is as follows: Two travelers are returning home from a vacation where they purchased identical souvenirs. These souvenirs are, of course, routinely destroyed by the airline. The souvenirs were purchased with cash in an open-air market and as a result the travelers do not have receipts. The airline official in charge of damage claims wants to compensate the travelers fairly but has no way of determining the actual purchase price of the souvenirs. In an attempt to avoid spurious claims the official proposes a method to determine the amount awarded. Each traveler must submit a claim that lies between a known minimum and maximum. (The minimum bound can be thought of as that level of claim below which the airline never disputes for cost reasons and the maximum bound can be thought of as the most the airline\u27s insurance company would pay absent a special policy). If the claims submitted are equal then both receive the amount claimed. However, if traveler 1 submits a lower claim than traveler 2, traveler 1 is considered honest and receives the lower claim plus a reward for honesty (ideally in frequent flyer miles thereby ensuring that the airline will have an opportunity to destroy those items which it missed on the first pass). Traveler 2 also receives the lower claim but in addition a symmetric penalty for lying is levied. Travelers report their claims simultaneously i.e., without knowledge of the other traveler\u27s claim. This ensures that simple, collusive agreements cannot be reached because any such agreement involves a non-credible promise (the rewards are structured such that there it is never in a rational player\u27s best interest to play as he agreed). Game theoretic analysis suggests that two rational players will report the minimum claim. Moreover, this result is a direct consequence of both players attempting to maximize their individual payoff. Adoption of this strategy does not, on the face of it, appear to be the best strategy because both players can clearly improve their payoff simply by submitting random large claims

A Socio-Psychological Approach to the Iterated Prisoner's Dilemma

Affect Control Theory (ACT), as a model of human interaction, attempts to capture a part of the human psyche that tends to go overlooked in the study of Artificial Intelligence: the role of emotion in decision making. It provides an empirically derived mathematical framework for the otherwise ethereal "feeling" that guide our every action, even in ways that may appear irrational. In this work, we apply BayesACT, a variant on classical ACT, to the much-studied Iterated Prisoner's Dilemma, showing that it appears to human players to approach the game more like a human than other computerized agents. Additionally, we expand into the networked version of this game, showing that the observed human behaviours of decision hysteresis, network structure invariance, and anti-correlation of cooperation and reward, are all emergent properties of the networked BayesACT agents

Experimental and theoretical investigations of the emergence and sustenance of prosocial behavior in groups

Das Ziel der vorliegenden Arbeit war es, Bedingungen, unter denen prosoziales Verhalten entsteht, zu untersuchen. Hierzu wurden Vorhersagen und Annahmen der evolutionären Spieltheorie auf menschliches Kooperationsverhalten angewendet. Kooperatives Verhalten wird als evolutionäres Rätsel betrachtet, da natürliche Selektion betrügerisches Verhalten im Laufe der Zeit eigentlich begünstigen sollte. Nichtsdestotrotz ist Kooperation überall in der Natur vorzufinden. Per Definition ist kooperatives Verhalten kostenverursachend für den Handelnden und bietet gleichzeitig Vorteile oder Gewinne für andere Personen. Betrug oder Defektion dagegen verursachen keine Kosten, aber die bereitgestellten Vorteile anderer können dennoch genutzt werden. Infolgedessen ist Kooperation ohne die Unterstützung von Mechanismen, die die Gefahr der Ausbeutung reduzieren, keine evolutionär stabile Strategie. In der vorliegenden Arbeit wurden nun folgende Aspekte untersucht: (i) reziprokes Verhalten in multiplen paarweisen Interaktionen und (ii) die Auswirkung von sozialen Strukturen auf dyadische Beziehungen im Gefangenendilemma; und (iii) die Verwendung von Bestrafung unter Berücksichtigung der Möglichkeit von Konflikteskalation im öffentlichen‐Güter‐Spiel. In Kapitel 1 wurde untersucht, ob und in welcher Weise sich unterschiedliche Anzahlen von Interaktionspartnern auf kooperatives Verhalten im wiederholten Gefangenendilemma (kurz IPD) auswirken. Gemäß den Annahmen der direkten Reziprozität zeigte sich, dass die Versuchspersonen im traditionellen IPD mit unbekanntem Endpunkt mehrheitlich kooperierten. Insgesamt entsprach das Verhalten reaktiven Strategien ähnlich zu großzügigem Tit‐For‐Tat . Wenn die Versuchspersonen mit mehreren Partnern in drei IPDs gleichzeitig interagierten, sankt die durchschnittliche Kooperativität allerdings signifikant ab. Weiterführende Analysen zeigten, dass diese Versuchspersonen nur eine kooperative Beziehung ähnlich der Beziehung aus dem ein‐Partner IPD etablieren konnten, dass aber keine Kooperation in einer zweiten Beziehung aufgebaut werden konnten (das Kooperationslevel der dritten Beziehung lag zwischen diesen beiden). Diese Resultate widersprechen der traditionellen Annahme der evolutionären Spieltheorie, die eine Unabhängigkeit von Spielen annimmt, da eine erhöhte Versuchung in einigen Beziehungen zu bestehen scheint, wenn man mit drei anstelle von nur einem Sozialpartner interagiert. All dies deutet daraufhin, dass Modelle explizit den Effekt von unterschiedlichen Anzahlen von Partnern mitaufnehmen sollten, um so dem differenzierenden Verhalten eines Individuums gerecht zu werden. Ein Anfang stellt hier die Erforschung von Kooperation in heterogenen Netzwerken dar. Die Auswirkung von sozialen Strukturen auf Kooperation wurde in Kapitel 2 betrachtet. Beziehungen können durch eine zugrundeliegende Netzwerkstruktur charakterisiert werden. Bisher wurde diese Gegebenheit in theoretischen Überlegungen zumeist ignoriert und erst kürzlich fanden Netzwerkstrukturen Berücksichtigung in Modellen. Empirische Erkenntnisse zu diesen Modellen gab es bisher kaum, so dass sich dieses Kapitel genau dieser Lücke widmete. Hier interagierten die Versuchspersonen in mehreren, unabhängigen IPDs entweder innerhalb eines statischen oder eines dynamischen Netzwerkes. In Letzterem hatten die Versuchspersonen die Möglichkeit ihre sozialen Verbindungen nach jeder Gefangenendilemma‐Runde zu verändern. In Übereinstimmung mit theoretischen Modellen war die Kooperation in den dynamischen Netzwerken höher als in den statischen. Darüber hinaus veränderten die Versuchspersonen der dynamischen Netzwerke ihr soziales Umfeld durch ein bevorzugtes Beenden von Beziehungen zu Defektoren. Hierdurch fand eine Sortierung innerhalb des Netzwerkes statt und es bildeten sich kooperative Cliquen. Diese Selbstorganisation ist bemerkenswert, weil sie zusätzlich zum Effekt der direkten Reziprozität auftrat und weil die Versuchspersonen die Cliquenbildung auf Netzwerkebene nicht wahrnehmen konnten. Zusammenfassend zeigen diese Resultate die hohe Bedeutung von dynamischen sozialen Netzwerken auf und belegen, dass Strukturen höherer Ordnung neben dem Verhalten auf Individuumsebene entstehen können, welche dann wiederum in Wechselwirkung zum Selektionsdruck stehen können. In Kapitel 3 wurde der Einfluss von kostenverursachender Bestrafung, die potentiell zwischen Versuchspersonen eskalieren kann, auf kooperatives Verhalten untersucht. Vierergruppen spielten das öffentliche‐Güter‐Spiel mit fünf aufeinanderfolgenden Bestrafungsrunden. In der Regel sind Bestrafer aufgrund des Experimentalaufbaus vor Vergeltung geschützt, hier war dies jedoch nicht der Fall. Tatsächlich entwickelten sich Sequenzen von kostenverursachender Bestrafung zwischen Versuchspersonen, sogenannte Vendetten. Sie traten besonders häufig auf, wenn die Bestrafung als ungerecht oder als beliebig eingestuft wurde. Diese Resultate stehen im Widerspruch zu theoretischen Modellen, in denen Vendetten nicht evolvieren, da sie zu kostenintensiv sind und Defektion die bessere Verhaltensalternative darstellt. Nichtsdestotrotz stieg die Kooperation im Laufe der Zeit an. Dies ist vermutlich darauf zurückzuführen, dass die Bestrafung der ersten Runde noch auf defektierende Gruppenmitglieder gerichtet war, welches ihre Motivation in das öffentliche Gut einzuzahlen letztlich erhöhte. Ferner schienen einige Versuchspersonen sogar den möglichen Ausbruch von kostenintensiven Vendetten zu antizipierten und verzögerten ihre Bestrafung bis zum letztmöglichen Zeitpunkt. Diese Resultate zeigen auf, dass Modelle einen wichtigen Aspekt bisher vermissen lassen, da sowohl Tiere als auch Menschen häufig Vergeltung üben und sich gerade in menschlichen Gesellschaften Vendetten finden lassen. Spekulativ ist anzunehmen, dass Equity und Reputation gerade solche Aspekte dar stellen. Zusammenfassend konnten mit dieser Arbeit Bedingungen identifizieren werden, unter denen Kooperation zwischen nicht‐verwandten Personen entsteht und unter denen Kooperation niedrig ausfällt. Auf der einen Seite konnten neuere Modelle zu dynamischen sozialen Netzwerken empirisch untermauert werden ‐ auf der anderen Seite wurden Schwachpunkte in anderen Modellen ausgewiesen. Zusätzlich trug diese Arbeit weitere Erkenntnisse zum Verständnis der kostenverursachenden Bestrafung und der direkten Reziprozität beim Menschen bei