Evolutionary Tournament-Based Comparison of Learning and Non-Learning Algorithms for Iterated Games

Evolutionary tournaments have been used effectively as a tool for comparing game-playing algorithms. For instance, in the late 1970's, Axelrod organized tournaments to compare algorithms for playing the iterated prisoner's dilemma (PD) game. These tournaments capture the dynamics in a population of agents that periodically adopt relatively successful algorithms in the environment. While these tournaments have provided us with a better understanding of the relative merits of algorithms for iterated PD, our understanding is less clear about algorithms for playing iterated versions of arbitrary single-stage games in an environment of heterogeneous agents. While the Nash equilibrium solution concept has been used to recommend using Nash equilibrium strategies for rational players playing general-sum games, learning algorithms like fictitious play may be preferred for playing against sub-rational players. In this paper, we study the relative performance of learning and non-learning algorithms in an evolutionary tournament where agents periodically adopt relatively successful algorithms in the population. The tournament is played over a testbed composed of all possible structurally distinct 2×2 conflicted games with ordinal payoffs: a baseline, neutral testbed for comparing algorithms. Before analyzing results from the evolutionary tournament, we discuss the testbed, our choice of representative learning and non-learning algorithms and relative rankings of these algorithms in a round-robin competition. The results from the tournament highlight the advantage of learning algorithms over players using static equilibrium strategies for repeated plays of arbitrary single-stage games. The results are likely to be of more benefit compared to work on static analysis of equilibrium strategies for choosing decision procedures for open, adapting agent society consisting of a variety of competitors.Repeated Games, Evolution, Simulation

The Collective Communication of Social Choice Messages

The research problem addressed in this dissertation is to develop a theory of collective communication. Collective communication is defined as social interaction mediated through messages whose production involves a collectivity. The focus of analysis is on social choice messages, messages that prescribe or proscribe the behavior of members of that collectivity. The theory developed here is used to describe the social choice messages necessary to realize common interests in specific economic environments and the collective communication systems necessary to communicate those messages in those environments. The theory of collective communication is developed in four steps. First, a mathematical theory of collective communication is derived from the unification of game theory and information theory. Building upon the work of von Neumann and Morgenstern, Shannon, Ashby and Conant, philosophical foundations are established and nineteen theorems are derived to predict the transmission of information in a basic game and in a metagame whose outcomes describe constraints to be imposed upon strategic behavior in the basic game. Second, this mathematical theory is formally interpreted as a social theory of collective communication. Third, these theorems are applied to a variety of political and social problems, including those of common property resource management, market failure, the provision of public goods, collective action and coordinated action. Finally, the empirical validity of this theory is tested against research on the development of property rights. The set of regulations and statutes governing mining activity in Nevada between 1858 and 1895 is studied using the techniques of content analysis and multiple linear regression analysis. The predicted relationship between the precision of mining law and the value of mine output is found to be strong, with R squares as high as 0.82347. The research instrument is determined to be reliable and the findings to be statistically significant at the 0.01 level. The evidence presented here is limited but sufficient to motivate the continued development of a unified theory of information and games and the use of mathematical modeling to study salient social problems in the collective communication of social choice messages

Foresighted policy gradient reinforcement learning: solving large-scale social dilemmas with rational altruistic punishment

Many important and difficult problems can be modeled as “social dilemmas”, like Hardin's Tragedy of the Commons or the classic iterated Prisoner's Dilemma. It is well known that in these problems, it can be rational for self-interested agents to promote and sustain cooperation by altruistically dispensing costly punishment to other agents, thus maximizing their own long-term reward. However, self-interested agents using most current multi-agent reinforcement learning algorithms will not sustain cooperation in social dilemmas: the algorithms do not sufficiently capture the consequences on the agent's reward of the interactions that it has with other agents. Recent more foresighted algorithms specifically account for such expected consequences, and have been shown to work well for the small-scale Prisoner's Dilemma. However, this approach quickly becomes intractable for larger social dilemmas. Here, we advance on this work and develop a “teach/learn” stateless foresighted policy gradient reinforcement learning algorithm that applies to Social Dilemma's with negative, unilateral side-payments, in the from of costly punishment. In this setting, the algorithm allows agents to learn the most rewarding actions to take with respect to both the dilemma (Cooperate/Defect) and the “teaching” of other agent's behavior through the dispensing of punishment. Unlike other algorithms, we show that this approach scales well to large settings like the Tragedy of the Commons. We show for a variety of settings that large groups of self-interested agents using this algorithm will robustly find and sustain cooperation in social dilemmas where adaptive agents can punish the behavior of other similarly adaptive agents

If you build it, they wille come : applying the lessons of collective action theory to the 1991 Persian Gulf War

La théorie de l'action collective a été appliquée aux problèmes de la répartition du fardeau entre les alliés et le financement des biens publics internationaux. Bien que la littérature concernant la théorie économique de l'action collective ait évolué, sa mise en application dans le domaine des relations internationales a stagné. Plusieurs questions de sécurité internationale se situent au niveau régional et mettent en jeu des biens collectifs rivaux et exclusifs. Ces questions de sécurité collective peuvent être reformulées en terme théorique comme "biens d'association". Les biens d'association ont tendance à être fourni efficacement. Une étude de cas portant sur la Guerre du Golfe de 1991 semble démontrer la pertinence de l'action collective et la théorie de club dans le cadre de coalitions militaires internationales. La théorie de l'action collective explique certaines relations causales déterminant le succès de la création de coalitions. Le leadership d'un acteur dominant peut forcer ses alliés à révéler leurs préférences et à payer en fonction de celles-ci. La technologie de l'agrégation de forces militaires pour mener une guerre offensive permet le remboursement si l'agrégation nécessaire n'est pas achevée, changeant ainsi le calcul des coûts-bénéfices. Cette technologie diminue le risque associé au leadership dans l'action collective et augmente ainsi la possibilité de coopération

The Current State of Normative Agent-Based Systems

Recent years have seen an increase in the application of ideas from the social sciences to computational systems. Nowhere has this been more pronounced than in the domain of multiagent systems. Because multiagent systems are composed of multiple individual agents interacting with each other many parallels can be drawn to human and animal societies. One of the main challenges currently faced in multiagent systems research is that of social control. In particular, how can open multiagent systems be configured and organized given their constantly changing structure? One leading solution is to employ the use of social norms. In human societies, social norms are essential to regulation, coordination, and cooperation. The current trend of thinking is that these same principles can be applied to agent societies, of which multiagent systems are one type. In this article, we provide an introduction to and present a holistic viewpoint of the state of normative computing (computational solutions that employ ideas based on social norms.) To accomplish this, we (1) introduce social norms and their application to agent-based systems; (2) identify and describe a normative process abstracted from the existing research; and (3) discuss future directions for research in normative multiagent computing. The intent of this paper is to introduce new researchers to the ideas that underlie normative computing and survey the existing state of the art, as well as provide direction for future research.Norms, Normative Agents, Agents, Agent-Based System, Agent-Based Simulation, Agent-Based Modeling

Community cooperation and social solidarity: a case study of community initiated strategic planning

This research explored the process of creating a shared future and the evolution of cooperative collective endeavours in a regional rural community through a bottom-up planning process that involved professionals, public leadership and residents of a rural region in Israel. Using the MT rural region in Israel as a case study, the research was an interpretive exploration of how this community changed the way it collectively functions to achieve individual and shared aspirations. It examined how the community restructured its patterns of interaction, changing the social dynamics – which people interacted with each other, how they interacted with each other, and who felt committed to whom. The motivation for this inquiry stemmed from my desire as a practitioner to better understand the processes by which communities learn to function cooperatively. What are the elements that contributed to enabling a community to create the conditions for collectively utilizing and sustaining common resources rather than dividing them up for private consumption and exploitative narrow interests? What type of cooperative mechanisms enabled people to accomplish together what they cannot accomplish alone? Specifically, there are three research questions: how the change process was initiated in MT, what was significant in the nature of participation in the planning process, and how the mechanisms for regional community cooperation evolved. It was a case study of the planning and development process that I facilitated in MT from 1994-1999 (prior to my intention to undertake research) and is based mainly upon recent interviews of the participants (in that process), their recollections, and retrospective interpretations of that experience. The case has been explored from the theoretical perspective of viewing society in general, and community life in particular, as processes of constructing shared social realities that produce certain collective behaviours of cooperation or non-cooperation (Berger and Luckmann, 1967). This research was about understanding the process of making social rules that incorporate shared meanings and sanctions (Giddens, 1986) for undertaking joint endeavours (Ostrom, 1990, 1992, Wenger, 1998). Specifically two primary insights have come out of this case analysis: 1. In the MT case there was a mutually reinforcing three-way interplay between the strengthening of commitments to mutual care on the regional level, the instrumental benefits from cooperative/joint endeavours, and the envisioning of a shared future. 2. The community development process was owned by the community (not by outside agencies) and they (the community members) set the rules for community involvement. They structured the social interactions which formed the basis for creating shared understandings as a collective to achieve their common future. These insights shed light on how a community's structuring of its interactions and development interventions influenced its ability to act in a collectively optimal manner. By looking at the interrelation between trust as a function of social esteem (Honneth, 1995) and risk taking linked to instrumental benefits of cooperation (Lewis, 2002; Taylor, 1976; White, 2003) we can better understand what contributes to the way some communities continue to miss opportunities (Ostrom 1992), while others are able to promote their collective development and mutual wellbeing. By examining the process of designing (not only the design itself) community development programmes (Block, 2009) and by observing participation not as technique but as an inherent part of the way a community begins structuring its social interactions with their tacit (Polanyi, 1966) and explicit meanings, we can better understand the role of practitioners. And finally, perhaps the elements of chance and opportunity that bring certain combinations of people together in a given time and space may need to be given more weight in what remains a very unpredictable non-linear field of professional practice

Plural Rationality and Interactive Decision Processes; Proceedings of an IIASA Summer Study on Plural Rationality and Interactive Decision Processes, Sopron, Hungary, August 16-26, 1984

These Proceedings report the scientific results of the Summer Study on Plural Rationality and Interactive Decision Processes organized jointly by IIASA and the Hungarian Committee for Applied Systems Analysis. Sixty-eight researchers from sixteen countries participated, most of them contributing papers or experiments. The Study gathered specialists from many disciplines, from philosophy and cultural anthropology, through decision theory, game theory and economics, to engineering and applied mathematics. Twenty-eight of the papers presented during the Study are included in this volume

Discourse on rationality: rational choice and critical theory

The thesis contrasts two hostile and divergent intellectual paradigms in social sciences: rational choice and critical theory. Both rational choice and critical theory offer contrasting perspectives on the structures of social interaction. However, both critical theory and rational choice theory share overlapping concerns ie., both are preoccupied with determining what rational can mean in the realm of social and political interaction. In the case of rational choice paradigm, instrumental reason forms the cornerstone of the theoretical edifice. Ever since the publication of Jurgen Habermas' The lhemy qf Communicative Action Vol. / (1984) and Vol. II (1986) instrumental reason has come under severe attack. His critique anchors on a theory of communicative reason. What makes Habermas' work distinctive is that he does not regard instrumental reason as the single inevitable concomitant of modernity. Habermas sees in modernity an alternative way of conceptualising social interaction in terms of communication rather than strategy. So in a way, his work is a challenge to the defenders of modernity aiming to build a unified social science Jurgen Habermas advances the notion of communicative reason as the centerpiece of a social theory as opposed to instrumental reason. By providing a systematic grounding of the concept of reason in human language, he hopes to establish normative basis of critical theory. This model of reaching agreement or consent constitutes a process of dialogue in which reasons are exchanged between participants. This process is perceived to be a joint search for consensus. Such a dialogic concept of collective choice would necessarily work not with fixed preferences to be amalgamated (as rational choice theories do) but with preferences that are altered or modified as competing reasons are advanced in the course of discussion. In rational discussion, the only thing supposed to count is the power of better argument. Both rational choice and critical theory conceptualise politics in different ways. Rational choice theories critique democratic mechanisms failing to generate general will. Consequently, the political prescriptions offered are limited government or market. On the contrary, the political implications of Habermas' theory of deliberative democracy is anchored in the notion of liberal public sphere envisaging a cognitivist, rationalist vision in which discourse forms a critical normative basis for evaluating the political and moral principles

Automated Service Negotiation Between Autonomous Computational Agents

PhDMulti-agent systems are a new computational approach for solving real world, dynamic and open system problems. Problems are conceptualized as a collection of decentralised autonomous agents that collaborate to reach the overall solution. Because of the agents autonomy, their limited rationality, and the distributed nature of most real world problems, the key issue in multi-agent system research is how to model interactions between agents. Negotiation models have emerged as suitable candidates to solve this interaction problem due to their decentralised nature, emphasis on mutual selection of an action, and the prevalence of negotiation in real social systems. The central problem addressed in this thesis is the design and engineering of a negotiation model for autonomous agents for sharing tasks and/or resources. To solve this problem a negotiation protocol and a set of deliberation mechanisms are presented which together coordinate the actions of a multiple agent system. In more detail, the negotiation protocol constrains the action selection problem solving of the agents through the use of normative rules of interaction. These rules temporally order, according to the agents' roles, communication utterances by specifying both who can say what, as well as when. Specifically, the presented protocol is a repeated, sequential model where offers are iteratively exchanged. Under this protocol, agents are assumed to be fully committed to their utterances and utterances are private between the two agents. The protocol is distributed, symmetric, supports bi and/or multi-agent negotiation as well as distributive and integrative negotiation. In addition to coordinating the agent interactions through normative rules, a set of mechanisms are presented that coordinate the deliberation process of the agents during the ongoing negotiation. Whereas the protocol normatively describes the orderings of actions, the mechanisms describe the possible set of agent strategies in using the protocol. These strategies are captured by a negotiation architecture that is composed of responsive and deliberative decision mechanisms. Decision making with the former mechanism is based on a linear combination of simple functions called tactics, which manipulate the utility of deals. The latter mechanisms are subdivided into trade-off and issue manipulation mechanisms. The trade-off mechanism generates offers that manipulate the value, rather than the overall utility, of the offer. The issue manipulation mechanism aims to increase the likelihood of an agreement by adding and removing issues into the negotiation set. When taken together, these mechanisms represent a continuum of possible decision making capabilities: ranging from behaviours that exhibit greater awareness of environmental resources and less to solution quality, to behaviours that attempt to acquire a given solution quality independently of the resource consumption. The protocol and mechanisms are empirically evaluated and have been applied to real world task distribution problems in the domains of business process management and telecommunication management. The main contribution and novelty of this research are: i) a domain independent computational model of negotiation that agents can use to support a wide variety of decision making strategies, ii) an empirical evaluation of the negotiation model for a given agent architecture in a number of different negotiation environments, and iii) the application of the developed model to a number of target domains. An increased strategy set is needed because the developed protocol is less restrictive and less constrained than the traditional ones, thus supporting development of strategic interaction models that belong more to open systems. Furthermore, because of the combination of the large number of environmental possibilities and the size of the set of possible strategies, the model has been empirically investigated to evaluate the success of strategies in different environments. These experiments have facilitated the development of general guidelines that can be used by designers interested in developing strategic negotiating agents. The developed model is grounded from the requirement considerations from both the business process management and telecommunication application domains. It has also been successfully applied to five other real world scenarios