Theory and applications of difference evaluation functions

ABSTRACT The credit assignment problem (which agents get credit or blame for system performance) is a key research area. For a team of agents collaborating to achieve a goal, the effectiveness of each individual agent must be calculated in order to give adequate feedback to each agent. We use the Difference Evaluation Function to find agent-specific feedback. The Difference Evaluation Function has given excellent empirical results in many domains, including air traffic control and mobile robot control. Though there has been some theoretical work that shows why Difference Evaluation Functions improve system performance, there has been no work to show when and under what conditions such improvements are realized. We apply an evolutionary game-theoretic analysis to show the theoretical advantages of the Difference Evaluation Function. We then focus on how to apply these multiagent learning methods to optimize distributed sensor networks in advanced power generation systems

Open-ended Learning in Symmetric Zero-sum Games

Zero-sum games such as chess and poker are, abstractly, functions that evaluate pairs of agents, for example labeling them `winner' and `loser'. If the game is approximately transitive, then self-play generates sequences of agents of increasing strength. However, nontransitive games, such as rock-paper-scissors, can exhibit strategic cycles, and there is no longer a clear objective -- we want agents to increase in strength, but against whom is unclear. In this paper, we introduce a geometric framework for formulating agent objectives in zero-sum games, in order to construct adaptive sequences of objectives that yield open-ended learning. The framework allows us to reason about population performance in nontransitive games, and enables the development of a new algorithm (rectified Nash response, PSRO_rN) that uses game-theoretic niching to construct diverse populations of effective agents, producing a stronger set of agents than existing algorithms. We apply PSRO_rN to two highly nontransitive resource allocation games and find that PSRO_rN consistently outperforms the existing alternatives.Comment: ICML 2019, final versio

Allocating Limited Resources to Protect a Massive Number of Targets using a Game Theoretic Model

Resource allocation is the process of optimizing the rare resources. In the area of security, how to allocate limited resources to protect a massive number of targets is especially challenging. This paper addresses this resource allocation issue by constructing a game theoretic model. A defender and an attacker are players and the interaction is formulated as a trade-off between protecting targets and consuming resources. The action cost which is a necessary role of consuming resource, is considered in the proposed model. Additionally, a bounded rational behavior model (Quantal Response, QR), which simulates a human attacker of the adversarial nature, is introduced to improve the proposed model. To validate the proposed model, we compare the different utility functions and resource allocation strategies. The comparison results suggest that the proposed resource allocation strategy performs better than others in the perspective of utility and resource effectiveness.Comment: 14 pages, 12 figures, 41 reference

Constitutional Democracy and Public Judgements

This paper proposes a new conceptual framework of a liberal social order, which emphasizes the freedom of action in social interaction and the freedom of participation in social rule-making process. Our articulation of public decision-making process can be interpreted as a formal way of capturing the essence of constitutional democracy, which is an impure mixture of constructivist rationalism and evolutionary rationalism, since we are bringing what is spontaneously evolved through individual experiments into the stage of public design and social choice of a new institutional set of rules. It is also construed as an impure mixture of perfect procedural fairness and pure procedural fairness, since the public judgements to be formed through public deliberations should pay due attention to the intrinsic value of procedures in conferring agency freedom to individuals, as well as to the instrumental value of procedures in expanding well-being freedom of individuals.

An Evolutionary Game Theoretic Model of Rhino Horn Devaluation

Rhino populations are at a critical level due to the demand for rhino horn and the subsequent poaching. Wildlife managers attempt to secure rhinos with approaches to devalue the horn, the most common of which is dehorning. Game theory has been used to examine the interaction of poachers and wildlife managers where a manager can either `dehorn' their rhinos or leave the horn attached and poachers may behave `selectively' or `indiscriminately'. The approach described in this paper builds on this previous work and investigates the interactions between the poachers. We build an evolutionary game theoretic model and determine which strategy is preferred by a poacher in various different populations of poachers. The purpose of this work is to discover whether conditions which encourage the poachers to behave selectively exist, that is, they only kill those rhinos with full horns. The analytical results show that full devaluation of all rhinos will likely lead to indiscriminate poaching. In turn it shows that devaluing of rhinos can only be effective when implemented along with a strong disincentive framework. This paper aims to contribute to the necessary research required for informed discussion about the lively debate on legalising rhino horn trade