Variance Reduction in Monte Carlo Counterfactual Regret Minimization (VR-MCCFR) for Extensive Form Games using Baselines

Learning strategies for imperfect information games from samples of interaction is a challenging problem. A common method for this setting, Monte Carlo Counterfactual Regret Minimization (MCCFR), can have slow long-term convergence rates due to high variance. In this paper, we introduce a variance reduction technique (VR-MCCFR) that applies to any sampling variant of MCCFR. Using this technique, per-iteration estimated values and updates are reformulated as a function of sampled values and state-action baselines, similar to their use in policy gradient reinforcement learning. The new formulation allows estimates to be bootstrapped from other estimates within the same episode, propagating the benefits of baselines along the sampled trajectory; the estimates remain unbiased even when bootstrapping from other estimates. Finally, we show that given a perfect baseline, the variance of the value estimates can be reduced to zero. Experimental evaluation shows that VR-MCCFR brings an order of magnitude speedup, while the empirical variance decreases by three orders of magnitude. The decreased variance allows for the first time CFR+ to be used with sampling, increasing the speedup to two orders of magnitude

The Effectiveness of Congklak Traditional Games on Students with Emotional and Behavioral Disorder

The purpose of this study was to determine the effect of traditional Congklak games on students with emotional and behavioral disorder social interactions in SLB E Bhina Putera. The study used a Single Subject Research (SSR) design. The person as a research subject is AS class III students. Based on the observation, AS shows some examples of its communication with the classmates that is sunken, rarely talk with friends, ignore the teacher's orders, always seen alone on rest hour. The results shows an increase in social interaction through the traditional game of Congklak. With all that set, it can be concluded that the use of Congklak traditional games can have a positive effect on the social interaction of students with emotional and behavioral disorder in SLB E Bhina Putera Surakarta

The game semantics of game theory

We use a reformulation of compositional game theory to reunite game theory with game semantics, by viewing an open game as the System and its choice of contexts as the Environment. Specifically, the system is jointly controlled by $n \geq 0$ noncooperative players, each independently optimising a real-valued payoff. The goal of the system is to play a Nash equilibrium, and the goal of the environment is to prevent it. The key to this is the realisation that lenses (from functional programming) form a dialectica category, which have an existing game-semantic interpretation. In the second half of this paper, we apply these ideas to build a compact closed category of `computable open games' by replacing the underlying dialectica category with a wave-style geometry of interaction category, specifically the Int-construction applied to the cartesian monoidal category of directed-complete partial orders

When Europe encounters urban governance: Policy Types, Actor Games and Mechanisms of cites Europeanization

This paper examines European Union (EU) causal mechanisms and policy instruments affecting the urban domain throughout the lenses of the Europeanization approach. Instead of looking at EU instruments that are formally/legally consecrated to cities, we use theoretical public policy analysis to explore the arenas and the causal mechanisms that structure the encounters between the EU and urban systems of governance. Policy instruments are related to policy arenas and in turn to different mechanisms of transmission thus originating a typology of European Policy Modes. The paper focuses on four different EU instruments in the in the macro-area of sustainable development and proposes potential game-theoretical models for each of them. In the conclusions we highlight the differences between this approach and the traditional analysis of EU urban policy, and suggest avenues for future empirical research based on typologies of policy instruments and modes of Europeanization

Games on graphs: A minor modification of payoff scheme makes a big difference

Various social dilemma games that follow different strategy updating rules have been studied on many networks.The reported results span the entire spectrum, from significantly boosting,to marginally affecting,to seriously decreasing the level of cooperation.Experimental results that are qualitatively different from theoretical prediction have also been reported.It is widely believed that the results are largely determined by three elements,including payoff matrices of the underlying 2*2 games,the way that the strategic states of the players are updated and the structure of the networks.Here we discuss the impact of a seemly non-essential mechanism -- what we refer to as a "payoff scheme". Specifically, in each round after the states of all of the players are determined,the payoff scheme is how each player's payoff is calculated.In addition to the two conventions in which either the accumulated or the averaged payoff is calculated from playing with all of the neighboring players,we here study the effects of calculating the payoff from pairing up with one random player from among the neighboring players. Based on probability theory, in a situation of uncorrelated events, the average payoff that involves all of the neighbors should,in principal,be equivalent to the payoff from pairing up with one neighbor.However,our simulation of games on graphs shows that, in many cases,the two payoff schemes lead to qualitatively different levels of cooperation.This finding appears to provide a possible explanation for a wide spectrum of observed behaviors in the literature.We have also observed that results from the randomly-pairing-one mechanism are more robust than the involving-all-neighbours mechanism because,in the former case, neither the other three main elements nor the initial states of the players have a large impact on the final level of cooperation compared with in the latter case.Comment: 23 pages,171 figure

Evolution of a supply chain management game for the trading agent competition

TAC SCM is a supply chain management game for the Trading Agent Competition (TAC). The purpose of TAC is to spur high quality research into realistic trading agent problems. We discuss TAC and TAC SCM: game and competition design, scientific impact, and lessons learnt

Classroom games in economics : a quantitative assessment of the 'beer game'

Using an experiment, I compare the use of the 'Beer Distribution' classroom game with the more traditional 'chalk and talk' approach to teach students about inventories and the macroeconomy. My empirical results confirm and extend our understanding of the relative strengths and weaknesses of the use of classroom games: the game tends to improve interest and motivation on average, though some students dislike their use; the game is effective at driving home its key messages, but it may wrongly lead students to disregard other important factors; the game is inferior where facts mastery or de nitional learning is required. Rather than an endorsement or a criticism of classroom games, the conclusion is cautionary advice on how to best make use of games within an overall course

A demand-driven approach for a multi-agent system in Supply Chain Management

This paper presents the architecture of a multi-agent decision support system for Supply Chain Management (SCM) which has been designed to compete in the TAC SCM game. The behaviour of the system is demand-driven and the agents plan, predict, and react dynamically to changes in the market. The main strength of the system lies in the ability of the Demand agent to predict customer winning bid prices - the highest prices the agent can offer customers and still obtain their orders. This paper investigates the effect of the ability to predict customer order prices on the overall performance of the system. Four strategies are proposed and compared for predicting such prices. The experimental results reveal which strategies are better and show that there is a correlation between the accuracy of the models' predictions and the overall system performance: the more accurate the prediction of customer order prices, the higher the profit. © 2010 Springer-Verlag Berlin Heidelberg

Experimental study and modelling of Networked Virtual Environment server traffic

The paradigm of virtual world environment arises as an useful tool in diverse fields such as e-Health or education, where they provide a new way of communication and interaction with end users. Networking capabilities play an important role in these systems, which motivates the study and understanding of the gaming network traffic. The present work focuses on Open Wonderland, a system that provides the basis for the development of Networked Virtual Environments with educational or health purposes. The goal of this paper is defining a testing environment and modelling the behaviour of the outgoing network traffic at the server side.Ministerio de Industria, Comercio y Turismo AAL-010000-2012-10Ministerio de Ciencia e Innovación TEC2009-10639-C04-0