selected for Journal Publication Fast Track in CP'15International audienc

Koriche, Frédéric

Lagrue, Sylvain

Piette, Eric

Tabary, Sébastien

English

HAL Descartes

General Game Playing with Stochastic CSP

The challenge of General Game Playing (GGP) is to devise artificial game playing algorithms that take as input the rules of any strategic game, described in the Game Description Language (GDL), and that effectively play without human intervention. The aim of this paper is to address the GGP challenge by casting GDL games (potentially with chance events) into the Stochastic Constraint Satisfaction Problem (SCSP). The stochastic constraint network of a game is decomposed into a sequence of μSCSPs (a.k.a. one-stage SCSP), each of them being associated to a game round. Winning strategies are searched by coupling the MAC algorithm, used to solve each μSCSP in turn, together with the UCB (Upper Confidence Bound) policy for approximating the values of those strategies obtained by the last μSCSP in the sequence. Extensive experiments conducted on various GDL games with different deliberation times per round, demonstrate that the MAC-UCB algorithm significantly outperforms the state-of-the-art UCT (upper confidence bounds for trees) algorithm

Maastricht University Research Portal

This paper has been published in Constraints 21, the Journal Fast Track issue of CP’1

   General Game Playing with Stochastic CSP - one pageabstractCitation for published version (APA):Piette, E., Koriche, F., Lagrue, S., & Tabary, S. (2015). General Game Playing with Stochastic CSP - onepage abstract. In The International Conference on Principles and Practice of Constraint Programming:(CP'15)Document status and date:Published: 01/09/2015Please check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There canbe important differences between the submitted version and the official published version of record.People interested in the research are advised to contact the author for the final version of the publication,or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyrightowners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with theserights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.• You may not further distribute the material or use it for any profit-making activity or commercial gain• You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above,please follow below link for the End User Agreement:www.umlib.nl/taverne-licenseTake down policyIf you believe that this document breaches copyright please contact us at:repository@maastrichtuniversity.nlproviding details and we will investigate your claim.Download date: 01 Oct. 2023General Game Playing with Stochastic CSPFrédéric Koriche, Sylvain Lagrue, Éric Piette, and Sébastien TabaryUniversité Lille-Nord de France CRIL - CNRS UMR 8188 Artois, F-62307 Lens{koriche,lagrue,epiette,tabary}@cril.frhttp://www.cril.univ-artois.fr/~{koriche,lagrue,epiette,tabary}Abstract. 1 The aim of General Game Playing (GGP) is to devise game playingalgorithms which are not dedicated to a particular strategic game, but are gen-eral enough to effectively play a wide variety of games. A tournament is heldevery year by AAAI, in which artificial game players are supplied the rules ofarbitrary new games and, without human intervention, have to play these gameoptimally. Games rules are described in a declarative representation language,called GDL for Game Description Language. The lastest version of this languageis expressive enough to describe finite multi-player games with uncertain andincomplete information. GGP algorithms include, among others, answer set pro-gramming methods, automated construction of evaluation functions, and Monte-Carlo methods such as Upper Confidence bounds for Trees (UCT). Beyond itsplay value, GGP offers a rigorous setting for modeling and analyzing sequentialdecision-making algorithms in multi-agent environments.By providing a declarative approach for representing and solving combinatorialproblems, Constraint Programming appears as a promising technology to addressthe GGP challenge. Currently, several constraint-based formalisms have alreadybeen proposed to model and solve games; they include notably Quantified CSP,Strategic CSP and Constraint Games. Most of these formalisms are, however,restricted to deterministic, perfect information games: during each round of thegame, players have full access to the current state and their actions have determin-istic effects. This paper focuses on stochastic games, with chance events, usingthe framework of stochastic constraint networks.More precisely, we study a fragment of the Stochastic Constraint SatisfactionProblem (SCSP), that captures GDL games with uncertain (but complete) infor-mation. Interestingly, the SCSP for this class of games can be decomposed intoa sequence of µSCSPs (a.k.a one-stage stochastic constraint satisfaction prob-lems). Based on this decomposition, we propose a sequential decision-makingalgorithm, MAC-UCB, that combines the MAC algorithm (Maintaining Arc Con-sistency) for solving each µSCSP, and the multi-armed bandits Upper ConfidenceBound (UCB) method for approximating the expected utility of strategies. Weshow that in practice MAC-UCB significantly outperforms (the multi-player ver-sion of) UCT, which is the state-of-the-art GGP algorithm for stochastic games.MAC-UCB also dominates FC-UCB, a variant where MAC is replaced with the For-ward Checking (FC) algorithm. Such conclusions are drawn from comparing theperformance of these algorithms, using extensive experiments (about 1, 350, 000face-offs) over a wide range of GDL games.1 This paper has been published in Constraints 21(1), the Journal Fast Track issue of CP’15.

General Game Playing with Stochastic CSP - one page abstract

The challenge of General Game Playing (GGP) is to devise game playing programs that take as input the rules of any strategic game, described in the Game Description Language (GDL), and that effectively play without human intervention. The aim of this paper is to address the GGP challenge by casting GDL games (potentially with chance events) into the Stochastic Constraint Satisfaction Problem (SCSP). The stochastic constraint network of a game is decomposed into a sequence of μSCSPs (also know as one-stage SCSP), each associated with a game round. Winning strategies are searched by coupling the MAC (Maintaining Arc Consistency) algorithm, used to solve each μSCSP in turn, together with the UCB (Upper Confidence Bound) policy for approximating the values of those strategies obtained by the last μSCSP in the sequence. Extensive experiments conducted on various GDL games with different deliberation times per round, demonstrate that the MAC-UCB algorithm significantly outperforms the state-of-the-art UCT (Upper Confidence bounds for Trees) algorithm

DIAL UCLouvain

General game playing with stochastic CSP

NARCIS 

The aim of General Game Playing (GGP) is to devise game playing algorithms which are not dedicated to a particular strategic game, but are general enough to effectively play a wide variety of games. A tournament is held every year by AAAI, in which artificial game players are supplied the rules of arbitrary new games and, without human intervention, have to play these game optimally. Games rules are described in a declarative representation language, called GDL for Game Description Language. The latest version of this language is expressive enough to describe finite multi-player games with uncertain and incomplete information. GGP algorithms include, among others, answer set programming methods, automated construction of evaluation functions, and Monte-Carlo methods such as Upper Confidence bounds for Trees (UCT). Beyond its play value, GGP offers a rigorous setting for modelling and analysing sequential decision-making algorithms in multi-agent environments. By providing a declarative approach for representing and solving combinatorial problems, Constraint Programming appears as a promising technology to address the GGP challenge. Currently, several constraint-based formalisms have already been proposed to model and solve games; they include notably Quantified CSP, Strategic CSP and Constraint Games. Most of these formalisms are, however, restricted to deterministic, perfect information games: during each round of the game, players have full access to the current state and their actions have deterministic effects. This paper focuses on stochastic games, with chance events, using the framework of stochastic constraint networks. More precisely, we study a fragment of the Stochastic Constraint Satisfaction Problem (SCSP), that captures GDL games with uncertain (but complete) information. Interestingly, the SCSP for this class of games can be decomposed into a sequence of mu-SCSPs (a.k.a one-stage stochastic constraint satisfaction problems). Based on this decomposition, we propose a sequential decision-making algorithm, MAC-UCB, that combines the MAC algorithm (Maintaining Arc Consistency) for solving each mu-SCSP, and the multi-armed bandits Upper Confidence Bound (UCB) method for approximating the expected utility of strategies. We show that in practice MAC-UCB significantly outperforms (the multi-player version of) UCT, which is the state-of-the-art GGP algorithm for stochastic games. MAC-UCB also dominates FC-UCB, a variant where MAC is replaced with the Forward Checking (FC) algorithm. Such conclusions are drawn from comparing the performance of these algorithms, using extensive experiments (about 1,350,000 face-offs) over a wide range of GDL games

HAL-Artois

   General Game Playing with Stochastic CSP - one pageabstractCitation for published version (APA):Piette, E., Koriche, F., Lagrue, S., & Tabary, S. (2015). General Game Playing with Stochastic CSP - onepage abstract. In The International Conference on Principles and Practice of Constraint Programming:(CP'15)Document status and date:Published: 01/09/2015Please check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There canbe important differences between the submitted version and the official published version of record.People interested in the research are advised to contact the author for the final version of the publication,or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyrightowners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with theserights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.• You may not further distribute the material or use it for any profit-making activity or commercial gain• You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above,please follow below link for the End User Agreement:www.umlib.nl/taverne-licenseTake down policyIf you believe that this document breaches copyright please contact us at:repository@maastrichtuniversity.nlproviding details and we will investigate your claim.Download date: 07 Jan. 2021General Game Playing with Stochastic CSPFrédéric Koriche, Sylvain Lagrue, Éric Piette, and Sébastien TabaryUniversité Lille-Nord de France CRIL - CNRS UMR 8188 Artois, F-62307 Lens{koriche,lagrue,epiette,tabary}@cril.frhttp://www.cril.univ-artois.fr/~{koriche,lagrue,epiette,tabary}Abstract. 1 The aim of General Game Playing (GGP) is to devise game playingalgorithms which are not dedicated to a particular strategic game, but are gen-eral enough to effectively play a wide variety of games. A tournament is heldevery year by AAAI, in which artificial game players are supplied the rules ofarbitrary new games and, without human intervention, have to play these gameoptimally. Games rules are described in a declarative representation language,called GDL for Game Description Language. The lastest version of this languageis expressive enough to describe finite multi-player games with uncertain andincomplete information. GGP algorithms include, among others, answer set pro-gramming methods, automated construction of evaluation functions, and Monte-Carlo methods such as Upper Confidence bounds for Trees (UCT). Beyond itsplay value, GGP offers a rigorous setting for modeling and analyzing sequentialdecision-making algorithms in multi-agent environments.By providing a declarative approach for representing and solving combinatorialproblems, Constraint Programming appears as a promising technology to addressthe GGP challenge. Currently, several constraint-based formalisms have alreadybeen proposed to model and solve games; they include notably Quantified CSP,Strategic CSP and Constraint Games. Most of these formalisms are, however,restricted to deterministic, perfect information games: during each round of thegame, players have full access to the current state and their actions have determin-istic effects. This paper focuses on stochastic games, with chance events, usingthe framework of stochastic constraint networks.More precisely, we study a fragment of the Stochastic Constraint SatisfactionProblem (SCSP), that captures GDL games with uncertain (but complete) infor-mation. Interestingly, the SCSP for this class of games can be decomposed intoa sequence of µSCSPs (a.k.a one-stage stochastic constraint satisfaction prob-lems). Based on this decomposition, we propose a sequential decision-makingalgorithm, MAC-UCB, that combines the MAC algorithm (Maintaining Arc Con-sistency) for solving each µSCSP, and the multi-armed bandits Upper ConfidenceBound (UCB) method for approximating the expected utility of strategies. Weshow that in practice MAC-UCB significantly outperforms (the multi-player ver-sion of) UCT, which is the state-of-the-art GGP algorithm for stochastic games.MAC-UCB also dominates FC-UCB, a variant where MAC is replaced with the For-ward Checking (FC) algorithm. Such conclusions are drawn from comparing theperformance of these algorithms, using extensive experiments (about 1, 350, 000face-offs) over a wide range of GDL games.1 This paper has been published in Constraints 21(1), the Journal Fast Track issue of CP’15.

https://cris.maastrichtuniversity.nl/ws/files/37362027/cp15.pdf

General Game Playing with Stochastic CSP

Abstract

Similar works

Full text

Available Versions

HAL Descartes

Maastricht University Research Portal

Maastricht University Research Portal

DIAL UCLouvain

NARCIS

DIAL UCLouvain

Maastricht University Research Portal

HAL-Artois

Maastricht University Research Portal

NARCIS