Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee & Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex), to allow comparison to Banerjee et al. We find that Q-learning converges to a high win rate in GGP. For the ϵ" role="presentation" style="display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; position: relative;">ϵ-greedy strategy, we propose a first enhancement, the dynamic ϵ" role="presentation" style="display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; overflow-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; position: relative;">ϵ algorithm. In addition, inspired by (Gelly & Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Computer Systems, Imagery and Medi

John A. T. Robinson. Re-dater le Nouveau Testament

Mehat André. John A. T. Robinson. Re-dater le Nouveau Testament. In: Revue de l'histoire des religions, tome 206, n°3, 1989. pp. 310-312

O. Prunet. La morale de Clément d'Alexandrie et le Nouveau Testament

Mehat André. O. Prunet. La morale de Clément d'Alexandrie et le Nouveau Testament. In: Revue de l'histoire des religions, tome 174, n°1, 1968. pp. 91-92

R. Minnerath. De Jérusalem à Rome : Pierre et l'unité de l'Église apostolique

Mehat André. R. Minnerath. De Jérusalem à Rome : Pierre et l'unité de l'Église apostolique. In: Revue de l'histoire des religions, tome 213, n°3, 1996. pp. 350-352

M. Smith. Clement of Alexandria and a Secret Gospel of Mark

Mehat André. M. Smith. Clement of Alexandria and a Secret Gospel of Mark. In: Revue de l'histoire des religions, tome 190, n°2, 1976. pp. 196-197

Annie Jaubert. Origène, Homélies sur Josué

Mehat André. Annie Jaubert. Origène, Homélies sur Josué. In: Revue de l'histoire des religions, tome 160, n°1, 1961. pp. 97-99

C. Grappe. Images de Pierre aux deux premiers siècles

Méhat André. C. Grappe. Images de Pierre aux deux premiers siècles. In: Revue de l'histoire des religions, tome 213, n°1, 1996. pp. 97-98

Thomas Schmeller. Paulus und die « Diatribe ». Eine vergleichende Stilinterpretation

Méhat André. Thomas Schmeller. Paulus und die « Diatribe ». Eine vergleichende Stilinterpretation. In: Revue de l'histoire des religions, tome 206, n°1, 1989. pp. 87-88

52. Chadwick (Henry). Early Christian Thought and the Classical Tradition, Studies in Justin, Clement, and Origen.

Mehat André. 52. Chadwick (Henry). Early Christian Thought and the Classical Tradition, Studies in Justin, Clement, and Origen.. In: Revue des Études Grecques, tome 83, fascicule 394-395, Janvier-juin 1970. pp. 268-269