Assessing the Potential of Classical Q-learning in General Game Playing

After the recent groundbreaking results of AlphaGo and AlphaZero, we have seen strong interests in deep reinforcement learning and artificial general intelligence (AGI) in game playing. However, deep learning is resource-intensive and the theory is not yet well developed. For small games, simple classical table-based Q-learning might still be the algorithm of choice. General Game Playing (GGP) provides a good testbed for reinforcement learning to research AGI. Q-learning is one of the canonical reinforcement learning methods, and has been used by (Banerjee $\&$ Stone, IJCAI 2007) in GGP. In this paper we implement Q-learning in GGP for three small-board games (Tic-Tac-Toe, Connect Four, Hex)\footnote{source code: https://github.com/wh1992v/ggp-rl}, to allow comparison to Banerjee et al.. We find that Q-learning converges to a high win rate in GGP. For the $\epsilon$-greedy strategy, we propose a first enhancement, the dynamic $\epsilon$ algorithm. In addition, inspired by (Gelly $\&$ Silver, ICML 2007) we combine online search (Monte Carlo Search) to enhance offline learning, and propose QM-learning for GGP. Both enhancements improve the performance of classical Q-learning. In this work, GGP allows us to show, if augmented by appropriate enhancements, that classical table-based Q-learning can perform well in small games.Comment: arXiv admin note: substantial text overlap with arXiv:1802.0594

On The Foundations of Digital Games

Computers have lead to a revolution in the games we play, and, following this, an interest for computer-based games has been sparked in research communities. However, this easily leads to the perception of a one-way direction of influence between that the field of game research and computer science. This historical investigation points towards a deep and intertwined relationship between research on games and the development of computers, giving a richer picture of both fields. While doing so, an overview of early game research is presented and an argument made that the distinction between digital games and non-digital games may be counter-productive to game research as a whole

Formal verification of AI software

The application of formal verification techniques to Artificial Intelligence (AI) software, particularly expert systems, is investigated. Constraint satisfaction and model inversion are identified as two formal specification paradigms for different classes of expert systems. A formal definition of consistency is developed, and the notion of approximate semantics is introduced. Examples are given of how these ideas can be applied in both declarative and imperative forms

IST Austria Technical Report

Board games, like Tic-Tac-Toe and CONNECT-4, play an important role not only in development of mathematical and logical skills, but also in emotional and social development. In this paper, we address the problem of generating targeted starting positions for such games. This can facilitate new approaches for bringing novice players to mastery, and also leads to discovery of interesting game variants. Our approach generates starting states of varying hardness levels for player 1 in a two-player board game, given rules of the board game, the desired number of steps required for player 1 to win, and the expertise levels of the two players. Our approach leverages symbolic methods and iterative simulation to efficiently search the extremely large state space. We present experimental results that include discovery of states of varying hardness levels for several simple grid-based board games. Also, the presence of such states for standard game variants like Tic-Tac-Toe on board size 4x4 opens up new games to be played that have not been played for ages since the default start state is heavily biased

Automatic Generation of Alternative Starting Positions for Simple Traditional Board Games

Simple board games, like Tic-Tac-Toe and CONNECT-4, play an important role not only in the development of mathematical and logical skills, but also in the emotional and social development. In this paper, we address the problem of generating targeted starting positions for such games. This can facilitate new approaches for bringing novice players to mastery, and also leads to discovery of interesting game variants. We present an approach that generates starting states of varying hardness levels for player~$1$ in a two-player board game, given rules of the board game, the desired number of steps required for player~$1$ to win, and the expertise levels of the two players. Our approach leverages symbolic methods and iterative simulation to efficiently search the extremely large state space. We present experimental results that include discovery of states of varying hardness levels for several simple grid-based board games. The presence of such states for standard game variants like $4 \times 4$ Tic-Tac-Toe opens up new games to be played that have never been played as the default start state is heavily biased.Comment: A conference version of the paper will appear in AAAI 201

WOPR in search of better strategies for board games

Board games have been challenging intellects and capturing imaginations for more than five thousand years. Researchers have been producing artificially intelligent players for more than fifty. Common artificial intelligence techniques applied to board games use alpha-beta pruning tree search techniques. This paper supplies a frame- work that accommodates many of the diverse aspects of board games, as well as exploring several alternatives for searching out ever better strategies. Techniques examined include optimizing artificial neural networks using genetic algorithms and backpropagation

Board Game Focused on Educational Support for Gaming Algorithms

Tato práce se zabývá oblastí umělé inteligence zvané jako ''Metody pro hraní her''. Cílem této bakalářské práce je navrhnout a implementovat software, který umožní uživateli snadněji pochopit principy herních algoritmů Minimax a Alfa-beta prořezávání. Typickými uživateli tohoto softwaru mohou být například studenti oboru umělá inteligence. Práci lze rozdělit do dvou hlavních částí. První, teoretická část, se snaží vysvělit koncept ''Metoda pro hraní her'', dále obsahuje popis návrhu softwaru a popis výukových přínosů aplikace. Druhá část práce je věnována popisu implementace softwaru, testování a diskuzi dosažených výsledků.This work deals with the part of field of artificial intelligence known as ''Methods of playing games''. The goal of this bachelor's thesis is to design and implement software that allows the user to more easily understand the principles of game algorithms Minimax and Alpha-beta pruning. Typical users of this software can be, for example, students of artificial intelligence. This work is divided into two main parts. The first theoretical part tries to explain the ''Method of playing games'' concept and subsequently contains detailed descriptions of software design and educational benefits. The second part of this work is devoted to a description of software implementation, testing and discussion of the achieved results.

Decision-making and control with metasurface-based diffractive neural networks

The ultimate goal of artificial intelligence is to mimic the human brain to perform decision-making and control directly from high-dimensional sensory input. All-optical diffractive neural networks provide a promising solution for implementing artificial intelligence with high-speed and low-power consumption. To date, most of the reported diffractive neural networks focus on single or multiple tasks that do not involve interaction with the environment, such as object recognition and image classification. In contrast, the networks that can perform decision-making and control, to our knowledge, have not been developed yet. Here, we propose using deep reinforcement learning to implement diffractive neural networks that imitate human-level decision-making and control capability. Such networks allow for finding optimal control policies through interaction with the environment and can be readily realized with the dielectric metasurfaces. The superior performances of these networks are verified by engaging three types of classic games, Tic-Tac-Toe, Super Mario Bros., and Car Racing, and achieving the same or even higher levels comparable to human players. Our work represents a solid step of advancement in diffractive neural networks, which promises a fundamental shift from the target-driven control of a pre-designed state for simple recognition or classification tasks to the high-level sensory capability of artificial intelligence. It may find exciting applications in autonomous driving, intelligent robots, and intelligent manufacturing

Autonomous Learning Multi-Model Classifier of 0-Order (ALMMo-0)

In this paper, a new type of 0-order multi-model classifier, called Autonomous Learning Multiple-Model (ALMMo-0), is proposed. The proposed classifier is non-iterative, feedforward and entirely data-driven. It automatically extracts the data clouds from the data per class and forms 0-order AnYa type fuzzy rule-based (FRB) sub-classifier for each class. The classification of new data is done using the “winner takes all” strategy according to the scores of confidence generated objectively based on the mutual distribution and ensemble properties of the data by the sub-classifiers. Numerical examples based on benchmark datasets demonstrate the high performance and computation-efficiency of the proposed classifier