A generic approach for generating interesting interactive Pac-Man opponents

This paper follows on from our previous work focused on formulating an efficient generic measure of user's satisfaction (`interest') when playing predator /prey games. Viewing the game from the predators' (i.e. opponents') perspective, a robust on-line neuroevolution learning mechanism has been presented capable of increasing --- independently of the initial behavior and playing strategy --- the well known Pac-Man game's interest as well as keeping that interest at high levels while the game is being played. This mechanism has also demonstrated high adaptability to changing PacMan playing strategies in a relatively simple playing stage. In the work presented here, we attempt to test the on-line learning mechanism over more complex stages and to explore the relation between the interest measure and the topology of the stage. Results show that the interest measure proposed is independent of the stage's complexity and topology, which demonstrates the approach 's generality for this game.peer-reviewe

Capturing player enjoyment in computer games

The current state-of-the-art in intelligent game design using Artificial Intelligence (AI) techniques is mainly focused on generating human-like and intelligent characters. Even though complex opponent behaviors emerge through various machine learning techniques, there is generally no further analysis of whether these behaviors contribute to the satisfaction of the player. The implicit hypothesis motivating this research is that intelligent opponent behaviors enable the player to gain more satisfaction from the game. This hypothesis may well be true; however, since no notion of entertainment or enjoyment is explicitly defined, there is therefore no evidence that a specific opponent behavior generates enjoyable games.peer-reviewe

Ms Pac-Man versus Ghost Team CEC 2011 competition

Games provide an ideal test bed for computational intelligence and significant progress has been made in recent years, most notably in games such as Go, where the level of play is now competitive with expert human play on smaller boards. Recently, a significantly more complex class of games has received increasing attention: real-time video games. These games pose many new challenges, including strict time constraints, simultaneous moves and open-endedness. Unlike in traditional board games, computational play is generally unable to compete with human players. One driving force in improving the overall performance of artificial intelligence players are game competitions where practitioners may evaluate and compare their methods against those submitted by others and possibly human players as well. In this paper we introduce a new competition based on the popular arcade video game Ms Pac-Man: Ms Pac-Man versus Ghost Team. The competition, to be held at the Congress on Evolutionary Computation 2011 for the first time, allows participants to develop controllers for either the Ms Pac-Man agent or for the Ghost Team and unlike previous Ms Pac-Man competitions that relied on screen capture, the players now interface directly with the game engine. In this paper we introduce the competition, including a review of previous work as well as a discussion of several aspects regarding the setting up of the game competition itself. © 2011 IEEE

A scheme for creating digital entertainment with substance

Computer games constitute a major branch of the entertainment industry nowadays. The financial and research potentials of making games more appealing (or else more interesting) are more than impressive. Interactive and cooperative characters can generate more realism in games and satisfaction for the player. Moreover, on-line (while play) machine learning techniques are able to produce characters with intelligent capabilities useful to any game’s context. On that basis, richer human-machine interaction through real-time entertainment, player and emotional modeling may provide means for effective adjustment of the non-player characters’ behavior in order to obtain games of substantial entertainment. This paper introduces a research scheme for creating NPCs that generate entertaining games which is based interdisciplinary on the aforementioned areas of research and is foundationally supported by several pilot studies on testbed games. Previous work and recent results are presented within this framework.peer-reviewe

EvoTanks: co-evolutionary development of game-playing agents

This paper describes the EvoTanks research project, a continuing attempt to develop strong AI players for a primitive 'Combat' style video game using evolutionary computational methods with artificial neural networks. A small but challenging feat due to the necessity for agent's actions to rely heavily on opponent behaviour. Previous investigation has shown the agents are capable of developing high performance behaviours by evolving against scripted opponents; however these are local to the trained opponent. The focus of this paper shows results from the use of co-evolution on the same population. Results show agents no longer succumb to trappings of local maxima within the search space and are capable of converging on high fitness behaviours local to their population without the use of scripted opponents

Interactive opponents generate interesting games

In this paper we present experiments on neuroevolution mechanisms applied to predator/prey multi-character computer games. Our test-bed is a computer game where the prey (i.e. player) has to avoid its predators by escaping through an exit without getting killed. By viewing the game from the predators’ (i.e. opponents’) perspective, we attempt off-line to evolve neural-controlled opponents, whose communication is based on partial implicit information, capable of playing effectively against computer-guided fixed strategy players. However, emergent near-optimal behaviors make the game less interesting to play. We therefore discuss the criteria that make a game interesting and, furthermore, we introduce a generic measure of this category of (i.e. predator/prey) computer games’ interest (i.e. player’s satisfaction from the game). Given this measure, we present an evolutionary mechanism for opponents that keep learning from a player while playing against it (i.e. on-line) and we demonstrate its efficiency and robustness in increasing and maintaining the game’s interest. Computer game opponents following this on-line learning approach show high adaptability to changing player strategies which provides evidence for the approach’s effectiveness against human players.peer-reviewe

Towards optimizing entertainment in computer games

Mainly motivated by the current lack of a qualitative and quantitative entertainment formulation of computer games and the procedures to generate it, this article covers the following issues: It presents the features—extracted primarily from the opponent behavior—that make a predator/prey game appealing; provides the qualitative and quantitative means for measuring player entertainment in real time, and introduces a successful methodology for obtaining games of high satisfaction. This methodology is based on online (during play) learning opponents who demonstrate cooperative action. By testing the game against humans, we confirm our hypothesis that the proposed entertainment measure is consistent with the judgment of human players. As far as learning in real time against human players is concerned, results suggest that longer games are required for humans to notice some sort of change in their entertainment.peer-reviewe

Neuroevolution in Games: State of the Art and Open Challenges

This paper surveys research on applying neuroevolution (NE) to games. In neuroevolution, artificial neural networks are trained through evolutionary algorithms, taking inspiration from the way biological brains evolved. We analyse the application of NE in games along five different axes, which are the role NE is chosen to play in a game, the different types of neural networks used, the way these networks are evolved, how the fitness is determined and what type of input the network receives. The article also highlights important open research challenges in the field.Comment: - Added more references - Corrected typos - Added an overview table (Table 1

AI in Computer Games: Generating Interesting Interactive Opponents by the use of Evolutionary Computation

Institute of Perception, Action and BehaviourWhich features of a computer game contribute to the player’s enjoyment of it? How can we automatically generate interesting and satisfying playing experiences for a given game? These are the two key questions addressed in this dissertation. Player satisfaction in computer games depends on a variety of factors; here the focus is on the contribution of the behaviour and strategy of game opponents in predator/prey games. A quantitative metric of the ‘interestingness’ of opponent behaviours is defined based on qualitative considerations of what is enjoyable in such games, and a mathematical formulation grounded in observable data is derived. Using this metric, neural-network opponent controllers are evolved for dynamic game environments where limited inter-agent communication is used to drive spatial coordination of opponent teams. Given the complexity of the predator task, cooperative team behaviours are investigated. Initial candidates are generated using off-line learning procedures operating on minimal neural controllers with the aim of maximising opponent performance. These example controllers are then adapted using on-line (i.e. during play) learning techniques to yield opponents that provide games of high interest. The on-line learning methodology is evaluated using two dissimilar predator/prey games with a number of different computer player strategies. It exhibits generality across the two game test-beds and robustness to changes of player, initial opponent controller selected, and complexity of the game field. The interest metric is also evaluated by comparison with human judgement of game satisfaction in an experimental survey. A statistically significant number of players were asked to rank game experiences with a test-bed game using perceived interestingness and their ranking was compared with that of the proposed interest metric. The results show that the interest metric is consistent with human judgement of game satisfaction. Finally, the generality, limitations and potential of the proposed methodology and techniques are discussed, and other factors affecting the player’s satisfaction, such as the player’s own strategy, are briefly considered. Future directions building on the work described herein are presented and discussed