Planning For Non-Player Characters By Learning From Demonstration

In video games, state of the art non-player character (NPC) behavior generation typically depends on hard-coding NPC actions. In many game situations however, it is hard to foresee how an NPC should behave to appear intelligent or to accommodate human preferences for NPC behavior. We advocate the creation of a more flexible method to allow players (and developers) to train NPCs to execute novel behaviors which are not hard-coded. In particular, we investigate search-based planning approaches using demonstration to guide the search through high-dimensional spaces that represent the full state of the game. To this end, we developed the Training Graph heuristic, an extension of the Experience Graph heuristic, that guides a search smoothly and effectively even when a demonstration is unreachable in the search space, and ensures that more of the demonstrations are utilized to better train the NPC\u27s behavior. To deal with variance in the initial conditions of such planning problems, we have developed heuristics in the Multi-Heuristic A* framework to adapt demonstration trace data to new problems. We evaluate our approach in the Creation Engine game engine by modifying The Elder Scrolls V: Skyrim (Skyrim) to accommodate our NPC behavior generators and experiments. In Skyrim, players are given quests which are composed of several objectives. NPCs in the game sometimes accompany the player on quests, but state-of-the-art companion NPC AI is not sophisticated enough to behave according to arbitrary player desires. We hope that our work will lead to the creation of trainable NPC AI. This will enable novel gameplay mechanics for video game players and may augment video game production by allowing developers to train NPCs instead of hard-coding complex behaviors

Non-determinism in the narrative structure of video games

PhD ThesisAt the present time, computer games represent a finite interactive system. Even in their more experimental forms, the number of possible interactions between player and NPCs (non-player characters) and among NPCs and the game world has a finite number and is led by a deterministic system in which events can therefore be predicted. This implies that the story itself, seen as the series of events that will unfold during gameplay, is a closed system that can be predicted a priori. This study looks beyond this limitation, and identifies the elements needed for the emergence of a non-finite, emergent narrative structure. Two major contributions are offered through this research. The first contribution comes in the form of a clear categorization of the narrative structures embracing all video game production since the inception of the medium. In order to look for ways to generate a non-deterministic narrative in games, it is necessary to first gain a clear understanding of the current narrative structures implemented and how their impact on users’ experiencing of the story. While many studies have observed the storytelling aspect, no attempt has been made to systematically distinguish among the different ways designers decide how stories are told in games. The second contribution is guided by the following research question: Is it possible to incorporate non-determinism into the narrative structure of computer games? The hypothesis offered is that non-determinism can be incorporated by means of nonlinear dynamical systems in general and Cellular Automata in particular

Enhancing automatic level generation for platform videogames

This dissertation addresses the challenge of improving automatic level generation processes for plat-form videogames. As Procedural Content Generation (PCG) techniques evolved from the creation of simple elements to the construction of complete levels and scenarios, the principles behind the generation algorithms became more ambitious and complex, representing features that beforehand were only possible with human design. PCG goes beyond the search for valid geometries that can be used as levels, where multiple challenges are represented in an adequate way. It is also a search for user-centred design content and the creativity sparks of humanly created content. In order to improve the creativity capabilities of such generation algorithms, we conducted part of our research directed to the creation of new techniques using more ambitious design patterns. For this purpose, we have implemented two overall structure generation algorithms and created an addi-tional adaptation algorithm. The later can transform simple branched paths into more compelling game challenges by adding items and other elements in specific places, such as gates and levers for their activation. Such approach is suitable to avoid excessive level linearity and to represent certain design patterns with additional content richness. Moreover, content adaptation was transposed from general design domain to user-centred principles. In this particular case, we analysed success and failure patterns in action videogames and proposed a set of metrics to estimate difficulty, taking into account that each user has a different perception of that concept. This type of information serves the generation algorithms to make them more directed to the creation of personalised experiences. Furthermore, the conducted research also aimed to the integration of different techniques into a common ground. For this purpose, we have developed a general framework to represent content of platform videogames, compatible with several titles within the genre. Our algorithms run over this framework, whereby they are generic and game independent. We defined a modular architecture for the generation process, using this framework to normalise the content that is shared by multiple modules. A level editor tool was also created, which allows human level design and the testing of automatic generation algorithms. An adapted version of the editor was implemented for the semi-automatic creation of levels, in which the designer may simply define the type of content that he/she desires, in the form of quests and missions, and the system creates a corresponding level structure. This materialises our idea of bridging human high-level design patterns with lower level automated generation algorithms. Finally, we integrated the different contributions into a game prototype. This implementation allowed testing the different proposed approaches altogether, reinforcing the validity of the proposed archi-tecture and framework. It also allowed performing a more complete gameplay data retrieval in order to strengthen and validate the proposed metrics regarding difficulty perceptions

Dynamic theme-based narrative systems

The advent of videogames, and the new forms of expressions they offered, sprouted the possibility of presenting narratives in ways that could capitalize on unique qualities of the media, most notably the agency found in their interactive nature. In spite of many people in the game studies’ field interested in how far said novelty could bring narrative experiences, most approached the creation of narrative systems from a structural approach (especially the classical Aristotelian one), and concurrently, with a bottom-up (characters defining a world) or top-down (world defining characters) perspective. While those more mainstream takes have been greatly progressing what interactive digital narrative can be, this research intended to take a bit of a detour, proposing a functionally similar system that emphasized thematic coherence and responsiveness above all else. Once the theoretical formulation was done, taking into consideration previously similar or tangential systems, a prototype would be developed to make a first step towards validating the proposal, and contribute to building a better understanding of the field’s possibilities

On the definition of non-player character behaviour for real-time simulated virtual environments.

Computer games with complex virtual worlds, which are populated by artificial characters and creatures, are the most visible application of artificial intelligence techniques. In recent years game development has been fuelled by dramatic advances in computer graphics hardware which have led to a rise in the quality of real-time computer graphics and increased realism in computer games. As a result of these developments video games are gaining acceptance and cultural significance as a form of art and popular culture. An important factor for the attainment of realism in games is the artificially intelligent behaviour displayed by the virtual entities that populate the games' virtual worlds. It is our firm belief that to further improve the behaviour of virtual entities, game AI development will have to mirror the advances achieved in game graphics. A major contributing factor for these advancements has been the advent of programmable shaders for real-time graphics, which in turn has been significantly simplified by the introduction of higher level programming languages for the creation of shaders. This has demonstrated that a good system can be vastly improved by the addition of a programming language. This thesis presents a similar (syntactic) approach to the definition of the behaviour of virtual entities in computer games. We introduce the term behaviour definition language (BDL), describing a programming language for the definition of game entity behaviour. We specify the requirements for this type of programming language, which are applied to the development and implementation of several behaviour definition languages, culminating in the design of a new game-genre independent behaviour definition (scripting) language. This extension programming language includes several game AI techniques within a single unified system, allowing the use of different methods of behaviour definition. A subset of the language (itself a BDL) was implemented as a proof of concept of this design, providing a framework for the syntactic definition of the behaviour of virtual entities in computer games

Collaborative narrative generation in persistent virtual environments

This thesis describes a multi-agent approach to generating narrative based on the activities of participants in large-scale persistent virtual environments, such as massivelymultiplayer online role-playing games (MMORPGs). These environments provide diverse interactive experiences for large numbers of simultaneous participants. Involving such participants in an overarching narrative experience has presented challenges due to the difficulty of incorporating the individual actions of so many participants into a single coherent storyline. Various approaches have been adopted in an attempt to solve this problem, such as guiding players to follow pre-designed storylines, or giving them goals to achieve that advance the storyline, or by having developers (‘dungeon masters’) adapt the narrative to the real-time actions of players. However these solutions can be inflexible, and/or fail to take player interaction into account, or do so only at the collective level, for groups of players. This thesis describes a different approach, in which embodied witness-narrator agents observe participants’ actions in a persistent virtual environment and generate narrative from reports of those actions. The generated narrative may be published to external audiences, e.g., via community websites, Internet chatrooms, or SMS text messages, or fed back into the environment in real-time to embellish and enhance the ongoing experience with new narrative elements derived from participants’ own achievements. The design and implementation of this framework is described in detail, and compared to related work. Results of evaluating the framework, both technically, and through a live study, are presented and discussed

Collaborative narrative generation in persistent virtual environments

This thesis describes a multi-agent approach to generating narrative based on the activities of participants in large-scale persistent virtual environments, such as massivelymultiplayer online role-playing games (MMORPGs). These environments provide diverse interactive experiences for large numbers of simultaneous participants. Involving such participants in an overarching narrative experience has presented challenges due to the difficulty of incorporating the individual actions of so many participants into a single coherent storyline. Various approaches have been adopted in an attempt to solve this problem, such as guiding players to follow pre-designed storylines, or giving them goals to achieve that advance the storyline, or by having developers (‘dungeon masters’) adapt the narrative to the real-time actions of players. However these solutions can be inflexible, and/or fail to take player interaction into account, or do so only at the collective level, for groups of players. This thesis describes a different approach, in which embodied witness-narrator agents observe participants’ actions in a persistent virtual environment and generate narrative from reports of those actions. The generated narrative may be published to external audiences, e.g., via community websites, Internet chatrooms, or SMS text messages, or fed back into the environment in real-time to embellish and enhance the ongoing experience with new narrative elements derived from participants’ own achievements. The design and implementation of this framework is described in detail, and compared to related work. Results of evaluating the framework, both technically, and through a live study, are presented and discussed

Personalized Game Content Generation and Recommendation for Gamified Systems

Gamification, that is, the usage of game content in non-game contexts, has been successfully employed in several application domains to foster engagement, as well as to influence the behavior of end users. Although gamification is often effective in inducing behavioral changes in citizens, the difficulty in retaining players and sustaining the acquired behavior over time, shows some limitations of this technology. That is especially unfortunate, because changing players’ demeanor (which have been shaped for a long time), cannot be immediately internalized; rather, the gamification incentive must be reinforced to lead to stabilization. This issue could be sourced from utilizing static game content and a one-size-fits-all strategy in generating the content during the game. This reveals the need for dynamic personalization over the course of the game. Our research hypothesis is that we can overcome these limitations with Procedural Content Generation (PCG) of playable units that appeal to each individual player and make her user experience more varied and compelling. In this thesis, we propose a deep, large and long solution, deployed in two main phases of Design and Integration to tackle these limitations. To support the former phase, we present a “PCG and Recommender system” to automate the generation and recommendation of playable units, named “Challenges”, which are Personalized and Contextualized on the basis of players’ preferences, skills, etc., and the game ulterior objectives. To this end, we develop a multi-layered framework to generate the personalized game content to be assigned and recommended to the players involved in the gamified system. To support the latter phase, we integrate two modules into the system including Machine Learning (ML) and Player Modeling, in order to optimize the challenge selection process and learning players’ behavior to further improve the personalization, by deriving the style of the player, respectively. We have carried out the implementation and evaluation of the proposed framework and its integration in two different contexts. First, we assess our Automatic Procedural Content Generation and Recommendation (APCGR) system within a large-scale and long-running open field experiment promoting sustainable urban mobility that lasted twelve weeks and involved more than 400 active players. Then, we implement the “Player Modeling” module (in the integration phase) in an educational interactive game domain to assess the performance of the proposed play style extraction approach. The contributions of this dissertation are a first step toward the application of machine learning in automating the procedural content generation and recommendation in gamification systems