ASPIRE Adaptive strategy prediction in a RTS environment

When playing a Real Time Strategy(RTS) game against the non-human player(bot) it is important that the bot can do different strategies to create a challenging experience over time. In this thesis we aim to improve the way the bot can predict what strategies the player is doing by analyzing the replays of the given players games. This way the bot can change its strategy based upon the known knowledge of the game state and what strategies the player have used before. We constructed a Bayesian Network to handle the predictions of the opponent's strategy and inserted that into a preexisting bot. Based on the results from our experiments we can state that the Bayesian Network adapted to the strategies our bot was exposed to. In addition we can see that the Bayesian Network only predicted the possible strategies given the obtained information about the game state.INFO390MASV-INF

Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

In this paper, we introduce a skill-balancing mechanism for adversarial non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The objective of this mechanism is to approximately match the skill level of an NPC to an opponent in real-time. We test the technique in the context of a First-Person Shooter (FPS) game. Specifically, the technique adjusts a reinforcement learning NPC's proficiency with a weapon based on its current performance against an opponent. Firstly, a catalogue of experience, in the form of stored learning policies, is built up by playing a series of training games. Once the NPC has been sufficiently trained, the catalogue acts as a timeline of experience with incremental knowledge milestones in the form of stored learning policies. If the NPC is performing poorly, it can jump to a later stage in the learning timeline to be equipped with more informed decision-making. Likewise, if it is performing significantly better than the opponent, it will jump to an earlier stage. The NPC continues to learn in real-time using reinforcement learning but its policy is adjusted, as required, by loading the most suitable milestones for the current circumstances.Comment: IEEE Conference on Computational Intelligence and Games (CIG). August 201

Natively Implementing Deep Reinforcement Learning into a Game Engine

Artificial intelligence (AI) increases the immersion that players can have while playing games. Modern game engines, a middleware software used to create games, implement simple AI behaviors that developers can use. Advanced AI behaviors must be implemented manually by game developers, which decreases the likelihood of game developers using advanced AI due to development overhead. A custom game engine and custom AI architecture that handled deep reinforcement learning was designed and implemented. Snake was created using the custom game engine to test the feasibility of natively implementing an AI architecture into a game engine. A snake agent was successfully trained using the AI architecture, but the learned behavior was suboptimal. Although the learned behavior was suboptimal, the AI architecture was successfully implemented into a custom game engine because a behavior was successfully learned

Virtual Battlespace Behavior Generation Through Class Imitation

Military organizations need realistic training scenarios to ensure mission readiness. Developing the skills required to differentiate combatants from non-combatants is very important for ensuring the international law of armed conflict is upheld. In Simulated Training Environments, one of the open challenges is to correctly simulate the appearance and behavior of combatant and non-combatant agents in a realistic manner. This thesis outlines the construction of a data driven agent that is capable of imitating the behaviors of the Virtual BattleSpace 2 behavior classes while our agent is configured to advance to a geographically specific goal. The approach and the resulting agent promotes and motivates the idea that Opponent and Non-Combatant behaviors inside of simulated environments can be improved through the use of behavioral imitation

Game Challenge: A Factorial Analysis Approach

Video games that customize to a player\u27s experience level and abilities have the potential to allow a broader range of players to become engaged and maintain interest as they progress in experience level. A game that uniquely customizes the player\u27s experience could attract additional demographics to gaming, which will result in a distinct edge in marketability and potential revenue. This thesis examines a subsection of adaptive gaming systems from the perspective of identifying game factors that alter the level of difficulty. Our focus is to provide a solution useful to both research and commercial gaming communities by developing a system that simulates results offline yet can be integrated into online play. While online performance is the main goal of an adaptive system, the offline simulation provides several benefits. Offline simulation allows the elimination of insignificant factors from inclusion in the training and evolution phase of machine learning algorithms. In addition it provides commercial games with a useful tool or method for performing game balancing and level tuning. To test our approach we designed a test-bed version of the game Pac-Man. The experimental testbed alters environment variables to evaluate their effect on a set of selected response variables. Observing the results of several response variables provides the potential to represent multiple player states, though our focus is on controlling the difficulty for a player. The testbed will simulate the actions of both Pac-Man and the ghosts over a variety of different settings and strategies. The evaluation of a factor\u27s significance and its effect size are calculated using a factorial analysis approach. This method allows the identification of factors relevant to both individual strategies, and the set of all player strategies. Finally, as a proof of concept for both the online and adaptation prospects of this method, we developed a prototype adaptive system. Utilizing the relevant factor effects calculated in the factorial analysis, the prototype adapts to control the progress of the game towards targeted response variable intervals

Artificial and Computational Intelligence in Games (Dagstuhl Seminar 12191)

This report documents the program and the outcomes of Dagstuhl Seminar 12191 "Artificial and Computational Intelligence in Games". The aim for the seminar was to bring together creative experts in an intensive meeting with the common goals of gaining a deeper understanding of various aspects of artificial and computational intelligence in games, to help identify the main challenges in game AI research and the most promising venues to deal with them. This was accomplished mainly by means of workgroups on 14 different topics (ranging from search, learning, and modeling to architectures, narratives, and evaluation), and plenary discussions on the results of the workgroups. This report presents the conclusions that each of the workgroups reached. We also added short descriptions of the few talks that were unrelated to any of the workgroups