Training Machine Learning Agents in a 3D Game Engine

Artificial intelligence (AI) and video games benefit from each other. Games provide a challenging domain for testing learning algorithms, and AI provides a framework to designing and implementing intelligent behavior, which reinforces meaningful play. Medium and small studios, and independent game developers, have limited resources to design, implement, and maintain agents with reactive behavior. In this research, we trained agents using machine learning (ML), aiming to find an alternative to expensive traditional algorithms for intelligent behavior used in video games. We use Unity as a game engine to implement the environments and TensorFlow for the neural network training

Reuse potential assessment framework for gamification-based smart city pilots

The paper proposes a unified framework for assessing the re-use potential for the Smart Engagement Pilot currently being realized in the city of Ghent (Belgium). The pilot aims to stimulate the digital engagement in users (citizens) by involving them in online and offline communities, and increasing the social capital through the use of ICT (Information and Communications Technology). To engage the citizens, the pilot makes use of Gamification based entities (intelligent wireless sensors) embedded in public hardware, through which innovative games are organized in places of interest (neighbourhood, parks, schools, etc.). Once finished, this pilot will be re-used in other European cities under the context of CIP SMART IP project. Since, the success of a pilot in one city doesn't guarantee its success in the other, an objective socio-economic-organizational reuse assessment becomes critical. To do this assessment, we propose a framework, which uses a Key Performance Indicator (KPI) based scorecard to determine the roadblocks and battlefields that could deter such a transition

The Development of Theoretical Framework for In-App Purchasing for the Gaming Industry

The gaming industry is a multi-billion dollar business has evolved from video arcade games in the 1970s/80s to video game consoles and online games in the 1990s/2000s. Today, games can be played on smart phones and tablets which are initially offered for free. They make money later by offering in-app upgrades which promises to enhance the gaming experience. When a gamer engages in this purchase, the term used is in-app purchasing. Normally, a frequent gamer is interested to buy upgrades. For this, a game company must understand the needs and wants of a gamer, and design an intelligent game system which gathers and process information about the behaviour of a gamer when he/she interacts (plays) with it. The game system will suggest a list of in-app(s) which are priced according to the effectiveness for the gamer to upgrade. This research-in-progress paper presents a theoretical framework to study in-app purchasing. The In-App Purchasing Theoretical Framework is backed by Behavioural Game Theory, which is to examine gamer’s behaviour, and Theory of Consumption Values, which identify the game’s values which are gained from his/her gaming experience

Game State and Action Abstracting Monte Carlo Tree Search for General Strategy Game-Playing

When implementing intelligent agents for strategy games, we observe that search-based methods struggle with the complexity of such games. To tackle this problem, we propose a new variant of Monte Carlo Tree Search which can incorporate action and game state abstractions. Focusing on the latter, we developed a game state encoding for turn-based strategy games that allows for a flexible abstraction. Using an optimization procedure, we optimize the agent's action and game state abstraction to maximize its performance against a rule-based agent. Furthermore, we compare different combinations of abstractions and their impact on the agent's performance based on the Kill the King game of the Stratega framework. Our results show that action abstractions have improved the performance of our agent considerably. Contrary, game state abstractions have not shown much impact. While these results may be limited to the tested game, they are in line with previous research on abstractions of simple Markov Decision Processes. The higher complexity of strategy games may require more intricate methods, such as hierarchical or time-based abstractions, to further improve the agent's performance

Affect and believability in game characters:a review of the use of affective computing in games

Virtual agents are important in many digital environments. Designing a character that highly engages users in terms of interaction is an intricate task constrained by many requirements. One aspect that has gained more attention recently is the effective dimension of the agent. Several studies have addressed the possibility of developing an affect-aware system for a better user experience. Particularly in games, including emotional and social features in NPCs adds depth to the characters, enriches interaction possibilities, and combined with the basic level of competence, creates a more appealing game. Design requirements for emotionally intelligent NPCs differ from general autonomous agents with the main goal being a stronger player-agent relationship as opposed to problem solving and goal assessment. Nevertheless, deploying an affective module into NPCs adds to the complexity of the architecture and constraints. In addition, using such composite NPC in games seems beyond current technology, despite some brave attempts. However, a MARPO-type modular architecture would seem a useful starting point for adding emotions

Decision Making from Confidence Measurement on the Reward Growth using Supervised Learning: A Study Intended for Large-Scale Video Games

peer reviewedVideo games have become more and more complex over the past decades. Today, players wander in visually and option- rich environments, and each choice they make, at any given time, can have a combinatorial number of consequences. However, modern artificial intelligence is still usually hard-coded, and as the game environments become increasingly complex, this hard-coding becomes exponentially difficult. Recent research works started to let video game autonomous agents learn instead of being taught, which makes them more intelligent. This contribution falls under this very perspective, as it aims to develop a framework for the generic design of autonomous agents for large-scale video games. We consider a class of games for which expert knowledge is available to define a state quality function that gives how close an agent is from its objective. The decision making policy is based on a confidence measurement on the growth of the state quality function, computed by a supervised learning classification model. Additionally, no stratagems aiming to reduce the action space are used. As a proof of concept, we tested this simple approach on the collectible card game Hearthstone and obtained encouraging results