ViZDoom Competitions: Playing Doom from Pixels

This paper presents the first two editions of Visual Doom AI Competition, held in 2016 and 2017. The challenge was to create bots that compete in a multi-player deathmatch in a first-person shooter (FPS) game, Doom. The bots had to make their decisions based solely on visual information, i.e., a raw screen buffer. To play well, the bots needed to understand their surroundings, navigate, explore, and handle the opponents at the same time. These aspects, together with the competitive multi-agent aspect of the game, make the competition a unique platform for evaluating the state of the art reinforcement learning algorithms. The paper discusses the rules, solutions, results, and statistics that give insight into the agents' behaviors. Best-performing agents are described in more detail. The results of the competition lead to the conclusion that, although reinforcement learning can produce capable Doom bots, they still are not yet able to successfully compete against humans in this game. The paper also revisits the ViZDoom environment, which is a flexible, easy to use, and efficient 3D platform for research for vision-based reinforcement learning, based on a well-recognized first-person perspective game Doom

Confessions of a live coder

This paper describes the process involved when a live coder decides to learn a new musical programming language of another paradigm. The paper introduces the problems of running comparative experiments, or user studies, within the field of live coding. It suggests that an autoethnographic account of the process can be helpful for understanding the technological conditioning of contemporary musical tools. The author is conducting a larger research project on this theme: the part presented in this paper describes the adoption of a new musical programming environment, Impromptu, and how this affects the author’s musical practice

Learning cloth manipulation with demonstrations

Recent advances in Deep Reinforcement learning and computational capabilities of GPUs have led to variety of research being conducted in the learning side of robotics. The main aim being that of making autonomous robots that are capable of learning how to solve a task on their own with minimal requirement for engineering on the planning, vision, or control side. Efforts have been made to learn the manipulation of rigid objects through the help of human demonstrations, specifically in the tasks such as stacking of multiple blocks on top of each other, inserting a pin into a hole, etc. These Deep RL algorithms successfully learn how to complete a task involving the manipulation of rigid objects, but autonomous manipulation of textile objects such as clothes through Deep RL algorithms is still not being studied in the community. The main objectives of this work involve, 1) implementing the state of the art Deep RL algorithms for rigid object manipulation and getting a deep understanding of the working of these various algorithms, 2) Creating an open-source simulation environment for simulating textile objects such as clothes, 3) Designing Deep RL algorithms for learning autonomous manipulation of textile objects through demonstrations.Peer ReviewedPreprin

Extending a Game Engine with Machine Learning and Artificial Intelligence

Since the early days of Artificial Intelligence (AI), video games have been a popular testbed for evaluating methods. However, not until a long time ago, this only included building AI agents for playing board games like chess. In the last decade, researchers have found out that games are also a rich source for other types of AI problems. At the same time, Machine Learning (ML) has entered its golden age with the advancements of deep learning. In games, this has led to a wide range of novel AI methods for solving various problems like playing games with super-human performance. Despite the significant advances, state-of-the-art AI methods are still far from being used in commercial games. One of the main reasons is that the tools used by researchers and game developers are different, which makes it difficult to use open-source codes in game projects. Furthermore, implementing these methods requires a moderate understanding of ML, which is not among the skill set of a regular game programmer. This calls for plug-and-play tools that enable game developers to deploy AI methods with minimum cost. In this thesis, we develop a library that enables game developers to use state-of-the-art ML methods in their commercial projects. This library integrates Tensorflow, a modern ML toolbox, into Unity, the most common game engine in the industry. This library uses C# with intuitive Keras-like API for building and training models. We have also implemented several state-of-the-art algorithms, including Proximal Policy Optimization (PPO), Matrix Adaptation Evolution Strategies (MA-ES), and Generative Adversarial Networks (GAN). This library can also be used along with Unity ML-Agents, the Unity plugin for building AI-training environments. Moreover, we provide various examples that demonstrate the library and the algorithms. One important example is a game called Calamachine Union, where the core game mechanic includes training the AI

Training Machine Learning Agents in a 3D Game Engine

Artificial intelligence (AI) and video games benefit from each other. Games provide a challenging domain for testing learning algorithms, and AI provides a framework to designing and implementing intelligent behavior, which reinforces meaningful play. Medium and small studios, and independent game developers, have limited resources to design, implement, and maintain agents with reactive behavior. In this research, we trained agents using machine learning (ML), aiming to find an alternative to expensive traditional algorithms for intelligent behavior used in video games. We use Unity as a game engine to implement the environments and TensorFlow for the neural network training

Developing an Open-Source Lightweight Game Engine with DNN Support

With the growth of artificial intelligence and deep learning technology, we have many active research works to apply the related techniques in various fields. To test and apply the latest machine learning techniques in gaming, it will be very useful to have a light-weight game engine for quick prototyping. Our game engine is implemented in a cost-effective way, in comparison to well-known commercial proprietary game engines, by utilizing open source products. Due to its simple internal architecture, our game engine is especially beneficial for modifying and reviewing the new functions through quick and repetitive tests. In addition, the game engine has a DNN (deep neural network) module, with which the proposed game engine can apply deep learning techniques to the game features, through applying deep learning algorithms in real-time. Our DNN module uses a simple C++ function interface, rather than additional programming languages and/or scripts. This simplicity enables us to apply machine learning techniques more efficiently and casually to the game applications. We also found some technical issues during our development with open sources. These issues mostly occurred while integrating various open source products into a single game engine. We present details of these technical issues and our solution

A framework for quantitative analysis of user-generated spatial data

This paper proposes a new framework for automated analysis of game-play metrics for aiding game designers in finding out the critical aspects of the game caused by factors like design modications, change in playing style, etc. The core of the algorithm measures similarity between spatial distribution of user generated in-game events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on a data set collected from a modern, multiplayer First Person Shooter, together with application examples of its use. The proposed framework can be used to accompany traditional testing tools and make the game design process more efficient