ASPIRE Adaptive strategy prediction in a RTS environment

When playing a Real Time Strategy(RTS) game against the non-human player(bot) it is important that the bot can do different strategies to create a challenging experience over time. In this thesis we aim to improve the way the bot can predict what strategies the player is doing by analyzing the replays of the given players games. This way the bot can change its strategy based upon the known knowledge of the game state and what strategies the player have used before. We constructed a Bayesian Network to handle the predictions of the opponent's strategy and inserted that into a preexisting bot. Based on the results from our experiments we can state that the Bayesian Network adapted to the strategies our bot was exposed to. In addition we can see that the Bayesian Network only predicted the possible strategies given the obtained information about the game state.INFO390MASV-INF

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

Strategi Adaptif Kelompok di Permainan Taktik Menggunakan Goal-Oriented Action Planning

Meningkatnya kompleksitas permainan elektronik modern seirama dengan peningkatan kebutuhan akan agen cerdas yang dapat dibangun dengan mudah. Salah satu permainan elektronik yang membutuhkan agen cerdas tersebut adalah permainan real-time tactics (RTT). Dalam tipe permainan ini perencanaan aksi yang baik dapat membuat permainan yang menantang bagi pemain. Penelitian ini mengekplorasi kemungkinan penggunaan dari (GOAP) pada sebuah permainan RTT. Dengan menggunakan GOAP, dinamisme taktik dapat dibentuk dengan waktu penggunaan yang tidak terasa berat dalam permainan. ================================================================================================================ Along with improvement of modern electronic games, necessity of an intelligent agent that easily build is needed. One of electronic games that need good intelligent agent is realtime tactics. In this game type, good action planning is necessary to provide best experience to the player. We explore usage possibility of Goal-Oriented Action Planning (GOAP) in tactical game. Using GOAP, tactic dinamism can be provided with reasonable amount of runtime

Штучний інтелект на основі нейронної мережі для гри в жанрі стратегія в реальному часі

Магістерська дисертація міститься на 118 сторінках та включає 44 рисунки, 5 таблицю та 31 бібліографічні посилання. Вона складається з наступних розділів: вступ, 5 розділів для основної частини, висновки, перелік посилань та 8 додатків. Ключові слова: нейронні мережі, штучний інтелект, ієрархічна мережа задач, стратегії в реальному часі, розподілена система навчання нейронних мереж. Магістерська дисертація присвячена розробці та опису штучного інтелекту з використанням нейронних мереж для гри жанрі стратегія в реальному часі. Актуальність обраної теми полягає в підвищенні ефективності агентів штучного інтелекту для ігор в жанрі стратегій в реальному часі, а також використання розподіленої системи з централізованим сервером як елементу покращення ефективності модулів штучного інтелекту. Метою роботи є створення системи агентів штучного інтелекту з використанням штучних мереж для забезпечення високої ефективності роботи ботів в іграх жанру стратегій в реальному часі. Об’єктом дослідження є штучний інтелект в іграх жанру стратегій в реальному часі, зокрема модулі мікроуправління (тактичний) та макроуправління (стратегічний), з використанням штучних нейронних мереж в комбінації з іншими підходами. Предметом дослідження є побудований з використанням штучних нейронних мереж штучний інтелект для гри в жанрі стратегії в реальному часі, а також розподілена система навчання штучного інтелекту з використанням централізованого серверу.The master's dissertation is on 118 pages and includes 44 figures, 5 tables and 31 bibliographic references. It consists of the following sections: introduction, 5 sections for the main part, conclusions, list of references and 8 appendices. The master's dissertation is devoted to the development and description of artificial intelligence using neural networks for the game genre of real-time strategy. The relevance of the chosen topic is to increase the efficiency of artificial intelligence agents for games in the genre of real-time strategy, as well as the use of a distributed system with a centralized server as an element of improving the efficiency of artificial intelligence modules. The aim of the work is to create a system of artificial intelligence agents using artificial networks to ensure high efficiency of bots in games genre strategy in real time. The object of research is artificial intelligence in real-time strategy games, including modules of microcontrol (tactical) and macrocontrol (strategic), using artificial neural networks in combination with other approaches. The subject of the study is artificial intelligence built using artificial neural networks to play in the genre of real-time strategy, as well as a distributed system of artificial intelligence training using a centralized server

Self Monitoring Goal Driven Autonomy Agents

The growing abundance of autonomous systems is driving the need for robust performance. Most current systems are not fully autonomous and often fail when placed in real environments. Via self-monitoring, agents can identify when their own, or externally given, boundaries are violated, thereby increasing their performance and reliability. Specifically, self-monitoring is the identification of unexpected situations that either (1) prohibit the agent from reaching its goal(s) or (2) result in the agent acting outside of its boundaries. Increasingly complex and open environments warrant the use of such robust autonomy (e.g., self-driving cars, delivery drones, and all types of future digital and physical assistants). The techniques presented herein advance the current state of the art in self-monitoring, demonstrating improved performance in a variety of challenging domains. In the aforementioned domains, there is an inability to plan for all possible situations. In many cases all aspects of a domain are not known beforehand, and, even if they were, the cost of encoding them is high. Self-monitoring agents are able to identify and then respond to previously unexpected situations, or never-before-encountered situations. When dealing with unknown situations, one must start with what is expected behavior and use that to derive unexpected behavior. The representation of expectations will vary among domains; in a real-time strategy game like Starcraft, it could be logically inferred concepts; in a mars rover domain, it could be an accumulation of actions\u27 effects. Nonetheless, explicit expectations are necessary to identify the unexpected. This thesis lays the foundation for self-monitoring in goal driven autonomy agents in both rich and expressive domains and in partially observable domains. We introduce multiple techniques for handling such environments. We show how inferred expectations are needed to enable high level planning in real-time strategy games. We show how a hierarchical structure of Goal-driven Autonomy (GDA) enables agents to operate within large state spaces. Within Hierarchical Task Network planning, we show how informed expectations identify states that are likely to prevent an agent from reaching its goals in dynamic domains. Finally, we give a model of expectations for self-monitoring at the meta-cognitive level, and empirical results of agents equipped with and without metacognitive expectations

Diseño e implementación de comportamientos inteligentes en StarCraft: Brood War

Los videojuegos se han convertido en los últimos años en el producto cultural que más beneficios aporta, por encima de la música y el cine. Esto es gracias a su constante crecimiento y multitud de géneros existentes. Uno de estos géneros, a su vez de los más antiguos son los juegos de estrategia, los cuales se pueden dividir en “Estrategia por turnos”, por ejemplo el ajedrez, donde los jugadores tienen un turno, normalmente limitado a cierta cantidad de tiempo, para llevar a cabo sus movimientos o acciones; y “Estrategia en tiempo real”, donde los jugadores juegan simultáneamente la partida teniendo que tomar multitud de diversas decisiones en un espacio de tiempo reducido. Los avances en la computación han hecho que esta segunda categoría tenga especial interés para el desarrollo de jugadores automáticos que usen Inteligencia Artificial, esto se debe a la necesidad de encontrar las mejores soluciones posibles en una cantidad de tiempo muy reducida, ya que la partida no puede detenerse por culpa de estos cálculos. Además, estos jugadores automáticos deben ser capaces de adaptarse a los cambios que ocurren a lo largo de la partida. Este trabajo se centra en el videojuego “StarCraft: Brood War” y en el desarrollo de un agente con el objetivo inicial de ser capaz de derrotar a la Inteligencia Artificial del propio juego. Para el desarrollo se ha utilizado el lenguaje Java, haciendo uso de la API JNI-BWAPI y ChaosLauncher para poder conectarse con el juego. La principal técnica utilizada para el desarrollo del agente han sido árboles de comportamiento, apoyándose en mapas de influencia.In recent years videogames have become more profitable cultural products than music and movies together thanks to its constant change and multiple genres. One of these genres is Strategy games, which can be divided into two groups. First one is called Turn Based Strategy. Chess is for instance one of the games that belong to this group. Some features of this group include that players play in turns or limited time per turn. On the other hand, the other group is called Real Time Strategy, where all players play at the same time making multiple decisions in a short time. The advances in computing have made Real Time Strategy very attractive for the development of automatic players using Artificial Intelligence. This is interesting due to the need of finding the set of best suitable actions in a limit amount of time. Notice that the game is continuously evolving and never stops. Thus, the sooner the decisions are taken, the better. Besides, this bots must be able to adapt themselves to the changes happening in the game. This work is focused in the videogame “StarCraft: Brood War” and the developing of an agent which aim is to be able to defeat the AI’s game. The bot has been developed in Java using the JNI-BWAPI API and ChaosLauncher for game connection. The main technique used has been behavior trees, supported with influence maps.Ingeniería Informátic

Goal Reasoning: Papers from the ACS Workshop

This technical report contains the 14 accepted papers presented at the Workshop on Goal Reasoning, which was held as part of the 2015 Conference on Advances in Cognitive Systems (ACS-15) in Atlanta, Georgia on 28 May 2015. This is the fourth in a series of workshops related to this topic, the first of which was the AAAI-10 Workshop on Goal-Directed Autonomy; the second was the Self-Motivated Agents (SeMoA) Workshop, held at Lehigh University in November 2012; and the third was the Goal Reasoning Workshop at ACS-13 in Baltimore, Maryland in December 2013

Applying Goal-Driven Autonomy to StarCraft

One of the main challenges in game AI is building agents that can intelligently react to unforeseen game situations. In real-time strategy games, players create new strategies and tactics that were not anticipated during development. In order to build agents capable of adapting to these types of events, we advocate the development of agents that reason about their goals in response to unanticipated game events. This results in a decoupling between the goal selection and goal execution logic in an agent. We present a reactive planning implementation of the Goal-Driven Autonomy conceptual model and demonstrate its application in StarCraft. Our system achieves a win rate of 73% against the built-in AI and outranks 48% of human players on a competitive ladder server