Methods of multi-agent movement control and coordination of groups of mobile units in a real-time strategy games

Tato práce nabízí metodu pro reaktivní řízení jednotek v real-time strategické (RTS) počitačové hře pomocí multi-agentních potenciálových polí. Klasická RTS hra StarCraft: Broodwar byla vybrána jako testovací platforma díky jejímu postavení na konkurenční scéně umělé inteligence (UI). Nabízená umělá inteligence ovládá své jednotky pomocí umístění různých potenciálových polí na objekty a na místa v herním světě. Snahou této práce je vylepšit předchozí metody využívajicí potenciálová pole.This thesis proposes an approach to Reactive Control in Real-Time Strategy (RTS) computer games using Multi-Agent Potential Fields. The classic RTS title StarCraft: Brooodwar has been chosen as testing platform due to its status in the competitive Artificial Intelligence (AI) scene. The proposed AI controls its units by placing different types of potential fields in objects and places around the game world. This work is an attempt to improve previous methods done with Potential Field in RTS

Evolving Effective Micro Behaviors for Real-Time Strategy Games

Real-Time Strategy games have become a new frontier of artificial intelligence research. Advances in real-time strategy game AI, like with chess and checkers before, will significantly advance the state of the art in AI research. This thesis aims to investigate using heuristic search algorithms to generate effective micro behaviors in combat scenarios for real-time strategy games. Macro and micro management are two key aspects of real-time strategy games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes against equal numbers of opponent units or win even when outnumbered. In this research, we use influence maps and potential fields as a basis representation to evolve micro behaviors. We first compare genetic algorithms against two types of hill climbers for generating competitive unit micro management. Second, we investigated the use of case-injected genetic algorithms to quickly and reliably generate high quality micro behaviors. Then we compactly encoded micro behaviors including influence maps, potential fields, and reactive control into fourteen parameters and used genetic algorithms to search for a complete micro bot, ECSLBot. We compare the performance of our ECSLBot with two state of the art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular real-time strategy game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. In addition, the same approach works to create competitive micro behaviors in another game SeaCraft. Using parallelized genetic algorithms to evolve parameters in SeaCraft we are able to speed up the evolutionary process from twenty one hours to nine minutes. We believe this work provides evidence that genetic algorithms and our representation may be a viable approach to creating effective micro behaviors for winning skirmishes in real-time strategy games

Online Build-Order Optimization for Real-Time Strategy Agents Using Multi-Objective Evolutionary Algorithms

The investigation introduces a novel approach for online build-order optimization in real-time strategy (RTS) games. The goal of our research is to develop an artificial intelligence (AI) RTS planning agent for military critical decision- making education with the ability to perform at an expert human level, as well as to assess a players critical decision- making ability or skill-level. Build-order optimization is modeled as a multi-objective problem (MOP), and solutions are generated utilizing a multi-objective evolutionary algorithm (MOEA) that provides a set of good build-orders to a RTS planning agent. We de ne three research objectives: (1) Design, implement and validate a capability to determine the skill-level of a RTS player. (2) Design, implement and validate a strategic planning tool that produces near expert level build-orders which are an ordered sequence of actions a player can issue to achieve a goal, and (3) Integrate the strategic planning tool into our existing RTS agent framework and an RTS game engine. The skill-level metric we selected provides an original and needed method of evaluating a RTS players skill-level during game play. This metric is a high-level description of how quickly a player executes a strategy versus known players executing the same strategy. Our strategic planning tool combines a game simulator and an MOEA to produce a set of diverse and good build-orders for an RTS agent. Through the integration of case-base reasoning (CBR), planning goals are derived and expert build- orders are injected into a MOEA population. The MOEA then produces a diverse and approximate Pareto front that is integrated into our AI RTS agent framework. Thus, the planning tool provides an innovative online approach for strategic planning in RTS games. Experimentation via the Spring Engine Balanced Annihilation game reveals that the strategic planner is able to discover build-orders that are better than an expert scripted agent and thus achieve faster strategy execution times

Study of Computational Intelligence Algorithms to Detect Behaviour Patterns

In order to achieve the game flow and increase player retention, it is important that games difficulty matches player skills. As a consequence, to evaluate how people play a game is a crucial component, because detecting gamers strategies in video-games, it is possible to fix the game difficulty. The main problem to detect the strategies is whether attributes selected to define the strategies correctly detect the actions of the player. To study the player strategies, we will use a Real Time Stategy (RTS) game. In a RTS the players make use of units and structures to secure areas of a map and/or destroy the opponents resources. In this work, we will extract the real-time information about the players strategies using a platform base on the RTS game. After gathering information, the attributes that define the player strategies are evaluated using unsupervised learning algorithm (K-Means and Spectral Clustering). Finally, we will study the similitude among several gameplays where players use different strategies.A fin de lograr que el flujo del juego mejore y la captación de jugadores aumente, es importante que la dificultad del juego se ajuste a las habilidades del jugador. Como consecuencia, evaluar como juega la gente un juego es un aspecto importante, porque detectando las estrategias de los jugadores en los vídeo juegos, permite adapta la dificultad del juego. El problema principal para detectar las estrategias es si los atributos seleccionados para definir las estrategias definen correctamente las acciones del jugador. Para estudiar las estrategias de los jugadores, usaremos un juego de estrategia en tiempo real (Reat Time Strategy (RTS) en inglés). En un RTS los jugadores hacen uso de unidades y estructuras para asegurar áreas del mapa y/o destruir los recursos de los oponentes. En este trabajo, extraeremos información en tiempo real acerca de las estrategias usando una plataforma basada en un juego de RTS. Después de recoger la información, los atributos que definen las estrategias de los jugadores son evaluados mediante algoritmos de aprendizaje no supervisado (K-Means y Spectral Clustering). Finalmente, estudiaremos la similitud entre diversas partidas donde los jugadores utilizar diferentes estrategias.Este trabajo ha sido financiado por Airbus Defence & Space (Proyecto Savier: FUAM-076914) y parcialmente por TIN2010-19872

Goal reasoning for autonomous agents using automated planning

Mención Internacional en el título de doctorAutomated planning deals with the task of finding a sequence of actions, namely a plan, which achieves a goal from a given initial state. Most planning research consider goals are provided by a external user, and agents just have to find a plan to achieve them. However, there exist many real world domains where agents should not only reason about their actions but also about their goals, generating new ones or changing them according to the perceived environment. In this thesis we aim at broadening the goal reasoning capabilities of planningbased agents, both when acting in isolation and when operating in the same environment as other agents. In single-agent settings, we firstly explore a special type of planning tasks where we aim at discovering states that fulfill certain cost-based requirements with respect to a given set of goals. By computing these states, agents are able to solve interesting tasks such as find escape plans that move agents in to safe places, hide their true goal to a potential observer, or anticipate dynamically arriving goals. We also show how learning the environment’s dynamics may help agents to solve some of these tasks. Experimental results show that these states can be quickly found in practice, making agents able to solve new planning tasks and helping them in solving some existing ones. In multi-agent settings, we study the automated generation of goals based on other agents’ behavior. We focus on competitive scenarios, where we are interested in computing counterplans that prevent opponents from achieving their goals. We frame these tasks as counterplanning, providing theoretical properties of the counterplans that solve them. We also show how agents can benefit from computing some of the states we propose in the single-agent setting to anticipate their opponent’s movements, thus increasing the odds of blocking them. Experimental results show how counterplans can be found in different environments ranging from competitive planning domains to real-time strategy games.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidenta: Eva Onaindía de la Rivaherrera.- Secretario: Ángel García Olaya.- Vocal: Mark Robert

Artificial intelligence in co-operative games with partial observability

This thesis investigates Artificial Intelligence in co-operative games that feature Partial Observability. Most video games feature a combination of both co-operation, as well as Partial Observability. Co-operative games are games that feature a team of at least two agents, that must achieve a shared goal of some kind. Partial Observability is the restriction of how much of an environment that an agent can observe. The research performed in this thesis examines the challenge of creating Artificial Intelligence for co-operative games that feature Partial Observability. The main contributions are that Monte-Carlo Tree Search outperforms Genetic Algorithm based agents in solving co-operative problems without communication, the creation of a co-operative Partial Observability competition promoting Artificial Intelligence research as well as an investigation of the effect of varying Partial Observability to Artificial Intelligence, and finally the creation of a high performing Monte-Carlo Tree Search agent for the game Hanabi that uses agent modelling to rationalise about other players

Примена виртуелних светова у истраживању теорије агената и инжењерском образовању

The focus of this doctoral dissertation is on exploring the potentials of virtual worlds, for applications in research and education. Regarding this, there are two central aspects that are explored in the dissertation. The first one considers the concept of autonomous agents, and agent theory in general, in the context of virtual worlds. The second aspect is related to the educational applications of virtual worlds, while especially focusing on the concept of virtual laboratories. An introduction to basic terminology related to the subject is given at the start of the dissertation. After that, a thorough analysis of the role of agents in virtual worlds is presented. This, among others, includes the analysis of the techniques that shape the agent’s behavior. The development of the virtual gamified educational system, specially dedicated to agents is then presented in the dissertation, along with a thorough description. While, in the end, analysis of the concept of virtual laboratories in STE (Science, Technology, and Engineering) disciplines is performed, and existing solutions are evaluated according to the criteria defined in the dissertation.Фокус ове докторске дисертације је на истраживању потенцијала виртуелних светова за примене у истраживањима и образовању. У вези са тим, постоје два главна аспекта која су обрађена у дисертацији. Први аспект се тиче концепта аутономних агената, као и теорије агената у целини, а у контексту виртуелних светова. Други аспект је везан за примену виртуелних светова у образовању, при чему је посебан акценат стављен на виртуелне лабораторије. На почетку дисертације је дат кратак увод који се тиче терминологије и појединих појмова везаних за област којом се ова дисертција бави. Након тога је представљена систематична и темељна анализа улоге агената у виртуелним световима. Између осталог, ово укључује и анализу техника потребних за обликовање понашања агената. Потом је у дисертацији детаљно представљен развој оригиналног виртуелног образовног система посвећеног агентима. На крају, анализиран је концепт виртуелних лабораторија у НТИ (наука, технологија, инжењерство) дисциплинама и извршена је евалуација постојећих решења у складу са критеријумима који су дефинисани у дисертацији