Procedural Content Generation for Real-Time Strategy Games

Videogames are one of the most important and profitable sectors in the industry of entertainment. Nowadays, the creation of a videogame is often a large-scale endeavor and bears many similarities with, e.g., movie production. On the central tasks in the development of a videogame is content generation, namely the definition of maps, terrains, non-player characters (NPCs) and other graphical, musical and AI-related components of the game. Such generation is costly due to its complexity, the great amount of work required and the need of specialized manpower. Hence the relevance of optimizing the process and alleviating costs. In this sense, procedural content generation (PCG) comes in handy as a means of reducing costs by using algorithmic techniques to automatically generate some game contents. PCG also provides advantages in terms of player experience since the contents generated are typically not fixed but can vary in different playing sessions, and can even adapt to the player herself. For this purpose, the underlying algorithmic technique used for PCG must be also flexible and adaptable. This is the case of computational intelligence in general and evolutionary algorithms in particular. In this work we shall provide an overview of the use of evolutionary intelligence for PCG, with special emphasis on its use within the context of real-time strategy games. We shall show how these techniques can address both playability and aesthetics, as well as improving the game AI

Game Artificial Intelligence: Challenges for the Scientific Community

This paper discusses some of the most interesting challenges to which the games research community members may face in the área of the application of arti cial or computational intelligence techniques to the design and creation of video games. The paper focuses on three lines that certainly will in uence signi cantly the industry of game development in the near future, speci cally on the automatic generation of content, the a ective computing applied to video games and the generation of behaviors that manage the decisions of entities not controlled by the human player.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Coevolutionary algorithms for the optimization of strategies for red teaming applications

Red teaming (RT) is a process that assists an organization in finding vulnerabilities in a system whereby the organization itself takes on the role of an “attacker” to test the system. It is used in various domains including military operations. Traditionally, it is a manual process with some obvious weaknesses: it is expensive, time-consuming, and limited from the perspective of humans “thinking inside the box”. Automated RT is an approach that has the potential to overcome these weaknesses. In this approach both the red team (enemy forces) and blue team (friendly forces) are modelled as intelligent agents in a multi-agent system and the idea is to run many computer simulations, pitting the plan of the red team against the plan of blue team. This research project investigated techniques that can support automated red teaming by conducting a systematic study involving a genetic algorithm (GA), a basic coevolutionary algorithm and three variants of the coevolutionary algorithm. An initial pilot study involving the GA showed some limitations, as GAs only support the optimization of a single population at a time against a fixed strategy. However, in red teaming it is not sufficient to consider just one, or even a few, opponent‟s strategies as, in reality, each team needs to adjust their strategy to account for different strategies that competing teams may utilize at different points. Coevolutionary algorithms (CEAs) were identified as suitable algorithms which were capable of optimizing two teams simultaneously for red teaming. The subsequent investigation of CEAs examined their performance in addressing the characteristics of red teaming problems, such as intransitivity relationships and multimodality, before employing them to optimize two red teaming scenarios. A number of measures were used to evaluate the performance of CEAs and in terms of multimodality, this study introduced a novel n-peak problem and a new performance measure based on the Circular Earth Movers‟ Distance. Results from the investigations involving an intransitive number problem, multimodal problem and two red teaming scenarios showed that in terms of the performance measures used, there is not a single algorithm that consistently outperforms the others across the four test problems. Applications of CEAs on the red teaming scenarios showed that all four variants produced interesting evolved strategies at the end of the optimization process, as well as providing evidence of the potential of CEAs in their future application in red teaming. The developed techniques can potentially be used for red teaming in military operations or analysis for protection of critical infrastructure. The benefits include the modelling of more realistic interactions between the teams, the ability to anticipate and to counteract potentially new types of attacks as well as providing a cost effective solution

Rules of engagement : competitive coevolutionary dynamics in computational systems

Given that evolutionary biologists have considered coevolutionary interactions since the dawn of Darwinism, it is perhaps surprising that coevolution was largely overlooked during the formative years of evolutionary computing. It was not until the early 1990s that Hillis' seminal work thrust coevolution into the spotlight. Upon attempting to evolve fixed-length sorting networks, a problem with a long and competitive history, Hillis found that his standard evolutionary algorithm was producing sub-standard networks. In response, he decided to reciprocally evolve a population of testlists against the sorting network population; thus producing a coevolutionary system. The result was impressive; coevolution not only outperformed evolution, but the best network it discovered was only one comparison longer than the best-known solution. For the first time, a coevolutionary algorithm had been successfully applied to problem-solving. Pre-Hillis, the shortcomings of standard evolutionary algorithms had been understood for some time: whilst defining an adequate fitness function can be as challenging as the problem one is hoping to solve, once achieved, the accumulation of fitness-improving mutations can push a population towards local optima that are difficult to escape. Coevolution offers a solution. By allowing the fitness of each evolving individual to vary (through competition) with other reciprocally evolving individuals, coevolution removes the requirement of a fitness yardstick. In conjunction, the reciprocal adaptations of each individual begin to erode local optima as soon as they appear. However, coevolution is no panacea. As a problem-solving tool, coevolutionary algorithms suffer from some debilitating dynamics, each a result of the relative fitness assessment of individuals. In a single-, or multi-, population competitive system, coevolution may stabilize at a suboptimal equilibrium, or mediocre stable state; analogous to the traditional problem of local optima. Populations may become highly specialized in an unanticipated (and undesirable) manner; potentially resulting in brittle solutions that are fragile to perturbation. The system may cycle; producing dynamics similar to the children's game rock-paper-scissors. Disengagement may occur, whereby one population out-performs another to the extent that individuals cannot be discriminated on the basis of fitness alone; thus removing selection pressure and allowing populations to drift. Finally, coevolution's relative fitness assessment renders traditional visualization techniques (such as the graph of fitness over time) obsolete; thus exacerbating each of the above problems. This thesis attempts to better understand and address the problems of coevolution through the design and analysis of simple coevolutionary models. 'Reduced virulence' - a novel technique specifically designed to tackle disengagement - is developed. Empirical results demonstrate the ability of reduced virulence to combat disengagement both in simple and complex domains, whilst outperforming the only known competitors. Combining reduced virulence with diversity maintenance techniques is also shown to counteract mediocre stability and over-specialization. A critique of the CIAO plot - a visualization technique developed to detect coevolutionary cycling - highlights previously undocumented ambiguities; experimental evidence demonstrates the need for complementary visualizations. Extending the scope of visualization, a first exploration into coevolutionary steering is performed; a technique allowing the user to interact with a coevolutionary system during run-time. Using a simple model incorporating reduced virulence, the coevolutionary steering demonstration highlights the future potential of such tools for both research and education. The role of neutrality in coevolution is discussed in detail. Whilst much emphasis is placed upon neutral networks in the evolutionary computation literature, the nature of coevolutionary neutrality is generally overlooked. Preliminary ideas for modelling coevolutionary neutrality are presented. Finally, whilst this thesis is primarily aimed at a computing audience, strong reference to evolutionary biology is made throughout. Exemplifying potential crossover, the CIAO plot, a tool previously unused in biology, is applied to a simulation of E. Coli, with results con rming empirical observations of real bacteria.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Bayesian opponent modeling in adversarial game environments.

This thesis investigates the use of Bayesian analysis upon an opponent¿s behaviour in order to determine the desired goals or strategy used by a given adversary. A terrain analysis approach utilising the A* algorithm is investigated, where a probability distribution between discrete behaviours of an opponent relative to a set of possible goals is generated. The Bayesian analysis of agent behaviour accurately determines the intended goal of an opponent agent, even when the opponent¿s actions are altered randomly. The environment of Poker is introduced and abstracted for ease of analysis. Bayes¿ theorem is used to generate an effective opponent model, categorizing behaviour according to its similarity with known styles of opponent. The accuracy of Bayes¿ rule yields a notable improvement in the performance of an agent once an opponent¿s style is understood. A hybrid of the Bayesian style predictor and a neuroevolutionary approach is shown to lead to effective dynamic play, in comparison to agents that do not use an opponent model. The use of recurrence in evolved networks is also shown to improve the performance and generalizability of an agent in a multiplayer environment. These strategies are then employed in the full-scale environment of Texas Hold¿em, where a betting round-based approach proves useful in determining and counteracting an opponent¿s play. It is shown that the use of opponent models, with the adaptive benefits of neuroevolution aid the performance of an agent, even when the behaviour of an opponent does not necessarily fit within the strict definitions of opponent ¿style¿.Engineering and Physical Sciences Research Council (EPSRC

Effective Team Strategies using Dynamic Scripting

Forming effective team strategies using heterogeneous agents to accomplish a task can be a challenging problem. The number of combinations of actions to look through can be enormous, and having an agent that is really good at a particular sub-task is no guarantee that agent will perform well on a team with members with different abilities. Dynamic Scripting has been shown to be an effective way of improving behaviours with adaptive game AI. We present an approach that modifies the scripting process to account for the other agents in a game. By analyzing an agent\u27s allies and opponents we can create better starting scripts for the agents to use. Creating better starting points for the Dynamic Scripting process and will minimize the number of iterations needed to learn effective strategies, creating a better overall gaming experience

Environmental policy evaluation in the service of sustainable development: influence of the OECD Environmental Performance Reviews from the perspective of institutional economics

This thesis examines what consequences, through which pathways, and under which circumstances, does country-level environmental policy evaluation, carried out by an intergovernmental organisation, have for institutional change towards sustainable development. The philosophical and theoretical frameworks of institutional economics are proposed as the lens through which to assess experiences from evaluation research. This is done in order to elaborate a conceptual framework for analysing the approach, the impact and the factors conditioning the influence of environmental policy evaluation in the context of sustainable development. The OECD Environmental Performance Review programme is examined through an in-depth qualitative case study approach. It is found that in spite of engendering relatively modest interest among the public, the reviews increase the legitimacy of environmental policies and authorities, and strengthen a performance-oriented environmental policy emphasising evaluation, monitoring and cost-effectiveness. These impacts operate through four parallel pathways, representing the purposes of learning and accountability. The type and intensity of influence depends on the capacity of the reviews to mobilise key change agents and to be sensitive to the context, as well as on the credibility and political weight of the OECD in the reviewed country. It is concluded that notwithstanding the distance of the prevailing OECD conception of sustainability from the institutionalist principles adopted as the benchmark for the appraisal in this thesis, the reviews contribute meaningfully to sustainable development by redressing asymmetries of power, thereby improving some of the conditions for deliberative democracy