Online Build-Order Optimization for Real-Time Strategy Agents Using Multi-Objective Evolutionary Algorithms

The investigation introduces a novel approach for online build-order optimization in real-time strategy (RTS) games. The goal of our research is to develop an artificial intelligence (AI) RTS planning agent for military critical decision- making education with the ability to perform at an expert human level, as well as to assess a players critical decision- making ability or skill-level. Build-order optimization is modeled as a multi-objective problem (MOP), and solutions are generated utilizing a multi-objective evolutionary algorithm (MOEA) that provides a set of good build-orders to a RTS planning agent. We de ne three research objectives: (1) Design, implement and validate a capability to determine the skill-level of a RTS player. (2) Design, implement and validate a strategic planning tool that produces near expert level build-orders which are an ordered sequence of actions a player can issue to achieve a goal, and (3) Integrate the strategic planning tool into our existing RTS agent framework and an RTS game engine. The skill-level metric we selected provides an original and needed method of evaluating a RTS players skill-level during game play. This metric is a high-level description of how quickly a player executes a strategy versus known players executing the same strategy. Our strategic planning tool combines a game simulator and an MOEA to produce a set of diverse and good build-orders for an RTS agent. Through the integration of case-base reasoning (CBR), planning goals are derived and expert build- orders are injected into a MOEA population. The MOEA then produces a diverse and approximate Pareto front that is integrated into our AI RTS agent framework. Thus, the planning tool provides an innovative online approach for strategic planning in RTS games. Experimentation via the Spring Engine Balanced Annihilation game reveals that the strategic planner is able to discover build-orders that are better than an expert scripted agent and thus achieve faster strategy execution times

Game Theory and Prescriptive Analytics for Naval Wargaming Battle Management Aids

NPS NRP Technical ReportThe Navy is taking advantage of advances in computational technologies and data analytic methods to automate and enhance tactical decisions and support warfighters in highly complex combat environments. Novel automated techniques offer opportunities to support the tactical warfighter through enhanced situational awareness, automated reasoning and problem-solving, and faster decision timelines. This study will investigate how game theory and prescriptive analytics methods can be used to develop real-time wargaming capabilities to support warfighters in their ability to explore and evaluate the possible consequences of different tactical COAs to improve tactical missions. This study will develop a conceptual design of a real-time tactical wargaming capability. This study will explore data analytic methods including game theory, prescriptive analytics, and artificial intelligence (AI) to evaluate their potential to support real-time wargaming.N2/N6 - Information WarfareThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

A Multi-Objective Approach to Tactical Maneuvering Within Real Time Strategy Games

The real time strategy (RTS) environment is a strong platform for simulating complex tactical problems. The overall research goal is to develop artificial intelligence (AI) RTS planning agents for military critical decision making education. These agents should have the ability to perform at an expert level as well as to assess a players critical decision-making ability or skill-level. The nature of the time sensitivity within the RTS environment creates very complex situations. Each situation must be analyzed and orders must be given to each tactical unit before the scenario on the battlefield changes and makes the decisions no longer relevant. This particular research effort of RTS AI development focuses on constructing a unique approach for tactical unit positioning within an RTS environment. By utilizing multiobjective evolutionary algorithms (MOEAs) for finding an \optimal positioning solution, an AI agent can quickly determine an effective unit positioning solution with a fast, rapid response. The development of such an RTS AI agent goes through three distinctive phases. The first of which is mathematically describing the problem space of the tactical positioning of units within a combat scenario. Such a definition allows for the development of a generic MOEA search algorithm that is applicable to nearly every scenario. The next major phase requires the development and integration of this algorithm into the Air Force Institute of Technology RTS AI agent. Finally, the last phase involves experimenting with the positioning agent in order to determine the effectiveness and efficiency when placed against various other tactical options. Experimental results validate that controlling the position of the units within a tactical situation is an effective alternative for an RTS AI agent to win a battle

A Real-time Strategy Agent Framework and Strategy Classifier for Computer Generated Forces

This research effort is concerned with the advancement of computer generated forces AI for Department of Defense (DoD) military training and education. The vision of this work is agents capable of perceiving and intelligently responding to opponent strategies in real-time. Our research goal is to lay the foundations for such an agent. Six research objectives are defined: 1) Formulate a strategy definition schema effective in defining a range of RTS strategies. 2) Create eight strategy definitions via the schema. 3) Design a real-time agent framework that plays the game according to the given strategy definition. 4) Generate an RTS data set. 5) Create an accurate and fast executing strategy classifier. 6) Find the best counterstrategies for each strategy definition. The agent framework is used to play the eight strategies against each other and generate a data set of game observations. To classify the data, we first perform feature reduction using principal component analysis or linear discriminant analysis. Two classifier techniques are employed, k-means clustering with k-nearest neighbor and support vector machine. The resulting classifier is 94.1% accurate with an average classification execution speed of 7.14 us. Our research effort has successfully laid the foundations for a dynamic strategy agent

Anti-war and the cyber triangle : strategic implications of cyber operations and cyber security for the state

[From the introduction:]The main driver for this choice of research was the growing influence of Internet-related issues in contemporary politics in various fields. 2009 saw an intensification of this link between information and communication technologies and international relations, particularly in the field of intelligence and military, with the revelation of notorious cyber operations such as AURORA, Ghostnet and Night Dragon (see chapter II). While those events started to attract the broader attention of academics, it was not until the discovery of the Stuxnet malware in 2010 (see chapter IV) that the issue gained momentum in other fields as well. A computer malware targeting a nuclear enrichment facility in a foreign country amidst a latent conflict certainly raised a lot of questions that demanded answers. Its sophisticated design and potential implications for international relations as well as strategic studies was one of the main inspirations for this research.While the emergence of literature on espionage and sabotage in conjunction with the Internet can be traced back to the 1990's, Kello recognises that even in 2013 it remains a weakly developed area, stating that '[t]he range of conceivable cyber conflict is poorly understood by scholars and decision-makers, and it is unclear how conventional security mechanisms, such as deterrence and collective defence apply to this phenomenon' (Kello, 2013: 7). Thus, the aim of this research is to contribute to the literature in this way '[…] in addition to elucidating empirical cyber events, scholars can guide the design of policies to affect them' (Kello, 2013: 38-39). Undertaking research in a field which is state-of-the-art and therefore, highly volatile, presents a particular academic challenge. It does also however enable a researcher to make a potentially crucial contribution, a dent, in the current debate. In areas of research in a vacuum exists, it is imperative for scholars to contribute to filling up that academic lacuna. The main outcome therefore is supposed to be a contribution to the academic debate on the strategic relevance and conduct of cyber operations and the state’s response to it. The intellectual tools developed as part of this research may be of future use for policy-makers. The underlying question for the research is: What are the strategic implications of cyber operations for the state?The Economist recently saw 'intensifying cyber threats' as one of the top challenges for 2014 (The Economist, 2014). The revelations of the past years, starting with Stuxnet, Operation AURORA, APT-1, Red October and activities derived from the NSA Documents revealed by whistleblower Edward Snowden indicate that this threat will not abate soon. More and more states are readying themselves for future conflicts by developing defensive as well offensive cyber operations capabilities (Lewis, 2013b: 9-55). The latest domain for conflict resolution is currently being explored and exploited too by a growing number of different stakeholders. Based on the increased number of stakeholders and the intensity and number of occurrences of said events (see section 3.5 and appendix), its contemporary relevance is high and has been increasing for several years and looks set to continue. Guiding principles in the field of strategy is an important part of this development. Though the debate on strategic implications of cyber operations started in the early 1990's, and promoted under the auspices of the RAND Corporation, '[i]ntellectually, we are in a position not unlike that faced 65 years ago as we began to develop our thinking about nuclear weapons' (Kramer, 2012: I). Nye agrees, stating that 'in comparison to the nuclear revolution in military affairs, strategic studies of the cyber domain are chronologically equivalent to 1960 but conceptually more equivalent to 1950. Analysts are still not clear about the lessons of offense, defense, deterrence, escalation, norms, arms control, or how they fit together into a national strategy' (Nye, 2011: 19). Thus, an intensive academic analysis of this field is pivotal, especially within the framework of strategic studies, in order to enable strategic adaptation and decision-making (Kello, 2013: 14). The timeliness of events, paired with the lack of a properly developed strategic framework, signify the increased contemporary relevance for research of the strategic implications of cyber operations for the state.Definitions are very important in political science, and only more so for research in the field of cyber operations. In the absence of commonly agreed upon definitions for cyber operations, and a multitude of other terms such as cyber warfare, digital warfare, information warfare, electronic warfare (see sub-sections 3.1 and 3.2 as well as section 4) which are at once related and disparate, mean that clarity in definitions is centrally important. While definitions might normally differ slightly, all elements included in the definition of cyber operations might vary. This includes the stakeholders (and their representation as entity in the cyber domain), the means to conduct cyber operations, the platform where it is conducted (for example all digital devices, Internet only, electromagnetic spectrum) and the operations through which it is conducted (for example, if cyber espionage is included or not).Therefore, the coherent and comprehensive definition is of vital importance for the understanding of the research and more so for its outcomes. The terminology of this research applies for the state in the cyber domain, cyber operations and cyber strategy. Thus, the three key definitions which are developed in this research can be found below.The state and its representation in the cyber domain is defined in chapter I: The state’s representation of the cyber domain is the Critical National Information Infrastructure (CNII). The CNII is composed of a particular part of the information infrastructure which is vital to the function of the state according to the state-teachings of Jellinek: territory, people and legitimate use of violence.The definition of cyber operations as developed in chapter II: A cyber operation is the targeted use and hack of digital code by any individual, group, organization or state using digital networks, systems and connected devices, which is directed against CNII in order to steal, alter, destroy information or disrupt and deny functionality with the ultimate aim to weaken and/ or harm a targeted political unit.Subsequently, the definition of a cyber strategy in chapter IV: The development and employment of cyber operations, potentially integrated and coordinated with other operational domains and forms of information operations, to achieve or support the achievement of political objectives

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Adaptive CGFs Based on Grammatical Evolution

Computer generated forces (CGFs) play blue or red units in military simulations for personnel training and weapon systems evaluation. Traditionally, CGFs are controlled through rule-based scripts, despite the doctrine-driven behavior of CGFs being rigid and predictable. Furthermore, CGFs are often tricked by trainees or fail to adapt to new situations (e.g., changes in battle field or update in weapon systems), and, in most cases, the subject matter experts (SMEs) review and redesign a large amount of CGF scripts for new scenarios or training tasks, which is both challenging and time-consuming. In an effort to overcome these limitations and move toward more true-to-life scenarios, a study using grammatical evolution (GE) to generate adaptive CGFs for air combat simulations has been conducted. Expert knowledge is encoded with modular behavior trees (BTs) for compatibility with the operators in genetic algorithm (GA). GE maps CGFs, represented with BTs to binary strings, and uses GA to evolve CGFs with performance feedback from the simulation. Beyond-visual-range air combat experiments between adaptive CGFs and nonadaptive baseline CGFs have been conducted to observe and study this evolutionary process. The experimental results show that the GE is an efficient framework to generate CGFs in BTs formalism and evolve CGFs via GA

Forging Wargamers

How do we establish or improve wargaming education, including sponsors, participants, and future designers? The question stems from the uncomfortable truth that the wargaming discipline has no foundational pipeline, no established pathway from novice to master. Consequently, the wargaming community stands at a dangerous precipice at the convergence of a stagnant labor force and a patchwork system of passing institutional war-gaming knowledge. Unsurprisingly, this can lead to ill-informed sponsors, poorly scoped wargames, an unreliable standard of wargaming expertise, and worst of all, risks the decline of wargaming as an educational and analytical tool. This fundamental challenge is a recurring theme throughout this volume and each author offers their own perspective and series of recommendations