Representing the Impacts of Information on Agent Decision Making in Close Combat Constructive Simulations

Individuals often must make decisions based on incomplete sets of information resulting in imperfect situational awareness. Situational awareness (SA) entails the perception of critical factors in the environment, comprehending those factors, and predicting future events. SA is derived from an individual’s processing of signals, cues, information, data, and knowledge from various sources including the individual’s task environment, expectations, mission, training, personal knowledge and other cognitive factors. Ultimately, SA allows for inference, deduction, and prediction tasks which are extremely important to high level decision-making. Information inputs impacting individuals SA emanate from the direct observation of the external environment, communication with others, and interaction with electronic systems. This research investigates methods that will enable the determination of the impact that different sets of information have on the soldier’s decision-making processes and ultimately on task performance. The research goal is to design a method that can be used to determine an optimal set of information that would provide a Ground Soldier with the most critical information elements. This goal is especially important when considering the use of information and communications technology devices of limited bandwidth for the purpose of delivery information to Ground Soldiers in the battlespace

SCALING REINFORCEMENT LEARNING THROUGH FEUDAL MULTI-AGENT HIERARCHY

Militaries conduct wargames for training, planning, and research purposes. Artificial intelligence (AI) can improve military wargaming by reducing costs, speeding up the decision-making process, and offering new insights. Previous researchers explored using reinforcement learning (RL) for wargaming based on the successful use of RL for other human competitive games. While previous research has demonstrated that an RL agent can generate combat behavior, those experiments have been limited to small-scale wargames. This thesis investigates the feasibility and acceptability of -scaling hierarchical reinforcement learning (HRL) to support integrating AI into large military wargames. Additionally, this thesis also investigates potential complications that arise when replacing the opposing force with an intelligent agent by exploring the ways in which an intelligent agent can cause a wargame to fail. The resources required to train a feudal multi-agent hierarchy (FMH) and a standard RL agent and their effectiveness are compared in increasingly complicated wargames. While FMH fails to demonstrate the performance required for large wargames, it offers insight for future HRL research. Finally, the Department of Defense verification, validation, and accreditation process is proposed as a method to ensure that any future AI application applied to wargames are suitable.Lieutenant Colonel, United States ArmyApproved for public release. Distribution is unlimited

A Novel Way of Using Simulations to Support Urban Security Operations

The growing importance of security operations in urban terrain has triggered many attempts to address the perceived gaps in the readiness of security forces for this type of combat. One way to tackle the problem is to employ simulation techniques. Simulations are widely used to support both mission rehearsal and mission analysis, but these two applications tend to be seen as distinctly separate. We argue that integrating them in a unified framework can bring significant benefits for end-users. We perform a structured walk-through of such a unified system, in which a novel approach to integration through the behaviour cloning enabled the system to capture the operational knowledge of security experts, which is often difficult to express verbally. This capability emerged as essential for the operation of the integrated system. We also illustrate how the interplay between the system components for the mission analysis and mission rehearsal is realized

MODELLING & SIMULATION HYBRID WARFARE Researches, Models and Tools for Hybrid Warfare and Population Simulation

The Hybrid Warfare phenomena, which is the subject of the current research, has been framed by the work of Professor Agostino Bruzzone (University of Genoa) and Professor Erdal Cayirci (University of Stavanger), that in June 2016 created in order to inquiry the subject a dedicated Exploratory Team, which was endorsed by NATO Modelling & Simulation Group (a panel of the NATO Science & Technology organization) and established with the participation as well of the author. The author brought his personal contribution within the ET43 by introducing meaningful insights coming from the lecture of \u201cFight by the minutes: Time and the Art of War (1994)\u201d, written by Lieutenant Colonel US Army (Rtd.) Robert Leonhard; in such work, Leonhard extensively developed the concept that \u201cTime\u201d, rather than geometry of the battlefield and/or firepower, is the critical factor to tackle in military operations and by extension in Hybrid Warfare. The critical reflection about the time - both in its quantitative and qualitative dimension - in a hybrid confrontation it is addressed and studied inside SIMCJOH, a software built around challenges that imposes literally to \u201cFight by the minutes\u201d, echoing the core concept expressed in the eponymous work. Hybrid Warfare \u2013 which, by definition and purpose, aims to keep the military commitment of both aggressor and defender at the lowest - can gain enormous profit by employing a wide variety of non-military tools, turning them into a weapon, as in the case of the phenomena of \u201cweaponization of mass migrations\u201d, as it is examined in the \u201cDies Irae\u201d simulation architecture. Currently, since migration it is a very sensitive and divisive issue among the public opinions of many European countries, cynically leveraging on a humanitarian emergency caused by an exogenous, inducted migration, could result in a high level of political and social destabilization, which indeed favours the concurrent actions carried on by other hybrid tools. Other kind of disruption however, are already available in the arsenal of Hybrid Warfare, such cyber threats, information campaigns lead by troll factories for the diffusion of fake/altered news, etc. From this perspective the author examines how the TREX (Threat network simulation for REactive eXperience) simulator is able to offer insights about a hybrid scenario characterized by an intense level of social disruption, brought by cyber-attacks and systemic faking of news. Furthermore, the rising discipline of \u201cStrategic Engineering\u201d, as envisaged by Professor Agostino Bruzzone, when matched with the operational requirements to fulfil in order to counter Hybrid Threats, it brings another innovative, as much as powerful tool, into the professional luggage of the military and the civilian employed in Defence and Homeland security sectors. Hybrid is not the New War. What is new is brought by globalization paired with the transition to the information age and rising geopolitical tensions, which have put new emphasis on hybrid hostilities that manifest themselves in a contemporary way. Hybrid Warfare is a deliberate choice of an aggressor. While militarily weak nations can resort to it in order to re-balance the odds, instead military strong nations appreciate its inherent effectiveness coupled with the denial of direct responsibility, thus circumventing the rules of the International Community (IC). In order to be successful, Hybrid Warfare should consist of a highly coordinated, sapient mix of diverse and dynamic combination of regular forces, irregular forces (even criminal elements), cyber disruption etc. all in order to achieve effects across the entire DIMEFIL/PMESII_PT spectrum. However, the owner of the strategy, i.e. the aggressor, by keeping the threshold of impunity as high as possible and decreasing the willingness of the defender, can maintain his Hybrid Warfare at a diplomatically feasible level; so the model of the capacity, willingness and threshold, as proposed by Cayirci, Bruzzone and Gunneriusson (2016), remains critical to comprehend Hybrid Warfare. Its dynamicity is able to capture the evanescent, blurring line between Hybrid Warfare and Conventional Warfare. In such contest time is the critical factor: this because it is hard to foreseen for the aggressor how long he can keep up with such strategy without risking either the retaliation from the International Community or the depletion of resources across its own DIMEFIL/PMESII_PT spectrum. Similar discourse affects the defender: if he isn\u2019t able to cope with Hybrid Threats (i.e. taking no action), time works against him; if he is, he can start to develop counter narrative and address physical countermeasures. However, this can lead, in the medium long period, to an unforeseen (both for the attacker and the defender) escalation into a large, conventional, armed conflict. The performance of operations that required more than kinetic effects drove the development of DIMEFIL/PMESII_PT models and in turn this drive the development of Human Social Culture Behavior Modelling (HCSB), which should stand at the core of the Hybrid Warfare modelling and simulation efforts. Multi Layers models are fundamental to evaluate Strategies and Support Decisions: currently there are favourable conditions to implement models of Hybrid Warfare, such as Dies Irae, SIMCJOH and TREX, in order to further develop tools and war-games for studying new tactics, execute collective training and to support decisions making and analysis planning. The proposed approach is based on the idea to create a mosaic made by HLA interoperable simulators able to be combined as tiles to cover an extensive part of the Hybrid Warfare, giving the users an interactive and intuitive environment based on the \u201cModelling interoperable Simulation and Serious Game\u201d (MS2G) approach. From this point of view, the impressive capabilities achieved by IA-CGF in human behavior modeling to support population simulation as well as their native HLA structure, suggests to adopt them as core engine in this application field. However, it necessary to highlight that, when modelling DIMEFIL/PMESII_PT domains, the researcher has to be aware of the bias introduced by the fact that especially Political and Social \u201cscience\u201d are accompanied and built around value judgement. From this perspective, the models proposed by Cayirci, Bruzzone, Guinnarson (2016) and by Balaban & Mileniczek (2018) are indeed a courageous tentative to import, into the domain of particularly poorly understood phenomena (social, politics, and to a lesser degree economics - Hartley, 2016), the mathematical and statistical instruments and the methodologies employed by the pure, hard sciences. Nevertheless, just using the instruments and the methodology of the hard sciences it is not enough to obtain the objectivity, and is such aspect the representations of Hybrid Warfare mechanics could meet their limit: this is posed by the fact that they use, as input for the equations that represents Hybrid Warfare, not physical data observed during a scientific experiment, but rather observation of the reality that assumes implicitly and explicitly a value judgment, which could lead to a biased output. Such value judgement it is subjective, and not objective like the mathematical and physical sciences; when this is not well understood and managed by the academic and the researcher, it can introduce distortions - which are unacceptable for the purpose of the Science - which could be used as well to enforce a narrative mainstream that contains a so called \u201ctruth\u201d, which lies inside the boundary of politics rather than Science. Those observations around subjectivity of social sciences vs objectivity of pure sciences, being nothing new, suggest however the need to examine the problem under a new perspective, less philosophical and more leaned toward the practical application. The suggestion that the author want make here is that the Verification and Validation process, in particular the methodology used by Professor Bruzzone in doing V&V for SIMCJOH (2016) and the one described in the Modelling & Simulation User Risk Methodology (MURM) developed by Pandolfini, Youngblood et all (2018), could be applied to evaluate if there is a bias and the extent of the it, or at least making clear the value judgment adopted in developing the DIMEFIL/PMESII_PT models. Such V&V research is however outside the scope of the present work, even though it is an offspring of it, and for such reason the author would like to make further inquiries on this particular subject in the future. Then, the theoretical discourse around Hybrid Warfare has been completed addressing the need to establish a new discipline, Strategic Engineering, very much necessary because of the current a political and economic environment which allocates diminishing resources to Defense and Homeland Security (at least in Europe). However, Strategic Engineering can successfully address its challenges when coupled with the understanding and the management of the fourth dimension of military and hybrid operations, Time. For the reasons above, and as elaborated by Leonhard and extensively discussed in the present work, addressing the concern posed by Time dimension is necessary for the success of any military or Hybrid confrontation. The SIMCJOH project, examined under the above perspective, proved that the simulator has the ability to address the fourth dimension of military and non-military confrontation. In operations, Time is the most critical factor during execution, and this was successfully transferred inside the simulator; as such, SIMCJOH can be viewed as a training tool and as well a dynamic generator of events for the MEL/MIL execution during any exercise. In conclusion, SIMCJOH Project successfully faces new challenging aspects, allowed to study and develop new simulation models in order to support decision makers, Commanders and their Staff. Finally, the question posed by Leonhard in terms of recognition of the importance of time management of military operations - nowadays Hybrid Conflict - has not been answered yet; however, the author believes that Modelling and Simulation tools and techniques can represent the safe \u201ctank\u201d where innovative and advanced scientific solutions can be tested, exploiting the advantage of doing it in a synthetic environment

Semantic correlation of behavior for the interoperability of heterogeneous simulations

A desirable goal of military simulation training is to provide large scale or joint exercises to train personnel at higher echelons. To help meet this goal, many of the lower echelon combatants must consist of computer generated forces with some of these echelons composed of units from different simulations. The object of the research described is to correlate the behaviors of entities in different simulations so that they can interoperate with one another to support simulation training. Specific source behaviors can be translated to a form in terms of general behaviors which can then be correlated to any desired specific destination simulation behavior without prior knowledge of the pairing. The correlation, however, does not result in 100% effectiveness because most simulations have different semantics and were designed for different training needs. An ontology of general behaviors and behavior parameters, a database of source behaviors written in terms of these general behaviors with a database of destination behaviors. This comparison is based upon the similarity of sub-behaviors and the behavior parameters. Source behaviors/parameters may be deemed similar based upon their sub-behaviors or sub-parameters and their relationship (more specific or more general) to destination behaviors/parameters. As an additional constraint for correlation, a conversion path from all required destination parameters to a source parameter must be found in order for the behavior to be correlated and thus executed. The length of this conversion path often determines the similarity for behavior parameters, both source and destination. This research has shown, through a set of experiments, that heuristic metrics, in conjunction with a corresponding behavior and parameter ontology, are sufficient for the correlation of heterogeneous simulation behavior. These metrics successfully correlated known pairings provided by experts and provided reasonable correlations for behaviors that have no corresponding destination behavior. For different simulations, these metrics serve as a foundation for more complex methods of behavior correlation

Principles and Concepts of Agent-Based Modelling for Developing Geospatial Simulations

The aim of this paper is to outline fundamental concepts and principles of the Agent-Based Modelling (ABM) paradigm, with particular reference to the development of geospatial simulations. The paper begins with a brief definition of modelling, followed by a classification of model types, and a comment regarding a shift (in certain circumstances) towards modelling systems at the individual-level. In particular, automata approaches (e.g. Cellular Automata, CA, and ABM) have been particularly popular, with ABM moving to the fore. A definition of agents and agent-based models is given; identifying their advantages and disadvantages, especially in relation to geospatial modelling. The potential use of agent-based models is discussed, and how-to instructions for developing an agent-based model are provided. Types of simulation / modelling systems available for ABM are defined, supplemented with criteria to consider before choosing a particular system for a modelling endeavour. Information pertaining to a selection of simulation / modelling systems (Swarm, MASON, Repast, StarLogo, NetLogo, OBEUS, AgentSheets and AnyLogic) is provided, categorised by their licensing policy (open source, shareware / freeware and proprietary systems). The evaluation (i.e. verification, calibration, validation and analysis) of agent-based models and their output is examined, and noteworthy applications are discussed.Geographical Information Systems (GIS) are a particularly useful medium for representing model input and output of a geospatial nature. However, GIS are not well suited to dynamic modelling (e.g. ABM). In particular, problems of representing time and change within GIS are highlighted. Consequently, this paper explores the opportunity of linking (through coupling or integration / embedding) a GIS with a simulation / modelling system purposely built, and therefore better suited to supporting the requirements of ABM. This paper concludes with a synthesis of the discussion that has proceeded. The aim of this paper is to outline fundamental concepts and principles of the Agent-Based Modelling (ABM) paradigm, with particular reference to the development of geospatial simulations. The paper begins with a brief definition of modelling, followed by a classification of model types, and a comment regarding a shift (in certain circumstances) towards modelling systems at the individual-level. In particular, automata approaches (e.g. Cellular Automata, CA, and ABM) have been particularly popular, with ABM moving to the fore. A definition of agents and agent-based models is given; identifying their advantages and disadvantages, especially in relation to geospatial modelling. The potential use of agent-based models is discussed, and how-to instructions for developing an agent-based model are provided. Types of simulation / modelling systems available for ABM are defined, supplemented with criteria to consider before choosing a particular system for a modelling endeavour. Information pertaining to a selection of simulation / modelling systems (Swarm, MASON, Repast, StarLogo, NetLogo, OBEUS, AgentSheets and AnyLogic) is provided, categorised by their licensing policy (open source, shareware / freeware and proprietary systems). The evaluation (i.e. verification, calibration, validation and analysis) of agent-based models and their output is examined, and noteworthy applications are discussed.Geographical Information Systems (GIS) are a particularly useful medium for representing model input and output of a geospatial nature. However, GIS are not well suited to dynamic modelling (e.g. ABM). In particular, problems of representing time and change within GIS are highlighted. Consequently, this paper explores the opportunity of linking (through coupling or integration / embedding) a GIS with a simulation / modelling system purposely built, and therefore better suited to supporting the requirements of ABM. This paper concludes with a synthesis of the discussion that has proceeded

Uncertainty and Error in Combat Modeling, Simulation, and Analysis

Due to the infrequent and competitive nature of combat, several challenges present themselves when developing a predictive simulation. First, there is limited data with which to validate such analysis tools. Secondly, there are many aspects of combat modeling that are highly uncertain and not knowable. This research develops a comprehensive set of techniques for the treatment of uncertainty and error in combat modeling and simulation analysis. First, Evidence Theory is demonstrated as a framework for representing epistemic uncertainty in combat modeling output. Next, a novel method for sensitivity analysis of uncertainty in Evidence Theory is developed. This sensitivity analysis method generates marginal cumulative plausibility functions (CPFs) and cumulative belief functions (CBFs) and prioritizes the contribution of each factor by the Wasserstein distance (also known as the Kantorovich or Earth Movers distance) between the CBF and CPF. Using this method, a rank ordering of the simulation input factors can be produced with respect to uncertainty. Lastly, a procedure for prioritizing the impact of modeling choices on simulation output uncertainty in settings where multiple models are employed is developed. This analysis provides insight into the overall sensitivities of the system with respect to multiple modeling choices

Game Theory and Prescriptive Analytics for Naval Wargaming Battle Management Aids

NPS NRP Technical ReportThe Navy is taking advantage of advances in computational technologies and data analytic methods to automate and enhance tactical decisions and support warfighters in highly complex combat environments. Novel automated techniques offer opportunities to support the tactical warfighter through enhanced situational awareness, automated reasoning and problem-solving, and faster decision timelines. This study will investigate how game theory and prescriptive analytics methods can be used to develop real-time wargaming capabilities to support warfighters in their ability to explore and evaluate the possible consequences of different tactical COAs to improve tactical missions. This study will develop a conceptual design of a real-time tactical wargaming capability. This study will explore data analytic methods including game theory, prescriptive analytics, and artificial intelligence (AI) to evaluate their potential to support real-time wargaming.N2/N6 - Information WarfareThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

A methodology for the quantification of doctrine and materiel approaches in a capability-based assessment

Due to the complexities of modern military operations and the technologies employed on today's military systems, acquisition costs and development times are becoming increasingly large. Meanwhile, the transformation of the global security environment is driving the U.S. military's own transformation. In order to meet the required capabilities of the next generation without buying prohibitively costly new systems, it is necessary for the military to evolve across the spectrum of doctrine, organization, training, materiel, leadership and education, personnel, and facilities (DOTMLPF). However, the methods for analyzing DOTMLPF approaches within the early acquisition phase of a capability-based assessment (CBA) are not as well established as the traditional technology design techniques. This makes it difficult for decision makers to decide if investments should be made in materiel or non-materiel solutions. This research develops an agent-based constructive simulation to quantitatively assess doctrine alongside materiel approaches. Additionally, life-cycle cost techniques are provided to enable a cost-effectiveness trade. These techniques are wrapped together in a decision-making environment that brings crucial information forward so informed and appropriate acquisition choices can be made. The methodology is tested on a future unmanned aerial vehicle design problem. Through the implementation of this quantitative methodology on the proof-of-concept study, it is shown that doctrinal changes including fleet composition, asset allocation, and patrol pattern were capable of dramatic improvements in system effectiveness at a much lower cost than the incorporation of candidate technologies. Additionally, this methodology was able to quantify the precise nature of strong doctrine-doctrine and doctrine-technology interactions which have been observed only qualitatively throughout military history. This dissertation outlines the methodology and demonstrates how potential approaches to capability-gaps can be identified with respect to effectiveness, cost, and time. When implemented, this methodology offers the opportunity to achieve system capabilities in a new way, improve the design of acquisition programs, and field the right combination of ways and means to address future challenges to national security.Ph.D.Committee Chair: Mavris, Dimitri; Committee Member: Bishop, Carlee; Committee Member: Brown, David; Committee Member: Costello, Mark; Committee Member: Schrage, Danie

Mao with Smart Phones and Internet? A Comparison of Classic Guerrilla Warfare with Fourth and Fifth Generation Warfare Using an Agent-Based Model for Simulation

Fourth Generation Warfare (4GW) theory shares many characteristics of classical guerrilla warfare (CGW) theory in security studies literature. Proponents claim that 4GW is a revolution in war that overturns traditional measures of military power, while critics counter that 4GW is simply CGW in an updated context. Another group posits Fifth Generation Warfare (5GW), which adds additional information-age technologies and uses “any and all means,” (military and extra-military) to attack both the enemy’s will and capability to resist. The irregular subset of 5GW strategies appear to be an extension of 4GW with the addition of advanced information-age technologies: mobile phones and internet spreading propaganda instantly to friendly groups as well as national and trans-national enemies, while unconventional tactics such as suicide bombings and terrorist actions attempt to drain the will of opponents to continue the fight. The CGW and 4/5GW strategies are modeled in an agent-based simulation to evaluate similarities and differences in speed to victory, territory controlled, and the identity of the winning side. Emergent behaviors are compared with historical data. Fourth Generation Warfare (4GW) as conceptualized by numerous military scholars shares many characteristics of guerrilla tactics in the classical military literature of Sun Tzu, Wellington, Clausewitz, Mao, and Giap. Proponents of 4GW claim that its development has significantly altered the ratio of strength of industrialized and guerrilla forces, and thus the likelihood of weaker forces (as measured in previous military contexts) prevailing against forces assessed by traditional measures as stronger. Critics point to a lack of intellectual rigor in defining the salient characteristics of 4GW and charge that it is simply a re-statement of classical guerrilla war (CGW) tactics, albeit with improved communications and propaganda capabilities in a social media cultural context. This research models CGW and 4GW in conjunction with the irregular subset of 5GW in an agent-based simulation using NetLogo software (Wilensky, 1999) in order to explore differences in time and probability of victory and increased area of territory controlled by 4GW and irregular 5GW forces. These forces are then pitted against their respective industrial-age and information-age opponents. Emergent behaviors offer insights into the similarities and differences of CGW. The outputs are then compared to historical data to help answer the question of whether 4/5GW comprise a significant military revolution that threatens to upend traditional measures of military superiority, or they are merely an adaptation of old tactics to a new context. The results generally favored the rebels in both CGW and 4/5GW scenarios. Increasing Red Communications capability in the 4/5GW scenario overall increased Red Territory controlled as compared to the CGW scenario. However, increasing Blue Communications capability also increased Red Territory gained in both models. This could be interpreted that an overall increase in communications capabilities leads to more aggressive tactics and more engagements for both sides. Blue and Red communications in the 4/5GW scenarios are also associated with a decrease in both Red and Blue time to victory, indicating that the pace of engagements is accelerated in the 4/5GW scenarios. Finally, the model comparing identity of victor after 10 years produced mixed results. An increase in Red Communications was associated with a decrease in the log-odds of Blue Victory after 10 years in 4/5GW model, as expected. However, an increase Blue Communications also appeared to be associated with an increase in the log-odds of Red Victory in the 4/5GW model, a somewhat contradictory result. The addition of 21st century technologies seemed to change the overall dynamic compared to CGW only in specific cases, and usually only marginally. The research project was purposefully designed so that the 4/5GW capabilities would be additions to a basic model of guerrilla warfare. There is danger that these additions were simply insufficient in modeling the true extent of the differences between the two concepts of war, and that 4/5GW tactics are, in fact, revolutionary and not evolutionary. Further study is required to answer the question conclusively