7,715 research outputs found
SCALING REINFORCEMENT LEARNING THROUGH FEUDAL MULTI-AGENT HIERARCHY
Militaries conduct wargames for training, planning, and research purposes. Artificial intelligence (AI) can improve military wargaming by reducing costs, speeding up the decision-making process, and offering new insights. Previous researchers explored using reinforcement learning (RL) for wargaming based on the successful use of RL for other human competitive games. While previous research has demonstrated that an RL agent can generate combat behavior, those experiments have been limited to small-scale wargames. This thesis investigates the feasibility and acceptability of -scaling hierarchical reinforcement learning (HRL) to support integrating AI into large military wargames. Additionally, this thesis also investigates potential complications that arise when replacing the opposing force with an intelligent agent by exploring the ways in which an intelligent agent can cause a wargame to fail. The resources required to train a feudal multi-agent hierarchy (FMH) and a standard RL agent and their effectiveness are compared in increasingly complicated wargames. While FMH fails to demonstrate the performance required for large wargames, it offers insight for future HRL research. Finally, the Department of Defense verification, validation, and accreditation process is proposed as a method to ensure that any future AI application applied to wargames are suitable.Lieutenant Colonel, United States ArmyApproved for public release. Distribution is unlimited
Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War
Algorithmic and data-driven decisions and recommendations are commonly used
in high-stakes decision-making settings such as criminal justice, medicine, and
public policy. We investigate whether it would have been possible to improve a
security assessment algorithm employed during the Vietnam War, using outcomes
measured immediately after its introduction in late 1969. This empirical
application raises several methodological challenges that frequently arise in
high-stakes algorithmic decision-making. First, before implementing a new
algorithm, it is essential to characterize and control the risk of yielding
worse outcomes than the existing algorithm. Second, the existing algorithm is
deterministic, and learning a new algorithm requires transparent extrapolation.
Third, the existing algorithm involves discrete decision tables that are common
but difficult to optimize over.
To address these challenges, we introduce the Average Conditional Risk
(ACRisk), which first quantifies the risk that a new algorithmic policy leads
to worse outcomes for subgroups of individual units and then averages this over
the distribution of subgroups. We also propose a Bayesian policy learning
framework that maximizes the posterior expected value while controlling the
posterior expected ACRisk. This framework separates the estimation of
heterogeneous treatment effects from policy optimization, enabling flexible
estimation of effects and optimization over complex policy classes. We
characterize the resulting chance-constrained optimization problem as a
constrained linear programming problem. Our analysis shows that compared to the
actual algorithm used during the Vietnam War, the learned algorithm assesses
most regions as more secure and emphasizes economic and political factors over
military factors.Comment: 40 pages, 19 figure
Survey on Evaluation Methods for Dialogue Systems
In this paper we survey the methods and concepts developed for the evaluation
of dialogue systems. Evaluation is a crucial part during the development
process. Often, dialogue systems are evaluated by means of human evaluations
and questionnaires. However, this tends to be very cost and time intensive.
Thus, much work has been put into finding methods, which allow to reduce the
involvement of human labour. In this survey, we present the main concepts and
methods. For this, we differentiate between the various classes of dialogue
systems (task-oriented dialogue systems, conversational dialogue systems, and
question-answering dialogue systems). We cover each class by introducing the
main technologies developed for the dialogue systems and then by presenting the
evaluation methods regarding this class
Deep learning for video game playing
In this article, we review recent Deep Learning advances in the context of
how they have been applied to play different types of video games such as
first-person shooters, arcade games, and real-time strategy games. We analyze
the unique requirements that different game genres pose to a deep learning
system and highlight important open challenges in the context of applying these
machine learning methods to video games, such as general game playing, dealing
with extremely large decision spaces and sparse rewards
- …