623 research outputs found
Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering
The application of artificial intelligence to simulate air-to-air combat
scenarios is attracting increasing attention. To date the high-dimensional
state and action spaces, the high complexity of situation information (such as
imperfect and filtered information, stochasticity, incomplete knowledge about
mission targets) and the nonlinear flight dynamics pose significant challenges
for accurate air combat decision-making. These challenges are exacerbated when
multiple heterogeneous agents are involved. We propose a hierarchical
multi-agent reinforcement learning framework for air-to-air combat with
multiple heterogeneous agents. In our framework, the decision-making process is
divided into two stages of abstraction, where heterogeneous low-level policies
control the action of individual units, and a high-level commander policy
issues macro commands given the overall mission targets. Low-level policies are
trained for accurate unit combat control. Their training is organized in a
learning curriculum with increasingly complex training scenarios and
league-based self-play. The commander policy is trained on mission targets
given pre-trained low-level policies. The empirical validation advocates the
advantages of our design choices.Comment: 22nd International Conference on Machine Learning and Applications
(ICMLA 23
Command Agent Belief Architecture to Support Commander Decision Making in Military Simulation
In the war, military conflicts have many aspects that are consistent with complexity theory e.g., the higher commander’s decision is directed at animate entity that react under hierarchical and self-organised structure in decentralised command and control for the collectivist dynamism of decomposed elements due to nonlinear complexity of warfare on the battlefield. Agent technology have been found to be suitable for modelling tactical behaviour of entities at multiple level of resolution under hierarchical command and control (C2) structure and provide a powerful abstraction mechanism required for designing simulations of complex and dynamic battlefield situations. Intelligent agents can potentially reduce the overhead on such experiments and studies. Command agents, plan how to carry out the operation and assign tasks to subordinate agents. They receive information from battlefield environment and use such information to build situation awareness and also to respond to unforeseen situations. In the paper, we have proposed a mechanism for modelling tactical behaviour of an intelligent agent by which higher command level entities should be able to synthesize their beliefs derived from the lower level sub ordinates entities. This paper presents a role-based belief, desire and intention mechanism to facilitate in the representation of military hierarchy, modelling of tactical behaviour based on agent current belief, teammate’s belief propagation, and coordination issues. Higher commander can view the battlefield information at different levels of abstraction based on concept of aggregation and disaggregation and take appropriate reactive response to any unforeseen circumstances happening in battlefield
Asimovian Adaptive Agents
The goal of this research is to develop agents that are adaptive and
predictable and timely. At first blush, these three requirements seem
contradictory. For example, adaptation risks introducing undesirable side
effects, thereby making agents' behavior less predictable. Furthermore,
although formal verification can assist in ensuring behavioral predictability,
it is known to be time-consuming. Our solution to the challenge of satisfying
all three requirements is the following. Agents have finite-state automaton
plans, which are adapted online via evolutionary learning (perturbation)
operators. To ensure that critical behavioral constraints are always satisfied,
agents' plans are first formally verified. They are then reverified after every
adaptation. If reverification concludes that constraints are violated, the
plans are repaired. The main objective of this paper is to improve the
efficiency of reverification after learning, so that agents have a sufficiently
rapid response time. We present two solutions: positive results that certain
learning operators are a priori guaranteed to preserve useful classes of
behavioral assurance constraints (which implies that no reverification is
needed for these operators), and efficient incremental reverification
algorithms for those learning operators that have negative a priori results
SCALING REINFORCEMENT LEARNING THROUGH FEUDAL MULTI-AGENT HIERARCHY
Militaries conduct wargames for training, planning, and research purposes. Artificial intelligence (AI) can improve military wargaming by reducing costs, speeding up the decision-making process, and offering new insights. Previous researchers explored using reinforcement learning (RL) for wargaming based on the successful use of RL for other human competitive games. While previous research has demonstrated that an RL agent can generate combat behavior, those experiments have been limited to small-scale wargames. This thesis investigates the feasibility and acceptability of -scaling hierarchical reinforcement learning (HRL) to support integrating AI into large military wargames. Additionally, this thesis also investigates potential complications that arise when replacing the opposing force with an intelligent agent by exploring the ways in which an intelligent agent can cause a wargame to fail. The resources required to train a feudal multi-agent hierarchy (FMH) and a standard RL agent and their effectiveness are compared in increasingly complicated wargames. While FMH fails to demonstrate the performance required for large wargames, it offers insight for future HRL research. Finally, the Department of Defense verification, validation, and accreditation process is proposed as a method to ensure that any future AI application applied to wargames are suitable.Lieutenant Colonel, United States ArmyApproved for public release. Distribution is unlimited
A Review of Platforms for the Development of Agent Systems
Agent-based computing is an active field of research with the goal of
building autonomous software of hardware entities. This task is often
facilitated by the use of dedicated, specialized frameworks. For almost thirty
years, many such agent platforms have been developed. Meanwhile, some of them
have been abandoned, others continue their development and new platforms are
released. This paper presents a up-to-date review of the existing agent
platforms and also a historical perspective of this domain. It aims to serve as
a reference point for people interested in developing agent systems. This work
details the main characteristics of the included agent platforms, together with
links to specific projects where they have been used. It distinguishes between
the active platforms and those no longer under development or with unclear
status. It also classifies the agent platforms as general purpose ones, free or
commercial, and specialized ones, which can be used for particular types of
applications.Comment: 40 pages, 2 figures, 9 tables, 83 reference
- …