A Framework for Delivering Contextually Appropriate Opportunities for Warfighter Practice

Computer-based modeling and simulation has been a training staple in the military domain since the first aircraft simulators were adopted. More recently, virtual environments based on modeling, simulation and serious games, have introduced relatively low-cost, yet high value additions to the learning environment. As these virtual environments have proliferated, many researchers have investigated the relationship between theoretical foundations of learning, learner development and content delivery, and applied their findings in an attempt to bolster learning, yet performance deficiencies continue to exist. This study asserts that performance deficiencies exist in part because of insufficient contextually appropriate opportunities to practice. This work is multi-disciplinary in nature. Its foundation is modeling and simulation engineering; the use of technology to deliver training. Educational psychology and human factors concepts explain the theoretical basis for modeling and simulation as an effective training delivery agent. The study\u27s thesis is that a framework for delivering contextually appropriate opportunities for warfighter practice can be applied to discover whether modeling, simulation and game-based virtual environments have the potential to improve individual performance for learners beyond the Novice Stage (e.g., Competent Stage) of skills acquisition. Furthermore, this conceptually appropriate practice (CAP) framework can be used to assess the potential of low fidelity virtual environments to provide targeted practice and to improve individual performance, not only during training in high-fidelity virtual environments (near transfer) but also in the live environment (far transfer). To evaluate the thesis, this study investigates the relationship of technology and learning science, and features an empirical evaluation of training effectiveness afforded by delivering additional training repetitions using both low-fidelity virtual environment simulator systems and high-fidelity aircraft simulators

Reinforcement Learning for UAV Attitude Control

Autopilot systems are typically composed of an "inner loop" providing stability and control, while an "outer loop" is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.Comment: 13 pages, 9 figure

Using Computational Agents to Design Participatory Social Simulations

In social science, the role of stakeholders is increasing in the development and use of simulation models. Their participation in the design of agent-based models (ABMs) has widely been considered as an efficient solution to the validation of this particular type of model. Traditionally, "agents" (as basic model elements) have not been concerned with stakeholders directly but via designers or role-playing games (RPGs). In this paper, we intend to bridge this gap by introducing computational or software agents, implemented from an initial ABM, into a new kind of RPG, mediated by computers, so that these agents can interact with stakeholders. This interaction can help not only to elicit stakeholders' informal knowledge or unpredicted behaviours, but also to control stakeholders' focus during the games. We therefore formalize a general participatory design method using software agents, and illustrate it by describing our experience in a project aimed at developing agent-based social simulations in the field of air traffic management.Participatory Social Simulations, Agent-Based Social Simulations, Computational Agents, Role-Playing Games, Artificial Maieutics, User-Centered Design