Training Intelligent Red Team Agents Via Reinforcement Deep Learning

NPS NRP Executive SummaryWargames are an essential tool for education, training and formulation of strategy. They are especially important in the evaluation of threats from, and strategies against, trained adversaries who present significant risk to friendly forces. We propose to develop a wargame adversary trained to defeat the current strategy of friendly forces, thereby allowing the evaluation of alternate strategies against an intelligent, simulated opponent. We will investigate the use of deep neural network (DNN) algorithms to solve a constrained stochastic reward-collecting path problem. Agents from a friendly (blue) team and an adversarial (red) team will be placed within a discrete environment. The blue team will be challenged to obtain a reward by achieving a fixed goal using a pre-determined strategy. Then, reinforcement learning will be used to train the red team to overcome the blue team's current strategy. Having thus trained a competent red team, the blue team's strategy can be altered to evaluate the efficacy of new strategies. This research will seek to evaluate the ability of different DNN algorithms to train the red team against various blue team strategies, in terms of both efficacy and efficiency, and the resiliency of the trained red team to subsequent changes in blue team strategy. We anticipate the results of this research to be summarized in a research poster and executive summary, in addition to a presentation and full technical report deliverable to the Topic Sponsor.Marine Corps Systems Command (MARCORSYSCOM)Marine Corps Systems Command (MARCORSYSCOM)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Training Intelligent Red Team Agents Via Reinforcement Deep Learning

NPS NRP Technical ReportWargames are an essential tool for education, training and formulation of strategy. They are especially important in the evaluation of threats from, and strategies against, trained adversaries who present significant risk to friendly forces. We propose to develop a wargame adversary trained to defeat the current strategy of friendly forces, thereby allowing the evaluation of alternate strategies against an intelligent, simulated opponent. We will investigate the use of deep neural network (DNN) algorithms to solve a constrained stochastic reward-collecting path problem. Agents from a friendly (blue) team and an adversarial (red) team will be placed within a discrete environment. The blue team will be challenged to obtain a reward by achieving a fixed goal using a pre-determined strategy. Then, reinforcement learning will be used to train the red team to overcome the blue team's current strategy. Having thus trained a competent red team, the blue team's strategy can be altered to evaluate the efficacy of new strategies. This research will seek to evaluate the ability of different DNN algorithms to train the red team against various blue team strategies, in terms of both efficacy and efficiency, and the resiliency of the trained red team to subsequent changes in blue team strategy. We anticipate the results of this research to be summarized in a research poster and executive summary, in addition to a presentation and full technical report deliverable to the Topic Sponsor.Marine Corps Systems Command (MARCORSYSCOM)Marine Corps Systems Command (MARCORSYSCOM)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Naval Research Program 2021 Annual Report

NPS NRP Annual ReportThe Naval Postgraduate School (NPS) Naval Research Program (NRP) is funded by the Chief of Naval Operations and supports research projects for the Navy and Marine Corps. The NPS NRP serves as a launch-point for new initiatives which posture naval forces to meet current and future operational warfighter challenges. NRP research projects are led by individual research teams that conduct research and through which NPS expertise is developed and maintained. The primary mechanism for obtaining NPS NRP support is through participation at NPS Naval Research Working Group (NRWG) meetings that bring together fleet topic sponsors, NPS faculty members, and students to discuss potential research topics and initiatives.Chief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.