Improving Navy MPTE Studies with Model-Driven Big Data

The goal of this research was to improve upon the ability of OPNAV N1 analysts to quickly and efficiently obtain experiment-based information from their computational models. The enhanced information will enable N1’s analysts to better support Navy leadership in resource and policy decisions that shape the future Navy and help it retain and develop its most talented Sailors. This project built on previous collaborations with N1 using data farming to enhance the information gleaned from their Navy talent management models, such as the Officer Strategic Analysis Model (OSAM) model, the Production Resource Optimization (PRO) model, and the Navy Total Force Strength Model (NTFSM). During this research period, (1) Ensign William Desousa (2015) investigated the behavior of economic inputs in NTFSM; (2) Lieutenant Peter Bazalaki (2016) used the new data farming capabilities we developed in OSAM to investigate Surface Warfare Officer (SWO) inventory across a breadth of possibilities; and (3) Lieutenant Allison Hogarth (2016) built, tested, and demonstrated a user interface in Excel that enables users of the PRO model to automatically execute a sophisticated design of experiments—the tool that enables this new capability is known as Production Resource Optimization Model With Experimental Design (PROMWED). In addition to working with the student-officers, the faculty supporting this project performed an empirical study of statistical software packages that may provide better understanding of the high-dimensional behavior of manpower models in the future (Erickson, Ankenman, & Sanchez 2016).Naval Research ProgramPrepared for Topic Sponsor: OPNAV N1; Research POC Name: Mr. Ian AndersoNPS-N16-N154-

Advancing the Application of Design of Experiments to Synthetic Theater Operations Research Model Data

NPS NRP Executive SummaryNavy leadership is interested in initiatives that can potentially increase the responsiveness of campaign analysis. Simulation-based campaign analysis is used to measure risk for investment options in how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM is a complex, time-consuming process. A simulated campaign may span months, involve scores of ships and battalions, hundreds of aircraft and installations, all executing thousands of interconnected missions involving numerous events in time and space. Creating, testing, and approving the inputs for a single design point (DP) requires a significant investment in analysts’ time and computing resources. Consequently, there are limits on the number of DPs that can be produced, executed, and analyzed during a study’s timeframe.N8 - Integration of Capabilities & ResourcesThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098).Approved for public release. Distribution is unlimited

Applying Design of Experiments (DOE) to Improve Synthetic Theater Operations Research Model (STORM) Data

NPS NRP Executive SummaryThe Navy uses simulation-based campaign analysis to help measure risk for investment options for how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM can be a complex, time-consuming process. The goal of this research is to apply design of experiment (DOE) methods in the selection and creation of design points (DPs) to minimize the number of modeling runs required for meaningful comparisons. Another objective is to understand how best DOE methods can complement traditional baseline and excursion modeling. In addition to regular reviews, the research deliverables will include: (1) A final brief and/or technical report, in addition to student theses (if applicable) (2) all findings, methods, and data used in the study; and (3) appropriate conference or journal papers related to this research.N8 - Integration of Capabilities & ResourcesThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Advancing the Application of Design of Experiments (DOE) to Synthetic Theater Operations Research Model (STORM) Data

NPS NRP Technical ReportThe Navy uses simulation-based campaign analysis to help measure risk for investment options for how best to equip, organize, supply, maintain, train, and employ our naval forces. The Synthetic Theater Operations Research Model (STORM) is a stochastic simulation model used to support campaign analysis by the U.S. Navy, Marine Corps, and Air Force. Building, testing, running, and analyzing campaign scenarios in STORM can be a complex, time-consuming process. The goal of this research is to apply Design Of Experiment (DOE) methods in the selection and creation of Design Points (DPs) to minimize the number of modeling runs required for meaningful comparisons. Another objective is to understand how best DOE methods can complement traditional baseline and excursion modeling. In addition to regular reviews, the research deliverables will include: (1) a final brief and/or technical report, in addition to student theses (if applicable); (2) all findings, methods, and data used in the study; and (3) appropriate conference or journal papers related to this research.N8 - Integration of Capabilities & ResourcesThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Sliced Full Factorial-Based Latin Hypercube Designs as a Framework for a Batch Sequential Design Algorithm

The article of record as published may be found at http://dx.doi.org/10.1080/00401706.2015.1108233When fitting complex models, such as finite element or discrete event simulations, the experiment design should exhibit desirable properties of both projectivity and orthogonality. To reduce experimental effort, sequential design strategies allow experimenters to collect data only until some measure of prediction precision is reached. In this article, we present a batch sequential experiment design method that uses sliced full factorial-based Latin hypercube designs (sFFLHDs), which are an extension to the concept of sliced orthogonal array-based Latin hypercube designs (OALHDs). At all stages of the sequential design, good univariate stratification is achieved. The structure of the FFLHDs also tends to produce uniformity in higher dimensions, especially at certain stages of the design. We show that our batch sequential design approach has good sampling and fitting qualities through both empirical studies and theoretical arguments. Supplementary materials are available online.USMC-PMMIONR/NPS CRUSE

Experimental Designs, Meta-Modeling, and Meta-learning for Mixed-Factor Systems with Large Decision Spaces

Many Air Force studies require a design and analysis process that can accommodate for the computational challenges associated with complex systems, simulations, and real-world decisions. For systems with large decision spaces and a mixture of continuous, discrete, and categorical factors, nearly orthogonal-and-balanced (NOAB) designs can be used as efficient, representative subsets of all possible design points for system evaluation, where meta-models are then fitted to act as surrogates to system outputs. The mixed-integer linear programming (MILP) formulations used to construct first-order NOAB designs are extended to solve for low correlation between second-order model terms (i.e., two-way interactions and quadratics). The resulting second-order approaches are shown to improve design performance measures for second-order model parameter estimation and prediction variance as well as for protection from bias due to model misspecification with respect to second-order terms. Further extensions are developed to construct batch sequential NOAB designs, giving experimenters more flexibility by creating multiple stages of design points using different NOAB approaches, where simultaneous construction of stages is shown to outperform design augmentation overall. To reduce cost and add analytical rigor, meta-learning frameworks are developed for accurate and efficient selection of first-order NOAB designs as well as of meta-models that approximate mixed-factor systems

Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) 2019 Annual Report

Prepared for: Dr. Brian Bingham, CRUSER DirectorThe Naval Postgraduate School (NPS) Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) provides a collaborative environment and community of interest for the advancement of unmanned systems (UxS) education and research endeavors across the Navy (USN), Marine Corps (USMC) and Department of Defense (DoD). CRUSER is a Secretary of the Navy (SECNAV) initiative to build an inclusive community of interest on the application of unmanned systems (UxS) in military and naval operations. This 2019 annual report summarizes CRUSER activities in its eighth year of operations and highlights future plans.Deputy Undersecretary of the Navy PPOIOffice of Naval Research (ONR)Approved for public release; distribution is unlimited

Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) 2019 Annual Report

Prepared for: Dr. Brian Bingham, CRUSER DirectorThe Naval Postgraduate School (NPS) Consortium for Robotics and Unmanned Systems Education and Research (CRUSER) provides a collaborative environment and community of interest for the advancement of unmanned systems (UxS) education and research endeavors across the Navy (USN), Marine Corps (USMC) and Department of Defense (DoD). CRUSER is a Secretary of the Navy (SECNAV) initiative to build an inclusive community of interest on the application of unmanned systems (UxS) in military and naval operations. This 2019 annual report summarizes CRUSER activities in its eighth year of operations and highlights future plans.Deputy Undersecretary of the Navy PPOIOffice of Naval Research (ONR)Approved for public release; distribution is unlimited

Naval Research Program 2021 Annual Report

NPS NRP Annual ReportThe Naval Postgraduate School (NPS) Naval Research Program (NRP) is funded by the Chief of Naval Operations and supports research projects for the Navy and Marine Corps. The NPS NRP serves as a launch-point for new initiatives which posture naval forces to meet current and future operational warfighter challenges. NRP research projects are led by individual research teams that conduct research and through which NPS expertise is developed and maintained. The primary mechanism for obtaining NPS NRP support is through participation at NPS Naval Research Working Group (NRWG) meetings that bring together fleet topic sponsors, NPS faculty members, and students to discuss potential research topics and initiatives.Chief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.