NPS NRP Executive SummaryAviation Depot Maintenance Throughput OptimizationN8 - Integration of Capabilities & ResourcesThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

MacKinnon, Douglas J.

Salmeron, Javier

English

NPS NRP Executive SummaryAviation Depot Maintenance Throughput OptimizationApproved for public release. Distribution is unlimited. This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)N8 - Integration of Capabilities & Resource

Calhoun, Institutional Archive of the Naval Postgraduate School

Calhoun: The NPS Institutional ArchiveDSpace RepositoryFaculty and Researchers Faculty and Researchers' Publications2019-12Aviation Depot Maintenance Throughput OptimizationSalmeron, Javier; MacKinnon, Douglas J.Monterey, California: Naval Postgraduate Schoolhttp://hdl.handle.net/10945/69922This publication is a work of the U.S. Government as defined in Title 17, UnitedStates Code, Section 101. Copyright protection is not available for this work in theUnited States.Downloaded from NPS Archive: CalhounNPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 1 of 6      MONTEREY, CALIFORNIA F/A-18 PERFORMANCE/PRICING MODEL ANALYSIS  Report Type: Final Report Period of Performance: 01/01/2019-12/31/2019  Project PI: Douglas MacKinnon, Ph.D. Project Co-PI: Javier Salmeron, Ph.D. Student Participation: Kyle Ellis, Maj USMC  Prepared for: Topic Sponsor: N81-Assessment Division Research POC Name: CDR Robert Alexander, USN  Research POC Contact Information: Robert.alexander@navy.mil      Distro A – Approved for public release; distribution is unlimited.NPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 1 of 6  EXECUTIVE SUMMARY Project Summary  In 2004, the Naval Aviation Enterprise transitioned to a system of Performance/Pricing Models (P/PM) in an effort to support decision makers and gain efficiency in its funding activities. P/PMs consist of two key elements: driver-based performance management, which identifies the organization’s desired outcomes and the drivers that have quantifiable impacts on them, and predictive analytics, which identify high-leverage drivers and forecast performance.  The F/A-18 Super Hornet has been the workhorse of the Navy and Marine Corps' tactical aviation squadrons for more than 30 years. However, over the last decade, the amount of hours flown in a mission capable (MC) status has steadily declined despite increases in funding and the utilization of the P/PMs to better allocate that funding. This work, in conjunction with efforts by the Chief of Naval Operations Assessment Division (N81) to assess the causes of this readiness decay, consists of a mixed methodology analysis of the following F/A-18 P/PMs: Flight Hour Program (FHP), Spares Model, and the Depot Readiness Assessment Model (DRAM) suite—including the Engine Depot Readiness Assessment Model (EDRAM) and the Airframe Depot Readiness Assessment Model (ADRAM). The seminal product of this research is a logical framework describing the relationships between the P/PMs with a particular focus on the model drivers determined as key to model performance. The framework serves as a foundation for future work in understanding the complexities of the Navy's aviation problem set. This research also illuminates the extent to which human decisions shape model outputs.  Keywords: F/A-18, Super Hornet, aviation, readiness, performance/pricing model, P/PM, Chief of Naval Operations Assessment Division, N81, Flight Hour Program, FHP, Spares Model, Depot Readiness Assessment Model, DRAM, Airframe Depot Readiness Assessment Model, ADRAM, Engine Depot Readiness Assessment Model, EDRAM, depot, Fleet Readiness Center, modeling and simulations, MS, mission capable, MC   Background The Department of Defense (DoD) is nearly seven times larger than the next largest government agency in terms of annual budget request (Headquarters USAF, 2016). The DoD therefore requires a more in-depth budgeting process than the single-year outlooks of smaller departments. Since 1960, the planning, programing, budgeting, and execution (PPBE) process has been the DoD’s primary method of allocating resources and is guided by the Quadrennial Defense Review, force development guidance, program guidance, and budget guidance (DoD Directive 7045.14, 2013).  The long-term prediction of allocations necessitates a systematic approach to requirement analysis. In the Navy, this responsibility falls largely on the Deputy Chief of Naval Operations for Integration of NPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 2 of 6  Capabilities and Resources who “exercises centralized supervision and coordination for the Navy’s capability study analysis and assessments, allocation and integration of the Navy’s resources and requirements in the PPBE, and determination of technical guidance” (Chief of Naval Operations [CNO], 2018). Subordinate to N8, N81 is responsible for the assessment of Navy’s warfighting capability including evaluating the Navy’s ability to counter current and future threats, the Program Objective Memorandum (POM) in the areas of manpower, fleet readiness, and logistics capability delivery (wholeness), and other areas of importance to the CNO such as tactical aviation readiness.  Part of N81’s assessment of the POM is the accreditation of models used during the budgeting phase by Navy resource sponsors of programs with operating appropriations of $50 million or more (CNO, 2006). These models are analytical tools used to relate budgeted costs to observed performance levels. However, during recent POM development cycles, performance levels have not met forecasted performance for many programs and N81 has been tasked with identifying the potential causes. N81 is attempting to determine if current level-of-effort thresholds are sufficient to accurately determine the appropriate level of accreditation, the cause of variance between P/PM-informed budget projections and subsequent execution, the accuracy of external P/PM inputs, and internal P/PM algorithms and cost estimation relationships. This thesis supports the efforts of N81 and focuses on the subset of models that are directly tied to tactical aviation, particularly the F/A-18 Super Hornet.  Findings and Conclusions This research consists of a qualitative analysis of all inputs and outputs of four aviation-related P/PMs.  It also utilizes quantitative methods such as regression analysis to attempt to link Super Hornet readiness rate to varying input or output parameters.  This provides N81 a robust framework to build a conceptually integrated model of the interdependencies of these four P/PMs. Our research is exploratory in nature and intended to be a foundation of future work. As such, most of the research questions remain unanswered.   Aviation readiness including, but not limited to, the Super Hornet cannot be described as simply a maintenance problem or a funding problem. It is a systems problem and must be addressed with a system dynamics approach. Unfortunately, the P/PMs studied in this research represent only a very limited piece of the system. Regardless of how well each individual model works in its own subsystem, the overall system cannot be adequately understood without modeling it in its entirety. This is not a novel conclusion; in fact, this realization was the basis for N81 to commission this work.   The human hand is prevalent throughout the system. It is evident beginning with the annual data calls in which stakeholders extract, format, and submit the execution and costing data used in the FHP to the EDRAM model manager using his “significant subject matter expertise regarding the realities of Navy aircraft repair as it is actually performed” to adjust the model inputs and constraints (Pandolfini & Phipps, 2015, p. 6). Furthermore, due to the PPBE process, planners make funding decisions based on multiple factors including changing operational environments, budgetary constraints, and leadership NPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 3 of 6  priorities. A systems dynamics approach is needed not only to model the interactions between P/PMs, but also to remove as much of the human element as possible.  Active duty positions with direct interaction with or oversight of the P/PMs are filled with mid-level officers who generally change positions every two to three years. By this point in their careers, these officers are adept at learning skills needed for new positions and can achieve a basic level of proficiency relatively quickly. Rules of thumb, standard operating procedures, and standardized languages help the officers transition, but basic skills do not give them a full understanding of the intricacies of a singular model, much less the relationships between related models.  Instead, this requires tacit knowledge, described by Nissen as knowledge which is “gained principally through experience and accumulated over time, organizational capabilities based upon tacit knowledge are difficult to imitate” (Nissen, 2006). P/PM model managers have extensive understanding of the feasibility of input data for the model, but relying so heavily on a model manager who eventually will vacate the position poses risks.  These risks are multiplied by what Theorgood called a “dearth of proper documentation of modeling and simulations (M&S) development efforts, [validation, verification, and accreditation] procedures, and most notable the configuration management plan” (Theorgood, 2005). Indeed, documentation obtained for this study was often incomplete. For example, P/PMs are required to be re-accredited every three years; however, a partial 2011 verification and validation (V&V) submission and the full 2015 accreditation report was the only documentation available for this study.   The Navy can mitigate risk in two ways: first, modifications to the P/PMs algorithms and business rules can reduce the reliance on managers and the knowledge they possess. In the EDRAM example above, the manager’s extensive knowledge of the feasibility of input data should be captured as constraints, thereby reducing the reliance on hard-to-reproduce tacit knowledge. Second, as suggested by Theorgood, a robust system for depositing and retaining M&S documentation should be developed. All development documentation, V&V submissions, accreditation reports, input data, and model projections should be retained and made readily assessable not only to the model owners and managers, but also to analytical activities at N81.   Recommendations for Further Research The next logical step for follow-on research is to conduct an in-depth, quantitative analysis to validate the assumptions and conclusions of this work, while also addressing its shortcomings. Researchers should focus on the entire decision supply chain, that is, from the depot to the flight line, in order to gain a sense of how the readiness goals that govern the depots represent actual utilization in the fleet. Special attention should also be given to integrating the DRAM suite and FHP.  This research has mapped the logical connections between three of the four models examined, but further work is needed to fully describe these interactions. This will not be a purely quantitative analysis, as there NPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 4 of 6  is still qualitative research needed to conceptualize exactly how the models interact. For example, the FHP produces a flight hour requirement in the OP-20 format. Those hours are used to generate the budgeted flight hours and the flight line entitlement, which become key drivers to the EDRAM and ADRAM, respectively. However, there is not a direct path from the FHP to these inputs. In fact, the OP-20 report passes through several iterations which include deletions and other adjustments; the same is true for the DRAM inputs. This process comprises both automated and manual adjustments whose logic and human-based rules need to be specified in as much detail as possible before any quantitative assessment can begin.   Similar investigation of the sparing models is also required. As mentioned above, the lack of model documentation severely hampered efforts to include these models in this work, but this does not diminish the importance of spare inventory to MC rates. The coming integration of the Naval Aviation Readiness-Based Sparing model and the fact that it was developed at the Naval Postgraduate School at the behest of the Naval Supply Systems Command means that technical documentation is readily available, as opposed to that from some of the legacy, proprietary models.   Once all four models and their interactions are sufficiently understood, the research can expand beyond the F/A-18 Super Hornet. Each type/model/series in the naval aviation inventory represents unique challenges in terms of maintaining MC aircraft. However, similar to the way the P/PMs inform funding decisions for all naval aircraft, the supply and maintenance pipelines are shared in part, if not wholly, across the enterprise. Therefore, the relationships described in this research, as well as those uncovered in follow-on work, will be translatable across the fleet. The techniques used may also lend themselves toward the evaluation of funding decision models used for the surface and subsurface Navy.  Finally, in September 2018, the Boston Consulting Group began a four-week assessment of Fleet Readiness Center (FRC) Southwest operations and coordination with Naval Supply Systems Command aboard Naval Air Station North Island in San Diego, CA. The aim of that arrangement was to launch a diagnostic pilot to highlight operational pain points-(sic), analyze overall Super Hornet readiness and top degraders, and prioritize key issues and opportunities with direct impact and linkages to readiness improvement. While this effort is focusing solely on the FRC, the Navy should consider a similar consulting arrangement to provide insight PPBE activities for aviation. This could bolster the findings of follow-on research described above.  References Chief of Naval Operations. (2006, October 26). OPNAV Performance/Pricing Models policy and procedures (OPNAVINST 5200.35). Washington, DC: Author.  Chief of Naval Operations. (2018, April 23). Mission, functions, and tasks of the office of the Chief of Naval Operations (OPNAVINST 5450.352A). Washington, DC: Author.  NPS NRP Executive Summary  Title: F/A-18 Performance/Pricing Model Analysis  Report Date: 30Sep2019 Project Number: NPS-19-N143-A  Naval Postgraduate School: Graduate School of Information Science (GSOIS)/Operations Research (OR) Page 5 of 6  Headquarters, U.S. Air Force. (2016). Planning, programming, budgeting and execution system training program. Washington, DC: U.S. Government Printing Office.   Mattis, J. (September 17, 2018). NDS implementation—mission capability of critical aviation platforms [Memorandum] Washington, DC: Department of Defense. Retrieved from https://www.documentcloud.org/documents/4999250-390505410-180917-Mattis-Memo.html#document/p1  Nissen, M. (2006). Harnessing knowledge dynamics principled organizational knowing & learning. Retrieved from https://doi.org/10.4018/978-1-59140-77  Pandolfini, P., & Phipps, L. (2015). Accreditation evaluation report for the Engine Depot Readiness Assessment Model. Laurel, MD: Johns Hopkins University Applied Physics Lab.  Secretary of the Navy (SECNAV) (1999), Verification, validation, and accreditation (VV&A) of models and simulations (SECNAV INST 5200.40) Washington DC.  Theorgood, P. (2005). Model process validation: an analysis of performance-based pricing programs (Master’s thesis). Retrieved from http://hdl.handle.net/10945/1911    Acronyms  ADRAM Airframe Depot Readiness Assessment Model CNO  Chief of Naval Operations DoD  Department of Defense DRAM  Depot Readiness Assessment Model EDRAM Engine Depot Readiness Assessment Model FHP  Flight Hour Program FRC  Fleet Readiness Center M&S  Modeling and simulation MC  Mission capable OPNAV N81 Chief of Naval Operations, Assessment Division P/PM  Performance/Pricing Model POM  Program Objective Memorandum PPBE  Planning, programming, budgeting, and execution USN  United States Navy V&V  Verification and validation  

Aviation Depot Maintenance Throughput Optimization

https://calhoun.nps.edu/bitstream/handle/10945/69922/NPS-19-N143-A_NRP_ExecSum_Salmeron.pdf?sequence=1

Aviation Depot Maintenance Throughput Optimization

Abstract

Similar works

Full text

Available Versions

Calhoun, Institutional Archive of the Naval Postgraduate School