Acquisition Management for System of Systems: Requirement Evolution and Acquisition Strategy Planning

Naval Postgraduate School Acquisition Research Progra

Multi-objective combinatorial optimization problems in transportation and defense systems

Multi-objective Optimization problems arise in many applications; hence, solving them efficiently is important for decision makers. A common procedure to solve such problems is to generate the exact set of Pareto efficient solutions. However, if the problem is combinatorial, generating the exact set of Pareto efficient solutions can be challenging. This dissertation is dedicated to Multi-objective Combinatorial Optimization problems and their applications in system of systems architecting and railroad track inspection scheduling. In particular, multi-objective system of systems architecting problems with system flexibility and performance improvement funds have been investigated. Efficient solution methods are proposed and evaluated for not only the system of systems architecting problems, but also a generic multi-objective set covering problem. Additionally, a bi-objective track inspection scheduling problem is introduced for an automated ultrasonic inspection vehicle. Exact and approximation methods are discussed for this bi-objective track inspection scheduling problem --Abstract, page iii

Program Management for Large Scale Engineering Programs

The goal of this whitepaper is to summarize the LAI research that applies to program management. The context of most of the research discussed in this whitepaper are large-scale engineering programs, particularly in the aerospace & defense sector. The main objective is to make a large number of LAI publications – around 120 – accessible to industry practitioners by grouping them along major program management activities. Our goal is to provide starting points for program managers, program management staff and system engineers to explore the knowledge accumulated by LAI and discover new thoughts and practical guidance for their everyday challenges. The whitepaper begins by introducing the challenges of programs in section 4, proceeds to define program management in section 5 and then gives an overview of existing program management frameworks in section 6. In section 7, we introduce a new program management framework that is tailored towards describing the early program management phases – up to the start of production. This framework is used in section 8 to summarize the relevant LAI research

LAI Whitepaper Series: “Lean Product Development for Practitioners”: Program Management for Large Scale Engineering Programs

The whitepaper begins by introducing the challenges of programs in section 4, proceeds to define program management in section 5 and then gives an overview of existing program management frameworks in section 6. In section 7, we introduce a new program management framework that is tailored towards describing the early program management phases – up to the start of production. This framework is used in section 8 to summarize the relevant LAI research

Workshop Report Air Force/LAI Workshop on Systems Engineering for Robustness

Workshop repor

Application of computational intelligence to explore and analyze system architecture and design alternatives

Systems Engineering involves the development or improvement of a system or process from effective need to a final value-added solution. Rapid advances in technology have led to development of sophisticated and complex sensor-enabled, remote, and highly networked cyber-technical systems. These complex modern systems present several challenges for systems engineers including: increased complexity associated with integration and emergent behavior, multiple and competing design metrics, and an expansive design parameter solution space. This research extends the existing knowledge base on multi-objective system design through the creation of a framework to explore and analyze system design alternatives employing computational intelligence. The first research contribution is a hybrid fuzzy-EA model that facilitates the exploration and analysis of possible SoS configurations. The second contribution is a hybrid neural network-EA in which the EA explores, analyzes, and evolves the neural network architecture and weights. The third contribution is a multi-objective EA that examines potential installation (i.e. system) infrastructure repair strategies. The final contribution is the introduction of a hierarchical multi-objective evolutionary algorithm (MOEA) framework with a feedback mechanism to evolve and simultaneously evaluate competing subsystem and system level performance objectives. Systems architects and engineers can utilize the frameworks and approaches developed in this research to more efficiently explore and analyze complex system design alternatives --Abstract, page iv

A System-of-Systems Approach to Enterprise Analytics Design: Acquisition Support in the Age of Machine Learning and Artificial Intelligence

Acquisition Research Program Sponsored Report SeriesSponsored Acquisition Research & Technical ReportsSystem-of-Systems (SoS) capability emerges from the collaboration of multiple systems, which are acquired from independent organizations. Even though the systems contribute to and benefit from the larger SoS, the data analytics and decision-making about the independent system is rarely shared across the SoS stakeholders. The objective of this work is to identify how the sharing of datasets and the corresponding analytics among SoS stakeholders can lead to an improved SoS capability. Our objective is to characterize how the sharing of connected data sets may lead to deployment of different predictive (predicting an outcome from data) and prescriptive (determining a preferred strategy) analytics and lead to better decision outcomes at the SoS level. We build and demonstrate a framework for this objective based on extensive literature review and generating appropriate predictive and prescriptive methodologies that can be used for SoS analysis: Additionally, we propose to utilize machine learning techniques to predict the SoS capability achievable by sharing pertinent datasets and to prescribe the information links between systems to enable this sharing. Two case studies demonstrate the use of the framework and prospects for meeting the objective. Highlights of our study are summarized next.Approved for public release; distribution is unlimited.Approved for public release; distribution is unlimited

A Mechanism Design Approach to Bandwidth Allocation in Tactical Data Networks

The defense sector is undergoing a phase of rapid technological advancement, in the pursuit of its goal of information superiority. This goal depends on a large network of complex interconnected systems - sensors, weapons, soldiers - linked through a maze of heterogeneous networks. The sheer scale and size of these networks prompt behaviors that go beyond conglomerations of systems or `system-of-systems\u27. The lack of a central locus and disjointed, competing interests among large clusters of systems makes this characteristic of an Ultra Large Scale (ULS) system. These traits of ULS systems challenge and undermine the fundamental assumptions of today\u27s software and system engineering approaches. In the absence of a centralized controller it is likely that system users may behave opportunistically to meet their local mission requirements, rather than the objectives of the system as a whole. In these settings, methods and tools based on economics and game theory (like Mechanism Design) are likely to play an important role in achieving globally optimal behavior, when the participants behave selfishly. Against this background, this thesis explores the potential of using computational mechanisms to govern the behavior of ultra-large-scale systems and achieve an optimal allocation of constrained computational resources Our research focusses on improving the quality and accuracy of the common operating picture through the efficient allocation of bandwidth in tactical data networks among self-interested actors, who may resort to strategic behavior dictated by self-interest. This research problem presents the kind of challenges we anticipate when we have to deal with ULS systems and, by addressing this problem, we hope to develop a methodology which will be applicable for ULS system of the future. We build upon the previous works which investigate the application of auction-based mechanism design to dynamic, performance-critical and resource-constrained systems of interest to the defense community. In this thesis, we consider a scenario where a number of military platforms have been tasked with the goal of detecting and tracking targets. The sensors onboard a military platform have a partial and inaccurate view of the operating picture and need to make use of data transmitted from neighboring sensors in order to improve the accuracy of their own measurements. The communication takes place over tactical data networks with scarce bandwidth. The problem is compounded by the possibility that the local goals of military platforms might not be aligned with the global system goal. Such a scenario might occur in multi-flag, multi-platform military exercises, where the military commanders of each platform are more concerned with the well-being of their own platform over others. Therefore there is a need to design a mechanism that efficiently allocates the flow of data within the network to ensure that the resulting global performance maximizes the information gain of the entire system, despite the self-interested actions of the individual actors. We propose a two-stage mechanism based on modified strictly-proper scoring rules, with unknown costs, whereby multiple sensor platforms can provide estimates of limited precisions and the center does not have to rely on knowledge of the actual outcome when calculating payments. In particular, our work emphasizes the importance of applying robust optimization techniques to deal with the uncertainty in the operating environment. We apply our robust optimization - based scoring rules algorithm to an agent-based model framework of the combat tactical data network, and analyze the results obtained. Through the work we hope to demonstrate how mechanism design, perched at the intersection of game theory and microeconomics, is aptly suited to address one set of challenges of the ULS system paradigm - challenges not amenable to traditional system engineering approaches

A System-of-Systems Approach to Enterprise Analytics Design: Acquisition Support in the Age of Machine Learning and Artificial Intelligence

Excerpt from the Proceedings of the Nineteenth Annual Acquisition Research SymposiumSystem-of-Systems (SoS) capability emerges from the collaboration of multiple systems, which are acquired from independent organizations. Even though the systems contribute to and benefit from the larger SoS, the data analytics and decision-making about the independent system is rarely shared across the SoS stakeholders. The objective of the research presented in this paper is to identify how the sharing of datasets and the corresponding analytics among SoS stakeholders can lead to an improved SoS capability. Our objective is to characterize how appropriate use of data sets may lead to deployment of different predictive (predicting an outcome from data) and prescriptive (determining a preferred strategy) analytics and lead to better decision outcomes at the SoS level. We build and demonstrate a framework for this objective, based on extensive literature review, which utilizes appropriate predictive and prescriptive methodologies for SoS analysis. Additionally, we propose to utilize machine learning techniques to predict the achievable SoS capability and identify sources of uncertainty derived by sharing partial datasets. A case study demonstrates the use of the framework and prospects for future improvements.Approved for public release; distribution is unlimited

A System-of-Systems Approach to Enterprise Analytics Design: Acquisition Support in the Age of Machine Learning and Artificial Intelligence

Excerpt from the Proceedings of the Nineteenth Annual Acquisition Research SymposiumSystem-of-Systems (SoS) capability emerges from the collaboration of multiple systems, which are acquired from independent organizations. Even though the systems contribute to and benefit from the larger SoS, the data analytics and decision-making about the independent system is rarely shared across the SoS stakeholders. The objective of the research presented in this paper is to identify how the sharing of datasets and the corresponding analytics among SoS stakeholders can lead to an improved SoS capability. Our objective is to characterize how appropriate use of data sets may lead to deployment of different predictive (predicting an outcome from data) and prescriptive (determining a preferred strategy) analytics and lead to better decision outcomes at the SoS level. We build and demonstrate a framework for this objective, based on extensive literature review, which utilizes appropriate predictive and prescriptive methodologies for SoS analysis. Additionally, we propose to utilize machine learning techniques to predict the achievable SoS capability and identify sources of uncertainty derived by sharing partial datasets. A case study demonstrates the use of the framework and prospects for future improvements.Approved for public release; distribution is unlimited