Enhancing the test and evaluation process: implementing agile development, test automation, and model-based systems engineering concepts

2020 Fall.Includes bibliographical references.With the growing complexity of modern systems, traditional testing methods are falling short. Test documentation suites used to verify the software for these types of large, complex systems can become bloated and unclear, leading to extremely long execution times and confusing, unmanageable test procedures. Additionally, the complexity of these systems can prevent the rapid understanding of complicated system concepts and behaviors, which is a necessary part of keeping up with the demands of modern testing efforts. Opportunities for optimization and innovation exist within the Test and Evaluation (T&E) domain, evidenced by the emergence of automated testing frameworks and iterative testing methodologies. Further opportunities lie with the directed expansion and application of related concepts such as Model-Based Systems Engineering (MBSE). This dissertation documents the development and implementation of three methods of enhancing the T&E field when applied to a real-world project. First, the development methodology of the system was transitioned from Waterfall to Agile, providing a more responsive approach when creating new features. Second, the Test Automation Framework (TAF) was developed, enabling the automatic execution of test procedures. Third, a method of test documentation using the Systems Modeling Language (SysML) was created, adopting concepts from MBSE to standardize the planning and analysis of test procedures. This dissertation provides the results of applying the three concepts to the development process of an airborne Electronic Warfare Management System (EWMS), which interfaces with onboard and offboard aircraft systems to receive and process the threat environment, providing the pilot or crew with a response solution for the protection of the aircraft. This system is representative of a traditional, long-term aerospace project that has been constantly upgraded over its lifetime. Over a two-year period, this new process produced a number of qualitative and quantitative results, including improving the quality and organization of the test documentation suite, reducing the minimum time to execute the test procedures, enabling the earlier identification of defects, and increasing the overall quality of the system under test. The application of these concepts generated many lessons learned, which are also provided. Transitioning a project's development methodology, modernizing the test approach, and introducing a new system of test documentation may provide significant benefits to the development of a system, but these types of process changes must be weighed against the needs of the project. This dissertation provides details of the effort to improve the effectiveness of the T&E process on an example project, as a framework for possible implementation on similar systems

Structuring Framework for Early Validation of Product Ideas

The advancing digitalization leads to new challenges in the development of new product ideas. The systems, which need to be developed, are becoming increasingly complex. As complexity rises, so does the slowness of organizations. In order to remain innovative and be able to react quickly to changes in the market, validation approaches offer great potential. An essential focus is the early, customer-centric and continuous validation of development artefacts. This is the only way to cope with shorter development cycles, and develop products that address a real need. To plan a validation, engineers need to have an idea about the possibilities within a validation. This paper presents a structuring framework for early validation of product ideas, which contains the three tools of Validation Map, Building Block Cards and Validation Canvas. The objective is to enable easier planning of validation experiments

Life cycle cost modelling as an aircraft design decision support tool

This report summarizes the work that has been carried out as part of the FLAVIIR project, a 5 year research program looking at technologies for future unmanned air vehicles. A novel classification of aircraft product definition is utilised and a framework to estimate the life cycle cost of aircraft using the product definition is presented. The architecture to estimate the life cycle cost and the associated models are described.The acquisition costs are estimated using a hierarchical structure and a discrete simulation model is used to estimate the maintenance and operation costs. The acquisition cost model uses an object oriented approach with libraries of materials and processes integrated into the cost model. Risk analysis is performed to identify the important design parameters and uncertainty in the model. The acquisition cost model developed has the capability to estimate the costs of aircraft structures manufactured using metal-based materials as well as non-metal-based materials.The discrete event simulation model estimates the operation and maintenance costs of a fleet of aircraft using the mission characteristics, aircraft performance and the logistics data as input. The aircraft performance parameters are calculated by using aerodynamic analysis along with performance analysis models and the simulation model utilises a novel methodology to link aircraft performance with survivability analysis for estimating the maintenance costs.A framework is presented in which the cost models developed can be integrated into the conceptual design process to facilitate the comparison between different configurations. The usage of the life cycle cost framework as a decision support tool is outlined and three case studies are presented which include composites vs metals trade-off analysis, optimisation studies and web deployment for real time cost estimation. The novel contributions of this research are outlined and interesting avenues for future research that can be pursued are identified

The Impact of Customer Collaboration on Agile Product Development Success in Technology Startups Within the Pacific Northwest

While agile software development is being adopted in more organizations recently, many products using the methodology are still failing in the market due to inadequate customer collaboration despite the purported benefits. Within start-ups, where speed and early market penetration can be the death or success of a company, understanding if using agile software development including adequate customer collaboration makes a significant difference is important. A study which investigates the impact of including customer collaboration in the agile product development process could uncover whether or not a product is successful within technology focused start-up’s in the Pacific Northwest. This research could allow technology focused start-ups to learn how to emulate success and avoid pitfalls using agile software development to create better, more transformative products for the world. The research question is: how does the inclusion of customer collaboration in the agile product development process by product owners impact the overall success of the product within Pacific Northwest technology start-up companies? This concept paper includes information on the nature of the study, the significance, relationship to cognate, a literature review beginning, and a significant amount of research related to the hypothesis

U.S. Unmanned Aerial Vehicles (UAVS) and Network Centric Warfare (NCW) impacts on combat aviation tactics from Gulf War I through 2007 Iraq

Unmanned, aerial vehicles (UAVs) are an increasingly important element of many modern militaries. Their success on battlefields in Afghanistan, Iraq, and around the globe has driven demand for a variety of types of unmanned vehicles. Their proven value consists in low risk and low cost, and their capabilities include persistent surveillance, tactical and combat reconnaissance, resilience, and dynamic re-tasking. This research evaluates past, current, and possible future operating environments for several UAV platforms to survey the changing dynamics of combat-aviation tactics and make recommendations regarding UAV employment scenarios to the Turkish military. While UAVs have already established their importance in military operations, ongoing evaluations of UAV operating environments, capabilities, technologies, concepts, and organizational issues inform the development of future systems. To what extent will UAV capabilities increasingly define tomorrow's missions, requirements, and results in surveillance and combat tactics? Integrating UAVs and concepts of operations (CONOPS) on future battlefields is an emergent science. Managing a transition from manned- to unmanned and remotely piloted aviation platforms involves new technological complexity and new aviation personnel roles, especially for combat pilots. Managing a UAV military transformation involves cultural change, which can be measured in decades.http://archive.org/details/usunmannedaerial109454211Turkish Air Force authors.Approved for public release; distribution is unlimited

An Integrated Framework to Assess ‘Leanness’ Performance in Distribution Centres

The theory behind lean philosophy is to create more value with less. Effective lean management enables organisations to exceed customer expectations while reducing costs. Despite the fact that numerous practices and approaches are used in the process of implementing lean philosophy and reducing waste within supply chain systems, little effort has been directed into assessing the leanness level of distribution and its impact on overall performance. Given the vital role of distribution units within supply chains, this research aims to develop a comprehensive lean assessment framework that integrates a selected set of statistical, analytical, and mathematical techniques in order to assess the ‘leanness’ level in the distribution business. Due to the limited number of published articles in the area of lean distribution, there are no clear definitions of the underlying factors and practices. Therefore, the primary phase of the proposed framework addresses the identification of lean distribution dimensional structure and practices. The other two phases of the framework discuss the development of a structured model for lean distribution and address the process to find a quantitative lean index for benchmarking lean implementation in distribution centres. Integrating the three phases provides the decision makers with an indicator of performance, subject to applying various lean practices. Incorporating the findings of a survey that sent to 700 distribution businesses in Ireland along with value stream mapping, modelling, simulation, and data envelopment analysis, has given the framework strength in the assessment of leanness. Research outcomes show that lean distribution consists of five key dimensions; workforce management, item replenishment, customers, transportation, and process quality. Lean practices associated with these dimensions are mainly focused on enhancing the communication channels with customers, simplifying the distribution networks structure, people participating in problem solving and a continuous improvement process, and increasing the reliability and efficiency of the distribution operations. The final output of the framework is two key leanness indices; one is set to measure the tactical leanness level, while the second index represents the leanness at the operational level. Both indices can effectively be used in evaluating the lean implementation process and conducting a benchmarking process based on the leanness level

Optimising Supply Chain Performance via Information Sharing and Coordinated Management

Supply chain management has attracted much attention in the last decade. There has been a noticeable shift from a traditional individual organisation-based management to an integrated management across the supply chain network since the end of the last century. The shift contributes to better decision making in the supply chain context, as it is necessary for a company to cooperate with other supply chain members by utilising relevant information such as inventory, demand and resource capacity. In other words, information sharing and coordinated management are essential mechanisms to improve supply chain performance. Supply chains may differ significantly in terms of industry sectors, geographic locations, and firm sizes. This study was based on case studies from small and medium sized manufacturing supply chains in People Republic of China. The study was motivated by the following facts. Firstly, small and medium enterprises have made a big contribution to China’s economic growth. Several studies revealed that most of the Chinese manufacturing enterprises became aware of the importance of supply chain management, but compared to western firms, the supply chain management level of Chinese firms had been lagging behind. Research on supply chain management and performance optimisation in Chinese small and medium sized enterprises (SMEs) was very scarce. Secondly, there had been plenty of studies in the literature that focused on two or three level supply chains whilst considering a number of uncertain factors (e.g. customer demand) or a single supply chain performance indicator (e.g. cost). However, the research on multiple stage supply chain systems with multiple uncertainties and multiple objectives based on real industrial cases had been spared and deserved more attention. One reason was due to the lack of reliable industrial data that required an enormous effort to collect the primary data and there was a serious concern about data confidentiality from the industry aspect. This study employed two SME manufacturing companies as case studies. The first one was in the Aluminium industry and another was in the Chemical industry. The aim was to better understand the characteristics of the supply chains in Chinese SMEs through performing in-depth case studies, and built models and tools to evaluate different strategies for improving their supply chain performance. The main contributions of this study included the following aspects. Firstly, this study generalised a supply chain model including a domestic supply chain part and an international supply chain part based on deep case studies with the emphasis on identifying key characteristics in the case supply chains, such as uncertainties, constraints and cost elements in association with flows and activities in the domestic supply chain and the international supply chain. Secondly, two important SCM issues, i.e. the integrated raw material procurement and finished goods production planning, and the international sales planning, were identified. Thirdly, mathematical models were formulated to represent the supply chain model taking into account multiple uncertainties. Fourthly, several operational strategies utilising the concepts of just-in-time, safety-stock/capacity, Kanban, and vendor managed inventory, were evaluated and compared with the case company's original strategy in various scenarios through simulation methods, which enabled quantification of the impact of information sharing on supply chain performance. Fifthly, a single objective genetic algorithm was developed to optimise the integrated raw material ordering and finished goods production decisions under (s, S) policy (a dynamic inventory control policy), which enabled the impact of coordinated management on supply chain performance to be quantified. Finally, a multiple objectives genetic algorithm considering both total supply chain cost and customer service level was developed to optimise the integrated raw material ordering and finished goods production with the international sales plan decisions under (s, S) policy in various scenarios. This also enabled the quantification of the impact of coordinated management on supply chain performances

Applicability Analysis of Performance Based Logistics Implementation for US Army Stryker Armored Vehicle to Improve Turkish Army Weapon System Support

Sponsored Report (for Acquisition Research Program)This research examines the total life cycle logistics system and performance based logistics (PBL) implementation. This report looks into the application of Defense Acquisition University''s 12-step PBL implementation model defined in Performance Based Logistics: A Program Manager''s Product Support Guide(March 2005) for the Stryker Light Armored Vehicle (LAV) support. The research consists of a literature review of the Total Life Cycle System Management, Supply Chain Management concepts and performance based logistics. It is followed by a comparative analysis of PBL application between Stryker Light Armored Vehicle (LAV) Program and Turkish Army''s advanced Armored Combat Vehicle (ACV) program support practices. This research provides recommendations to improve Turkish Army''s weapon system support, and maintainability based on findings and introduces potentials through implementation of PBL practices for the Turkish Army.Naval Postgraduate School Acquisition Research ProgramApproved for public release; distribution is unlimited