Software Quality Assurance

Telecom networks are composed of very complex software-controlled systems. In recent years, business and technology needs are pushing vendors towards service agility where they must continuously develop, deliver, and improve such software over very short cycles. Moreover, being critical infrastructure, Telecom systems must meet important operational, legal, and regulatory requirements in terms of quality and performance to avoid outages. To ensure high quality software, processes and models must be put in place to enable quick and easy decision making across the development cycle. In this chapter, we will discuss the background and recent trends in software quality assurance. We will then introduce BRACE: a cloud-based, fully-automated tool for software defect prediction, reliability and availability modeling and analytics. In particular, we will discuss a novel Software Reliability Growth Modeling (SRGM) algorithm that is the core of BRACE. The algorithm provides defect prediction for both early and late stages of the software development cycle. To illustrate and validate the tool and algorithm, we also discuss key use cases, including actual defect and outage data from two large-scale software development projects from telecom products. BRACE is being successfully used by global teams of various large-scale software development projects

Models for Testing Modifiable Systems

The work describes reliability and security growth models for modifiable software systems as a result of revisions and tests performed for specified input data areas. The work shows that the known reliability growth models are of monotonically increasing type, which is not in line with current multi-version team technologies of software development that are primarily based on the open-source code. The authors suggest new non-monotonically increasing models of software reliability evaluation and planning that allow taking into account the effect of decreased reliability resulting from updates or wavefront errors. The work describes the elaborated bigeminal and generic reliability evaluation model as well as the models and test planning procedures. The work includes calculated expressions for the evaluation of the model accuracy and shows that the developed models are adequate to real data. An example is given of transition from probability models to fuzzy models in case of incomplete basic data. The work provides general recommendations for selection of software tool testing models

A RISK-INFORMED DECISION-MAKING METHODOLOGY TO IMPROVE LIQUID ROCKET ENGINE PROGRAM TRADEOFFS

This work provides a risk-informed decision-making methodology to improve liquid rocket engine program tradeoffs with the conflicting areas of concern affordability, reliability, and initial operational capability (IOC) by taking into account psychological and economic theories in combination with reliability engineering. Technical program risks are associated with the number of predicted failures of the test-analyze-and-fix (TAAF) cycle that is based on the maturity of the engine components. Financial and schedule program risks are associated with the epistemic uncertainty of the models that determine the measures of effectiveness in the three areas of concern. The affordability and IOC models' inputs reflect non-technical and technical factors such as team experience, design scope, technology readiness level, and manufacturing readiness level. The reliability model introduces the Reliability- As-an-Independent-Variable (RAIV) strategy that aggregates fictitious or actual hotfire tests of testing profiles that differ from the actual mission profile to estimate the system reliability. The main RAIV strategy inputs are the physical or functional architecture of the system, the principal test plan strategy, a stated reliability-bycredibility requirement, and the failure mechanisms that define the reliable life of the system components. The results of the RAIV strategy, which are the number of hardware sets and number of hot-fire tests, are used as inputs to the affordability and the IOC models. Satisficing within each tradeoff is attained by maximizing the weighted sum of the normalized areas of concern subject to constraints that are based on the decision-maker's targets and uncertainty about the affordability, reliability, and IOC using genetic algorithms. In the planning stage of an engine program, the decision variables of the genetic algorithm correspond to fictitious hot-fire tests that include TAAF cycle failures. In the program execution stage, the RAIV strategy is used as reliability growth planning, tracking, and projection model. The main contributions of this work are the development of a comprehensible and consistent risk-informed tradeoff framework, the RAIV strategy that links affordability and reliability, a strategy to define an industry or government standard or guideline for liquid rocket engine hot-fire test plans, and an alternative to the U.S. Crow/AMSAA reliability growth model applying the RAIV strategy

Bayesian zero-failure reliability demonstration

We study required numbers of tasks to be tested for a technical system, including systems with built-in redundancy, in order to demonstrate its reliability with regard to its use in a process after testing, where the system has to function for different types of tasks, which we assume to be independent. We consider optimal numbers of tests as required for Bayesian reliability demonstration in terms of failure-free periods, which is suitable in case of catastrophic failures, and in terms of the expected number of failures in a process after testing. We explicitly assume that testing reveals zero failures. For the process after testing, we consider both deterministic and random numbers of tasks. We also consider optimal numbers of tasks to be tested when aiming at minimal total expected costs, including costs of testing and of failures in the process after testing. Cost and time constraints on testing are also included in the analysis. We consider such reliability demonstration for a single type of task, as well as for multiple types of tasks to be performed by one system. We also consider optimal Bayesian reliability demonstration testing in combination with flexibility in the system redundancy, where more components can be installed to reduce test effort. For systems with redundancy, we restrict attention to systems with exchangeable components, with testing only at the component level. We use the Bayesian approach with the Binomial model and Beta prior distributions for the failure probabilities. We discuss the influence of choice of prior distribution on the required zero-failure test numbers, where these inferences are very sensitive to the choice of prior distribution, which requires careful attention to the interpretation of non-informativeness of priors

Development of Availability and Sustainability Spares Optimization Models for Aircraft Reparables

The Republic of Singapore Air Force (RSAF) conducts Logistics Support Analysis (LSA) studies in various engineering and logistics efforts on the myriad of weapon systems. In these studies, inventory spares provisioning, availability and sustainability analyses are key focus areas to ensure asset sustenance. In particular, OPUS10, a commercial-off-the-shelf software, is extensively used to conduct reparable spares optimization in acquisition programs. However, it is limited in its ability to conduct availability and sustainability analyses of time-varying operational demands, crucial in Operations & Support (O&S) and contingency planning. As the RSAF seeks force structure expansion to include more sophisticated weapon systems, the operating environment will become more complex. Agile and responsive logistics solutions are needed to ensure the RSAF engineering community consistently pushes for deepening competencies, particularly in LSA capabilities. This research is aimed at the development of a model solution that combines optimization and sustainability capabilities to meet the dynamic requirements in O&S and contingency planning. In particular, a unique dynamic operational profile conversion model was developed to realize these capabilities. It is envisaged that the research would afford the ease of use, versatility, speed and accuracy required in LSA studies, to provide the necessary edge in inventory reparable spares modeling

Resilience, Reliability, and Recoverability (3Rs)

Recent natural and human-made disasters, mortgage derivatives crises, and the need for stable systems in different areas have renewed interest in the concept of resilience, especially as it relates to complex industrial systems with mechanical failures. This concept in the engineering systems (infrastructure) domain could be interpreted as the probability that system conditions exceed an irrevocable tipping point. But the probability in this subject covers the different areas that different approaches and indicators can evaluate. In this context, reliability engineering is used the reliability (uptime) and recoverability (downtime) indicators (or performance indicators) as the most useful probabilistic tools for performance measurement. Therefore, our research penalty area is the resilience concept in combination with reliability and recoverability. It must be said that the resilience evaluators must be considering a diversity of knowledge sources. In this thesis, the literature review points to several important implications for understanding and applying resilience in the engineering area and The Arctic condition. Indeed, we try to understand the application and interaction of different performance-based resilience concepts. In this way, a collection of the most popular performance-based resilience analysis methods with an engineering perspective is added as a state-of-the-art review. The performance indicators studies reveal that operational conditions significantly affect the components, industry activities, and infrastructures performance in various ways. These influential factors (or heterogeneity) can broadly be studied into two groups: observable and unobservable risk factors in probability analysis of system performance. The covariate-based models (regression), such as proportional hazard models (PHM), and their extent are the most popular methods for quantifying observable and unobservable risk factors. The report is organized as follows: After a brief introduction of resilience, chapters 2,3 priorly provide a comprehensive statistical overview of the reliability and recoverability domain research by using large scientific databases such as Scopus and Web of Science. As the first subsection, a detailed review of publications in the reliability and recoverability assessment of the engineering systems in recent years (since 2015) is provided. The second subsection of these chapters focuses on research done in the Arctic region. The last subsection presents covariate-based reliability and recoverability models. Finally, in chapter 4, the first part presents the concept and definitions of resilience. The literature reviews four main perspectives: resilience in engineering systems, resilience in the Arctic area, the integration of “Resilience, Reliability, and Recoverability (3Rs)”, and performance-based resilience models