A SOFTWARE TESTING ASSESSMENT TO MANAGE PROJECT TESTABILITY

The demand for testing services is, to a large extend a ?derived demand? influenced directly by the manner in which prior developed activities are undertaken. The early stages of a structured software development life cycle (SDLC) project can often run behind schedule, shrinking the time available for performing adequate testing especially when software release deadlines have to be met. This situation fosters the need to influence pre-testing activities and manage the testing effort efficiently. Our research examines how to measure testability of a SDLC project before testing begins. It builds on the ?design for testability? perspective by introducing a ?manage for testability? perspective. Software testability focuses on whether the activities of the SDLC process are progressing in ways that enable the testing team to find software product defects if they exist. To address this challenge, we develop a software testing assessment. This assessment is designed to provide testing managers with information needed to: (1) influence pre-testing activities in ways that ultimately increase testing efficiency and effectiveness, and (2) plan testing resources to optimize efficient and effective testing. We developed specific software testing assessment measures through interviews with key informants. We present data collected for the measures for large-scale structured software development projects to illustrate the assessment?s usefulness and application

A Reference Model for Software and System Inspections. White Paper

Software Quality Assurance (SQA) is an important component of the software development process. SQA processes provide assurance that the software products and processes in the project life cycle conform to their specified requirements by planning, enacting, and performing a set of activities to provide adequate confidence that quality is being built into the software. Typical techniques include: (1) Testing (2) Simulation (3) Model checking (4) Symbolic execution (5) Management reviews (6) Technical reviews (7) Inspections (8) Walk-throughs (9) Audits (10) Analysis (complexity analysis, control flow analysis, algorithmic analysis) (11) Formal method Our work over the last few years has resulted in substantial knowledge about SQA techniques, especially the areas of technical reviews and inspections. But can we apply the same QA techniques to the system development process? If yes, what kind of tailoring do we need before applying them in the system engineering context? If not, what types of QA techniques are actually used at system level? And, is there any room for improvement.) After a brief examination of the system engineering literature (especially focused on NASA and DoD guidance) we found that: (1) System and software development process interact with each other at different phases through development life cycle (2) Reviews are emphasized in both system and software development. (Figl.3). For some reviews (e.g. SRR, PDR, CDR), there are both system versions and software versions. (3) Analysis techniques are emphasized (e.g. Fault Tree Analysis, Preliminary Hazard Analysis) and some details are given about how to apply them. (4) Reviews are expected to use the outputs of the analysis techniques. In other words, these particular analyses are usually conducted in preparation for (before) reviews. The goal of our work is to explore the interaction between the Quality Assurance (QA) techniques at the system level and the software level

A research review of quality assessment for software

Measures were recommended to assess the quality of software submitted to the AdaNet program. The quality factors that are important to software reuse are explored and methods of evaluating those factors are discussed. Quality factors important to software reuse are: correctness, reliability, verifiability, understandability, modifiability, and certifiability. Certifiability is included because the documentation of many factors about a software component such as its efficiency, portability, and development history, constitute a class for factors important to some users, not important at all to other, and impossible for AdaNet to distinguish between a priori. The quality factors may be assessed in different ways. There are a few quantitative measures which have been shown to indicate software quality. However, it is believed that there exists many factors that indicate quality and have not been empirically validated due to their subjective nature. These subjective factors are characterized by the way in which they support the software engineering principles of abstraction, information hiding, modularity, localization, confirmability, uniformity, and completeness

A dynamic systems engineering methodology research study. Phase 2: Evaluating methodologies, tools, and techniques for applicability to NASA's systems projects

A study of NASA's Systems Management Policy (SMP) concluded that the primary methodology being used by the Mission Operations and Data Systems Directorate and its subordinate, the Networks Division, is very effective. Still some unmet needs were identified. This study involved evaluating methodologies, tools, and techniques with the potential for resolving the previously identified deficiencies. Six preselected methodologies being used by other organizations with similar development problems were studied. The study revealed a wide range of significant differences in structure. Each system had some strengths but none will satisfy all of the needs of the Networks Division. Areas for improvement of the methodology being used by the Networks Division are listed with recommendations for specific action

Designing and evaluating the usability of a machine learning API for rapid prototyping music technology

To better support creative software developers and music technologists' needs, and to empower them as machine learning users and innovators, the usability of and developer experience with machine learning tools must be considered and better understood. We review background research on the design and evaluation of application programming interfaces (APIs), with a focus on the domain of machine learning for music technology software development. We present the design rationale for the RAPID-MIX API, an easy-to-use API for rapid prototyping with interactive machine learning, and a usability evaluation study with software developers of music technology. A cognitive dimensions questionnaire was designed and delivered to a group of 12 participants who used the RAPID-MIX API in their software projects, including people who developed systems for personal use and professionals developing software products for music and creative technology companies. The results from the questionnaire indicate that participants found the RAPID-MIX API a machine learning API which is easy to learn and use, fun, and good for rapid prototyping with interactive machine learning. Based on these findings, we present an analysis and characterization of the RAPID-MIX API based on the cognitive dimensions framework, and discuss its design trade-offs and usability issues. We use these insights and our design experience to provide design recommendations for ML APIs for rapid prototyping of music technology. We conclude with a summary of the main insights, a discussion of the merits and challenges of the application of the CDs framework to the evaluation of machine learning APIs, and directions to future work which our research deems valuable

Space Transportation Avionics Technology Symposium. Volume 1: Executive summary

The focus of the symposium was to examine existing and planned avionics technology processes and products and to recommend necessary changes for strengthening priorities and program emphases. Innovative changes in avionics technology development and design processes, identified during the symposium, are needed to support the increasingly complex, multi-vehicle, integrated, autonomous space-based systems. Key technology advances make such a major initiative viable at this time: digital processing capabilities, integrated on-board test/checkout methods, easily reconfigurable laboratories, and software design and production techniques

Adapting a Stress Testing Framework to a Multi-module Security-oriented Spring Application

Programmeeritakse mitmekomponendilist süsteemi. Kolm põhikomponenti on järgmised: põhiserver (Spring rakendus), mobiilirakendused (iOS, Android), klienditeeninduse veebiportaalid. Kõige tähtsam süsteemi töös on põhiserver, kuna see on enamuse veebiportaalide ning mobiilirakenduste päringute sihtpunkt. See on mitmemooduliline projekt, kus kõik moodulid suhtlevad omavahel. Potentsiaalselt hakkab süsteemi kasutama sadu tuhandeid inimesi – kümneid tuhandeid paralleelseid sessioone. Seetõttu tuleb läbi viia süsteemi ulatuslik koormustestimine. Kahjuks on nii, et koormustestimise raamistikud oma originaalseisus ei sobi antud süsteemi testimiseks. Seega, koormustestimise raamistiku tuleb seadistada ning laiendada selleks, et see toetaks antud süsteemi spetsiifilisi protokolle ja võimaldaks testida kõiki komponente üheskoos. Hetkel on saadaval palju koormustestimise raamistikke. Mõned nendest on: Locust, Apache JMeter, Gatling Project. Need raamistikud erinevad üksteisest programmeerimiskeele, eriomaduste ning põhiloogika järgi. Kuna tegu on kommertsprojektiga, peab valitud koormustestimise raamistik vastama kliendi funktsionaalsete ja mittefunktsionaalsete nõuetele. Kuna koormustestimist viiakse läbi ainult põhiserveril, peab seadistama ja laiendama valitud raamistikku, et simuleerida teisi süsteemi komponente ja serveri protokolle. See töö annab kiire ülevaate varem mainitud koormustestimise raamistikest eriomaduste järgi, valib raamistiku, mida kohandatakse antud projekti raames koormustestimise läbi viimiseks ning kirjeldab kohandamise protsessi. Samuti toob see töö välja mõned koormustestimise raamistike piirangud ning kirjeldab meetodeid nende ületamiseks. Viimaks, süsteemi testitakse valitud raamistiku abil ning esitatakse ja valideeritakse tulemusi.A multi-component system is being build. Three main components are: backend server (Spring application), mobile applications (iOS, Android), customer service web portals. Our main concern is the backend server, because it is the destination of the majority of requests from customer service web portals and mobile applications. It is a multi-module project where all modules communicate to each other. The system is going to be used potentially by hundreds thousands of users with tens thousands of simultaneous usages. Therefore, extensive stress-testing must be conducted. Unfortunately, stress-testing frameworks in the original state are not suitable for the given system. Thus a stress-testing framework must be configured and extended to the point it supports the system’s specific protocols and can test all the system’s components together. There are numerous of stress-testing frameworks available. Some examples are: Locust, Apache JMeter, Gatling Project. These frameworks differ in terms of coding language, features and core logic. As it is a commercial project, the chosen stress-testing framework must also comply with client’s functional and non-functional requirements. Due to stress-testing being conducted only on the backend server component, the selected stress-testing framework must be configured/extended to simulate other components and the required server protocols. The thesis provides a brief comparison of the available stress-testing frameworks based on their features and written code language and define the one which is going to be adapted to conduct the stress-testing within the project and how the adaptation is done. The thesis also points out some of stress-testing frameworks’ limitations with techniques to overcome them. Finally, the system is tested using the selected testing framework and the results are presented and validated

EXEL : a language for interactive behavioral synthesis

This paper describes a new input language for behavioral synthesis called EXEL. EXEL is a powerful language that permits the user to specify partially designed structures in the language. It employs a mixed graphic/textual user interface to enhance user interactivity. EXEL's design model is comprehensive: it permits specification of synchronous and asynchronous behavior, and allows specification of general timing constraints. A flexible type construct permits the user to define operators and components to be used in the description. Finally, it simplifies compilation by using a small set of constructs for specifying timing and asynchronouos behavior. The compiler for EXEL runs on SUN-3 workstations and is written in C and SUNVIEW