Software test automation : a design and tool selection approach for a heterogeneous environment

textThis report describes a design approach for implementing a software test automation solution that can accommodate existing test processes in an organization. The process of implementing a software test automation solution is a large undertaking and requires careful planning to avoid unsuccessful implementations. This report outlines a design that can integrate with existing business and development processes in an organization, and recommends automation and development frameworks for achieving the test automation goals. Considerations for a heterogeneous test environment with varying types of supported operating systems, such as Windows and Linux, and multiple test execution environments, such as Java and .NET, have been made in this design and in the tool selections for the system implementation. The report also describes some of the challenges and caveats of automation in a heterogeneous environment along with recommended solutions to these challenges.Electrical and Computer Engineerin

Space station structures and dynamics test program

The design, construction, and operation of a low-Earth orbit space station poses challenges for development and implementation of technology. One specific challenge is the development of a dynamics test program for defining the space station design requirements, and identifying and characterizing phenomena affecting the space station's design and development. The test proposal, as outlined, is a comprehensive structural dynamics program to be launched in support of the space station (SS). Development of a parametric data base and verification of the mathematical models and analytical analysis tools necessary for engineering support of the station's design, construction, and operation provide the impetus for the dynamics test program. The four test phases planned are discussed: testing of SS applicable structural concepts; testing of SS prototypes; testing of actual SS structural hardware; and on-orbit testing of SS construction

IEEE Standard 1500 Compliance Verification for Embedded Cores

Core-based design and reuse are the two key elements for an efficient system-on-chip (SoC) development. Unfortunately, they also introduce new challenges in SoC testing, such as core test reuse and the need of a common test infrastructure working with cores originating from different vendors. The IEEE 1500 Standard for Embedded Core Testing addresses these issues by proposing a flexible hardware test wrapper architecture for embedded cores, together with a core test language (CTL) used to describe the implemented wrapper functionalities. Several intellectual property providers have already announced IEEE Standard 1500 compliance in both existing and future design blocks. In this paper, we address the problem of guaranteeing the compliance of a wrapper architecture and its CTL description to the IEEE Standard 1500. This step is mandatory to fully trust the wrapper functionalities in applying the test sequences to the core. We present a systematic methodology to build a verification framework for IEEE Standard 1500 compliant cores, allowing core providers and/or integrators to verify the compliance of their products (sold or purchased) to the standar

Design evolution of the orbiter reaction control subsystem

The challenges of space shuttle orbiter reaction control subsystem development began with selection of the propellant for the subsystem. Various concepts were evaluated before the current Earth storable, bipropellant combination was selected. Once that task was accomplished, additional challenges of designing the system to satisfy the wide range of requirements dictated by operating environments, reusability, and long life were met. Verification of system adequacy was achieved by means of a combination of analysis and test. The studies, the design efforts, and the test and analysis techniques employed in meeting the challenges are described

Space station structures and dynamics test program

The design, construction, and operation of a low-Earth orbit space station poses unique challenges for development and implementation of new technology. The technology arises from the special requirement that the station be built and constructed to function in a weightless environment, where static loads are minimal and secondary to system dynamics and control problems. One specific challenge confronting NASA is the development of a dynamics test program for: (1) defining space station design requirements, and (2) identifying the characterizing phenomena affecting the station's design and development. A general definition of the space station dynamic test program, as proposed by MSFC, forms the subject of this report. The test proposal is a comprehensive structural dynamics program to be launched in support of the space station. The test program will help to define the key issues and/or problems inherent to large space structure analysis, design, and testing. Development of a parametric data base and verification of the math models and analytical analysis tools necessary for engineering support of the station's design, construction, and operation provide the impetus for the dynamics test program. The philosophy is to integrate dynamics into the design phase through extensive ground testing and analytical ground simulations of generic systems, prototype elements, and subassemblies. On-orbit testing of the station will also be used to define its capability

Test evaluation trials present different challenges for trial managers compared to Intervention trials

Introduction Test evaluation trials present different challenges for trial managers compared to intervention trials. There has been very little research on the management of test evaluation trials and how this impacts on trial success, in comparison with intervention trials. Evaluations of medical tests present specific challenges, because they are a pivot point bridging the complexities of pathways prompting testing with treatment decision-making. We systematically explored key differences in the trial design and management of test evaluation trials compared to intervention trials at the different stages of study design and delivery. We identified challenges in test evaluation trials that were more pronounced than in intervention trials, based on experience from 10 test evaluation trials. Methods We formed a focus group of 7 trial managers and a statistician who had been involved in the day-to-day management of both test evaluation trials and intervention trials. We used discussion and content analysis to group challenges from 10 trials into a structured thematic format. The trials covered a range of medical conditions, diagnostic tests, clinical pathways and conditions including chronic kidney disease, chronic pelvic pain, colitis, detrusor over-activity, group B streptococcal colonisation, tuberculosis and colorectal, lung, ovarian and thyroid cancers. Results We identified 10 common themes underlying challenges that are more pronounced in test evaluation compared to intervention trials. We illustrate these themes with examples from 10 trials, including with 31 specific challenges we experienced. The themes were ethics/governance; accessing patient populations; recruitment; patient preference; test processes, clinical pathways and samples storage; uncertainty of diagnostic results; verifying diagnosis (reference standard); follow-up; adverse effects; and diagnostic impact. Conclusion We present 10 common themes, including 31 challenges, in test evaluation trials that will be helpful to others designing and managing future test evaluation trials. Proactive identification of potential challenges at the design and planning stages of test evaluation trials will enable strategies to improve trial design and management that may be different from standard strategies used for intervention trials. Future work could extend this topic to include challenges for other trial stakeholders including participants, clinicians, statisticians and funders

Structural load challenges during space shuttle development

The challenges that resulted from the unique configuration of the space shuttle and capabilities developed to meet these challenges are described. The methods and the organization that were developed to perform dynamic loads analyses on the space shuttle configuration and to assess dynamic data developed after design are discussed. Examples are presented from the dynamic loads analysis of the lift-off and maximum dynamic pressure portion of ascent. Also shown are orbital flight test results, for which selected predicted responses are compared to measured data for the lift-off and high-dynamic-pressure times of ascent. These results have generally verified the design analysis. However, subscale testing was found to be deficient in predicting full-scale results in two areas: the ignition overpressure at lift-off and the aerodynamics/plume interactions at high-q boost. In these areas, the results of the flight test program were accommodated with no impact to the vehicle design

Challenges and guidelines on designing test cases for test bots

Test bots are automated testing tools that autonomously and periodically run a set of test cases that check whether the system under test meets the requirements set forth by the customer. The automation decreases the amount of time a development team spends on testing. As development projects become larger, it is important to focus on improving the test bots by designing more effective test cases because otherwise time and usage costs can increase greatly and misleading conclusions from test results might be drawn, such as false positives in the test execution. However, literature currently lacks insights on how test case design affects the effectiveness of test bots. This paper uses a case study approach to investigate those effects by identifying challenges in designing tests for test bots. Our results include guidelines for test design schema for such bots that support practitioners in overcoming the challenges mentioned by participants during our study.Comment: To be published in IEEE/ACM 42nd International Conference on Software Engineering Workshops (ICSEW'20), May 23--29, 2020, Seoul, Republic of Kore