Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing

Context: Verification and validation (V&V) activities make up 20 to 50 percent of the total development costs of a software system in practice. Test automation is proposed to lower these V&V costs but available research only provides limited empirical data from industrial practice about the maintenance costs of automated tests and what factors affect these costs. In particular, these costs and factors are unknown for automated GUI-based testing. Objective: This paper addresses this lack of knowledge through analysis of the costs and factors associated with the maintenance of automated GUI-based tests in industrial practice. Method: An empirical study at two companies, Siemens and Saab, is reported where interviews about, and empirical work with, Visual GUI Testing is performed to acquire data about the technique's maintenance costs and feasibility. Results: 13 factors are observed that affect maintenance, e.g. tester knowledge/experience and test case complexity. Further, statistical analysis shows that developing new test scripts is costlier than maintenance but also that frequent maintenance is less costly than infrequent, big bang maintenance. In addition a cost model, based on previous work, is presented that estimates the time to positive return on investment (ROI) of test automation compared to manual testing. Conclusions: It is concluded that test automation can lower overall software development costs of a project whilst also having positive effects on software quality. However, maintenance costs can still be considerable and the less time a company currently spends on manual testing, the more time is required before positive, economic, ROI is reached after automation

Advancing automation and robotics technology for the Space Station Freedom and for the US economy

The progress made by levels 1, 2, and 3 of the Office of Space Station in developing and applying advanced automation and robotics technology is described. Emphasis is placed upon the Space Station Freedom Program responses to specific recommendations made in the Advanced Technology Advisory Committee (ATAC) progress report 10, the flight telerobotic servicer, and the Advanced Development Program. Assessments are presented for these and other areas as they apply to the advancement of automation and robotics technology for the Space Station Freedom

Cyber-Virtual Systems: Simulation, Validation & Visualization

We describe our ongoing work and view on simulation, validation and visualization of cyber-physical systems in industrial automation during development, operation and maintenance. System models may represent an existing physical part - for example an existing robot installation - and a software simulated part - for example a possible future extension. We call such systems cyber-virtual systems. In this paper, we present the existing VITELab infrastructure for visualization tasks in industrial automation. The new methodology for simulation and validation motivated in this paper integrates this infrastructure. We are targeting scenarios, where industrial sites which may be in remote locations are modeled and visualized from different sites anywhere in the world. Complementing the visualization work, here, we are also concentrating on software modeling challenges related to cyber-virtual systems and simulation, testing, validation and verification techniques for them. Software models of industrial sites require behavioural models of the components of the industrial sites such as models for tools, robots, workpieces and other machinery as well as communication and sensor facilities. Furthermore, collaboration between sites is an important goal of our work.Comment: Preprint, 9th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2014

Automatic software for controlling cryogenic systems

A technical discussion of the lessons learned during the seven years of software development/testing which occurred on the Liquid Oxygen System for the Space Shuttle at the Kennedy Space Center is given. Problems which were solved during these years came into four distinct phases: design/debug before simulation runs, verification using simulation with models up through Space Transportation System-1 launch, hardware usage from first launch to Space Transportation System-5 launch, and future use. Each problem/solution describes the apparent problem requirements/constraints, usable alternatives, selected action, and results

A framework of justification criteria for advanced manufacturing technology implementation in small and medium enterprises

Today in order to stay in businesses and prosper, Small and Medium Enterprises (SMEs) are seeking higher electiveness and competitiveness across the entire cycle of marketing, product design, manufacture, test and sales. SMEs play an increasingly important role in all aspects of competitiveness: both products and production techniques, but also management methods, the organization of the firm and human resources training. One of the ways by which SMEs can achieve a competitive advantage in manufacturing is through the implementation of Advanced Manufacturing Technology (AMT). An increasing number of them have chosen and are choosing various levels of AMT as the solution. Realizing the importance of SMEs, an attempt has been made in this paper to review the application of AMT in SMEs. Also, a framework has been offered for the implementation of AMT in SMEs. Finally, a summary of findings and conclusions are presented

Software support for manufacturing operations in Belgian SMEs: one size fits all?

Manufacturing companies face a big challenge to bridge the gap between their business and manufacturing processes. The urge to increase efficiency makes it necessary to align the business and manufacturing processes. Small and Medium-sized Enterprises (SMEs) experience several barriers to adopt software support for manufacturing operations. This paper gives an overview of a research study conducted in Belgian SMEs. The research studied the current adoption of software support for manufacturing operations and the barriers that SMEs experience to invest in this type of software. The study is concluded with a number of considerations to enable the adoption of software support for manufacturing operations by SMEs