Infrastructure Support for Controlled Experimentation with Software Testing and Regression Testing Techniques

Where the development, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support rigorous controlled experimentation with testing techniques, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of testing techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with software testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has and can be expected to have on controlled experimentation with testing techniques

Bringing Light into the Dark - Improving Students’ Black-Box Testing Competencies using Game-Design Elements

As software becomes increasingly complex, there is a growing need to enhance quality assurance in software engineering. However, the lack of qualified human resources is a barrier to performing software testing activities in software companies. At the same time, software testing can be considered a tedious task and is often not done at the necessary level of detail, e.g., designing test cases. However, it is crucial for novice programmers and testers to acquire and improve their testing competencies, and to utilize testing techniques, e.g., black-box testing. Teaching software testing is often based on theoretical instructions, resulting in limited practical experience. As a result, students may not develop the necessary testing mindset, highlighting the need for more extensive software testing education. To address this issue, this paper utilizes a design science research approach to implement a gamified learning system that promotes black-box testing competencies with empirical insights from a field test

Mutating database queries

A set of mutation operators for SQL queries that retrieve information from a database is developed and tested against a set of queries drawn from the NIST SQL Conformance Test Suite. The mutation operators cover a wide spectrum of SQL features, including the handling of null values. Additional experiments are performed to explore whether the cost of executing mutants can be reduced using selective mutation or the test suite size can be reduced by using an appropriate ordering of the mutants. The SQL mutation approach can be helpful in assessing the adequacy of database test cases and their development, and as a tool for systematically injecting faults in order to compare different database testing technique