The empire infrastructure for testing experimentation

It is widely believed that there is not enough experimentation in the field of computer science. One area in particular in which additional experimentation is needed involves regression testing of large, real software. Regression testing is the expensive process of validating changes made to previously tested software. One reason why there is not enough experimentation in this area is the substantial infrastructure required for such experimentation. We have developed an infrastructure designed for performing various testing experiments on a large, real test subject. This paper describes the requirements, design, and implementation issues associated with the development of our infrastructure. We also performed an experiment to validate the infrastructure, in which we investigated the affects test granularity has on regression test selection techniques. The experiment shows the usefulness of our infrastructure. We believe that our infrastructure can serve as a prototype for future development of similar infrastructures involving other large test subjects

The Impact of Test Suite Granularity on the CostEffectiveness of Regression Testing

Regression testing is an expensive testing process used to validate software following modifications. The cost-effectiveness of regression testing techniques varies with characteristics of test suites. One such characteristic, test suite granularity, involves the way in which test inputs are grouped into test cases within a test suite. Various cost-benefits tradeoffs have been attributed to choices of test suite granularity, but almost no research has formally examined these tradeoffs. To address this lack, we conducted several controlled experiments, examining the effects of test suite granularity on the costs and benefits of several controlled experiments, examining the effects of test suite granularity on the costs and benefits of several regression testing methodologies across six releases of two non-trivial software systems. Our results expose essential tradeoffs to consider when designing test suites for use in regression testing evolving systems

The Impact of Test Suite Granularity on the Cost-Effectiveness of Regression Testing

Regression testing is an expensive testing process used to validate software following modi cations. The cost-eectiveness of regression testing techniques varies with characteristics of test suites. One suchcharacteristic, test suite granularity, involves the way in which test inputs are grouped into test cases within a test suite. Various cost-bene ts tradeos have been attributed to choices of test suite granularity, but almost no research has formally examined these tradeos. To address this lack, we conducted several controlled experiments, examining the eects of test suite granularity on the costs and bene ts of several regression testing methodologies across six releases of two non-trivial software systems. Our results expose essential tradeos to consider when designing test suites for use in regression testing evolving systems