Time-Space Efficient Regression Testing for Configurable Systems

Configurable systems are those that can be adapted from a set of options. They are prevalent and testing them is important and challenging. Existing approaches for testing configurable systems are either unsound (i.e., they can miss fault-revealing configurations) or do not scale. This paper proposes EvoSPLat, a regression testing technique for configurable systems. EvoSPLat builds on our previously-developed technique, SPLat, which explores all dynamically reachable configurations from a test. EvoSPLat is tuned for two scenarios of use in regression testing: Regression Configuration Selection (RCS) and Regression Test Selection (RTS). EvoSPLat for RCS prunes configurations (not tests) that are not impacted by changes whereas EvoSPLat for RTS prunes tests (not configurations) which are not impacted by changes. Handling both scenarios in the context of evolution is important. Experimental results show that EvoSPLat is promising. We observed a substantial reduction in time (22%) and in the number of configurations (45%) for configurable Java programs. In a case study on a large real-world configurable system (GCC), EvoSPLat reduced 35% of the running time. Comparing EvoSPLat with sampling techniques, 2-wise was the most efficient technique, but it missed two bugs whereas EvoSPLat detected all bugs four times faster than 6-wise, on average.Comment: 14 page

A Survey on Software Testing Techniques using Genetic Algorithm

The overall aim of the software industry is to ensure delivery of high quality software to the end user. To ensure high quality software, it is required to test software. Testing ensures that software meets user specifications and requirements. However, the field of software testing has a number of underlying issues like effective generation of test cases, prioritisation of test cases etc which need to be tackled. These issues demand on effort, time and cost of the testing. Different techniques and methodologies have been proposed for taking care of these issues. Use of evolutionary algorithms for automatic test generation has been an area of interest for many researchers. Genetic Algorithm (GA) is one such form of evolutionary algorithms. In this research paper, we present a survey of GA approach for addressing the various issues encountered during software testing.Comment: 13 Page

Feedback driven adaptive combinatorial testing

The configuration spaces of modern software systems are too large to test exhaustively. Combinatorial interaction testing (CIT) approaches, such as covering arrays, systematically sample the configuration space and test only the selected configurations. The basic justification for CIT approaches is that they can cost-effectively exercise all system behaviors caused by the settings of t or fewer options. We conjecture, however, that in practice many such behaviors are not actually tested because of masking effects – failures that perturb execution so as to prevent some behaviors from being exercised. In this work we present a feedback-driven, adaptive, combinatorial testing approach aimed at detecting and working around masking effects. At each iteration we detect potential masking effects, heuristically isolate their likely causes, and then generate new covering arrays that allow previously masked combinations to be tested in the subsequent iteration. We empirically assess the effectiveness of the proposed approach on two large widely used open source software systems. Our results suggest that masking effects do exist and that our approach provides a promising and efficient way to work around them

A Chemical Reaction Optimization Approach to Prioritize the Regression Test Cases of Object-Oriented Programs

Regression test case prioritization is used to improve certain performance goals. Limited resources force to choose an effective prioritization technique, which makes an ordering of the test cases so that the most suitable test case will be executed first. Executing regression test cases for a fixed time is all about time aware test case prioritization. Regression test case prioritization using chemical reaction optimization (CRO) for object-oriented programs is proposed in this paper. The effectiveness of the test case ordering was measured using average percentage of faults detected (APFD). Experiments on three object-oriented subject programs involving three different techniques were performed to judge the proposed approach. The empirical results indicate that the algorithm implemented using CRO gives a higher APFD value than the other two techniques

A Comprehensive Framework for Testing Database-Centric Software Applications

The database is a critical component of many modern software applications. Recent reports indicate that the vast majority of database use occurs from within an application program. Indeed, database-centric applications have been implemented to create digital libraries, scientific data repositories, and electronic commerce applications. However, a database-centric application is very different from a traditional software system because it interacts with a database that has a complex state and structure. This dissertation formulates a comprehensive framework to address the challenges that are associated with the efficient and effective testing of database-centric applications. The database-aware approach to testing includes: (i) a fault model, (ii) several unified representations of a program's database interactions, (iii) a family of test adequacycriteria, (iv) a test coverage monitoring component, and (v) tools for reducing and re-ordering a test suite during regression testing.This dissertation analyzes the worst-case time complexity of every important testing algorithm. This analysis is complemented by experiments that measure the efficiency and effectiveness of thedatabase-aware testing techniques. Each tool is evaluated by using it to test six database-centric applications. The experiments show thatthe database-aware representations can be constructed with moderate time and space overhead. The adequacy criteria call for test suitesto cover 20% more requirements than traditional criteria and this ensures the accurate assessment of test suite quality. It is possibleto enumerate data flow-based test requirements in less than one minute and coverage tree path requirements are normally identified in no morethan ten seconds. The experimental results also indicate that the coverage monitor can insert instrumentation probes into all six of theapplications in fewer than ten seconds. Although instrumentation may moderately increase the static space overhead of an application, the coverage monitoring techniques only increase testing time by 55% on average. A coverage tree often can be stored in less than five seconds even though the coverage report may consume up to twenty-fivemegabytes of storage. The regression tester usually reduces or prioritizes a test suite in under five seconds. The experiments also demonstrate that the modified test suite is frequently more streamlined than the initial tests

Predicting Test Case Verdicts Using TextualAnalysis of Commited Code Churns

Background: Continuous Integration (CI) is an agile software development practice that involves producing several clean builds of the software per day. The creation of these builds involve running excessive executions of automated tests, which is hampered by high hardware cost and reduced development velocity. Goal: The goal of our research is to develop a method that reduces the number of executed test cases at each CI cycle.Method: We adopt a design research approach with an infrastructure provider company to develop a method that exploits Ma-chine Learning (ML) to predict test case verdicts for committed sourcecode. We train five different ML models on two data sets and evaluate their performance using two simple retrieval measures: precision and recall. Results: While the results from training the ML models on the first data-set of test executions revealed low performance, the curated data-set for training showed an improvement on performance with respect to precision and recall. Conclusion: Our results indicate that the method is applicable when training the ML model on churns of small size

Preemptive regression testing of workflow-based web services

published_or_final_versio