1,100 research outputs found

    History Based Multi Objective Test Suite Prioritization in Regression Testing Using Genetic Algorithm

    Get PDF
    Regression testing is the most essential and expensive testing activity which occurs throughout the software development life cycle. As Regression testing requires executions of many test cases it imposes the necessity of test case prioritization process to reduce the resource constraint. Test case prioritization technique schedule the test case in an order that increase the chance of early fault detection. In this paper we propose a genetic algorithm based prioritization technique which uses the historical information of system level test cases to prioritize test cases to detect most severe faults early. In addition the proposed approach also calculates weight factor for each requirement to achieve customer satisfaction and to improve the rate of severe fault detection. To validate the proposed approach we performed controlled experiments over industry projects which proved the proposed approach effectiveness in terms of average percentage of fault detected

    Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration

    Full text link
    Testing in Continuous Integration (CI) involves test case prioritization, selection, and execution at each cycle. Selecting the most promising test cases to detect bugs is hard if there are uncertainties on the impact of committed code changes or, if traceability links between code and tests are not available. This paper introduces Retecs, a new method for automatically learning test case selection and prioritization in CI with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases. The Retecs method uses reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure history. In a constantly changing environment, where new test cases are created and obsolete test cases are deleted, the Retecs method learns to prioritize error-prone test cases higher under guidance of a reward function and by observing previous CI cycles. By applying Retecs on data extracted from three industrial case studies, we show for the first time that reinforcement learning enables fruitful automatic adaptive test case selection and prioritization in CI and regression testing.Comment: Spieker, H., Gotlieb, A., Marijan, D., & Mossige, M. (2017). Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration. In Proceedings of 26th International Symposium on Software Testing and Analysis (ISSTA'17) (pp. 12--22). AC

    Test case prioritization approaches in regression testing: A systematic literature review

    Get PDF
    Context Software quality can be assured by going through software testing process. However, software testing phase is an expensive process as it consumes a longer time. By scheduling test cases execution order through a prioritization approach, software testing efficiency can be improved especially during regression testing. Objective It is a notable step to be taken in constructing important software testing environment so that a system's commercial value can increase. The main idea of this review is to examine and classify the current test case prioritization approaches based on the articulated research questions. Method Set of search keywords with appropriate repositories were utilized to extract most important studies that fulfill all the criteria defined and classified under journal, conference paper, symposiums and workshops categories. 69 primary studies were nominated from the review strategy. Results There were 40 journal articles, 21 conference papers, three workshop articles, and five symposium articles collected from the primary studies. As for the result, it can be said that TCP approaches are still broadly open for improvements. Each approach in TCP has specified potential values, advantages, and limitation. Additionally, we found that variations in the starting point of TCP process among the approaches provide a different timeline and benefit to project manager to choose which approaches suite with the project schedule and available resources. Conclusion Test case prioritization has already been considerably discussed in the software testing domain. However, it is commonly learned that there are quite a number of existing prioritization techniques that can still be improved especially in data used and execution process for each approach

    The effectiveness of different test case prioritization techniques in continuous integration

    Get PDF
    Abstract. Regression testing is often performed in software development to increase the quality of the software. Especially continuous integration (CI) environments face challenges related to the efficiency of the regression testing feedback loop. It is important to get feedback about the changes made as soon as possible so that work can be started on fixing the potential issues caused by the changes. Regression testing is optimized by prioritizing test cases into an order that increases the early fault detection rate. However, different test case prioritization (TCP) approaches and techniques set different requirements, such as access to source code, for the environment. As the CI environments set high time and resource constraints for the TCP, not all techniques are applicable and some care should be put into selecting a suitable TCP technique. This thesis consists of a literature review that aims to consider CI environments’s special requirements, map out the most common TCP techniques as well as evaluate their applicability to these CI environments. Based on the benefits and drawbacks noted for the approaches, it seems likely that history-based TCP approaches could be the best fit for these resource intensive CI environments.Tiivistelmä. Regressiotestausta käytetään ohjelmistokehityksessä ohjelmiston laadun parantamiseen. Regressiotestauksen palautesilmukan (engl. feedback loop) tehokkuus tuottaa haasteita varsinkin jatkuvan integraation kehitysympäristöissä. On tärkeää, että kehittäjät saavat palautetta tekemiensä muutosten mahdollisista epätoivotuista vaikutuksista, jotta tarvittavat korjaustyöt voidaan aloittaa mahdollisimman pikaisesti. Regressiotestausta voidaan optimoida laittamalla testitapaukset sellaiseen tärkeysjärjestykseen, joka kasvattaa aikaista virheiden havaitsemistahtia (engl. early fault detection rate). Eri priorisointitekniikat asettavat kuitenkin erilaisia vaatimuksia suoritusympäristölle. Esimerkiksi jotkin tekniikat tarvitsevat näkyvyyden lähdekoodiin. Testitapausten priorisointitekniikan valinta kannattaa tehdä huolella jatkuvan integraation kehitysympäristöissä, sillä kyseisillä ympäristöillä on korkeita aika- ja resurssienkäyttövaatimuksia. Tämä tutkielma koostuu kirjallisuuskatsauksesta, jonka tarkoituksena on tarkastella jatkuvan integraation kehitysympäristöjen erikoisvaatimuksia, kartoittaa yleisimmät testitapausten priorisointitekniikat sekä arvioida näiden tekniikoiden soveltuvuutta jatkuvan integraation kehitysympäristöihin. Tunnistettujen testitapausten priorisoinnin lähestymistapojen etujen ja haittojen perusteella vaikuttaisi todennäköiseltä, että historiapohjaiset lähestymistavat voisivat soveltua parhaiten jatkuvan integraation resurssi-intensiivisiin kehitysympäristöihin

    An improved method for test case prioritization by incorporating historical test case data

    Get PDF
    AbstractTest case prioritization reorders test cases from a previous version of a software system for the current release to optimize regression testing. We have previously introduced a technique for test case prioritization using historical test case performance data. The technique was based on a test case prioritization equation, which directly computes the priority of each test case using the historical information of the test case using an equation with constant coefficients. This technique was compared just with random ordering approach. In this paper, we present an enhancement of the aforementioned technique in two ways. First, we propose a new prioritization equation with variable coefficients gained according to the available historical performance data, which acts as a feedback from the previous test sessions. Second, a family of comprehensive empirical studies has been conducted to evaluate the performance of the technique. We have compared the proposed technique with our previous technique and the technique proposed by Kim and Porter. The experimental results demonstrate the effectiveness of the proposed technique in accelerating the rate of fault detection in history-based test case prioritization

    An empirical study on the use of defect prediction for test case prioritization

    Get PDF
    Test case prioritization has been extensively re-searched as a means for reducing the time taken to discover regressions in software. While many different strategies have been developed and evaluated, prior experiments have shown them to not be effective at prioritizing test suites to find real faults. This paper presents a test case prioritization strategy based on defect prediction, a technique that analyzes code features - such as the number of revisions and authors - to estimate the likelihood that any given Java class will contain a bug. Intuitively, if defect prediction can accurately predict the class that is most likely to be buggy, a tool can prioritize tests to rapidly detect the defects in that class. We investigated how to configure a defect prediction tool, called Schwa, to maximize the likelihood of an accurate prediction, surfacing the link between perfect defect prediction and test case prioritization effectiveness. Using 6 real-world Java programs containing 395 real faults, we conducted an empirical evaluation comparing this paper's strategy, called G-clef, against eight existing test case prioritization strategies. The experiments reveal that using defect prediction to prioritize test cases reduces the number of test cases required to find a fault by on average 9.48% when compared with existing coverage-based strategies, and 10.4% when compared with existing history-based strategies
    • …
    corecore