3 research outputs found
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
Testing in Continuous Integration (CI) involves test case prioritization,
selection, and execution at each cycle. Selecting the most promising test cases
to detect bugs is hard if there are uncertainties on the impact of committed
code changes or, if traceability links between code and tests are not
available. This paper introduces Retecs, a new method for automatically
learning test case selection and prioritization in CI with the goal to minimize
the round-trip time between code commits and developer feedback on failed test
cases. The Retecs method uses reinforcement learning to select and prioritize
test cases according to their duration, previous last execution and failure
history. In a constantly changing environment, where new test cases are created
and obsolete test cases are deleted, the Retecs method learns to prioritize
error-prone test cases higher under guidance of a reward function and by
observing previous CI cycles. By applying Retecs on data extracted from three
industrial case studies, we show for the first time that reinforcement learning
enables fruitful automatic adaptive test case selection and prioritization in
CI and regression testing.Comment: Spieker, H., Gotlieb, A., Marijan, D., & Mossige, M. (2017).
Reinforcement Learning for Automatic Test Case Prioritization and Selection
in Continuous Integration. In Proceedings of 26th International Symposium on
Software Testing and Analysis (ISSTA'17) (pp. 12--22). AC
Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review
Regression testing is an essential activity to assure that software code
changes do not adversely affect existing functionalities. With the wide
adoption of Continuous Integration (CI) in software projects, which increases
the frequency of running software builds, running all tests can be
time-consuming and resource-intensive. To alleviate that problem, Test case
Selection and Prioritization (TSP) techniques have been proposed to improve
regression testing by selecting and prioritizing test cases in order to provide
early feedback to developers. In recent years, researchers have relied on
Machine Learning (ML) techniques to achieve effective TSP (ML-based TSP). Such
techniques help combine information about test cases, from partial and
imperfect sources, into accurate prediction models. This work conducts a
systematic literature review focused on ML-based TSP techniques, aiming to
perform an in-depth analysis of the state of the art, thus gaining insights
regarding future avenues of research. To that end, we analyze 29 primary
studies published from 2006 to 2020, which have been identified through a
systematic and documented process. This paper addresses five research questions
addressing variations in ML-based TSP techniques and feature sets for training
and testing ML models, alternative metrics used for evaluating the techniques,
the performance of techniques, and the reproducibility of the published
studies