Assuring the model evolution of protocol software specifications by regression testing process improvement

A preliminary version of this paper has been presented at the 10th International Conference on Quality Software (QSIC 2010).Model-based testing helps test engineers automate their testing tasks so that they are more cost-effective. When the model is changed because of the evolution of the specification, it is important to maintain the test suites up to date for regression testing. A complete regeneration of the whole test suite from the new model, although inefficient, is still frequently used in the industry, including Microsoft. To handle specification evolution effectively, we propose a test case reusability analysis technique to identify reusable test cases of the original test suite based on graph analysis. We also develop a test suite augmentation technique to generate new test cases to cover the change-related parts of the new model. The experiment on four large protocol document testing projects shows that our technique can successfully identify a high percentage of reusable test cases and generate low-redundancy new test cases. When compared with a complete regeneration of the whole test suite, our technique significantly reduces regression testing time while maintaining the stability of requirement coverage over the evolution of requirements specifications. Copyright © 2011 John Wiley & Sons, Ltd.link_to_subscribed_fulltex

Colored model based testing for software product lines (CMBT-SWPL)

Over the last decade, the software product line domain has emerged as one of the mostpromising software development paradigms. The main benefits of a software product lineapproach are improvements in productivity, time to market, product quality, and customersatisfaction.Therefore, one topic that needs greater emphasis is testing of software product lines toachieve the required software quality assurance. Our concern is how to test a softwareproduct line as early as possible in order to detect errors, because the cost of error detectedIn early phases is much less compared to the cost of errors when detected later.The method suggested in this thesis is a model-based, reuse-oriented test technique calledColored Model Based Testing for Software Product Lines (CMBT-SWPL). CMBT-SWPLis a requirements-based approach for efficiently generating tests for products in a soft-ware product line. This testing approach is used for validation and verification of productlines. It is a novel approach to test product lines using a Colored State Chart (CSC), whichconsiders variability early in the product line development process. More precisely, the vari-ability will be introduced in the main components of the CSC. Accordingly, the variabilityis preserved in test cases, as they are generated from colored test models automatically.During domain engineering, the CSC is derived from the feature model. By coloring theState Chart, the behavior of several product line variants can be modeled simultaneouslyin a single diagram and thus address product line variability early. The CSC representsthe test model, from which test cases using statistical testing are derived.During application engineering, these colored test models are customized for a specificapplication of the product line. At the end of this test process, the test cases are generatedagain using statistical testing, executed and the test results are ready for evaluation. Inxaddition, the CSC will be transformed to a Colored Petri Net (CPN) for verification andsimulation purposes.The main gains of applying the CMBT-SWPL method are early detection of defects inrequirements, such as ambiguities incompleteness and redundancy which is then reflectedin saving the test effort, time, development and maintenance costs