Search based algorithms for test sequence generation in functional testing

Information and Software Technology (DOI: 10.1016/j.infsof.2014.07.014)The generation of dynamic test sequences from a formal specification, complementing traditional testing methods in order to find errors in the source code. Objective In this paper we extend one specific combinatorial test approach, the Classification Tree Method (CTM), with transition information to generate test sequences. Although we use CTM, this extension is also possible for any combinatorial testing method. Method The generation of minimal test sequences that fulfill the demanded coverage criteria is an NP-hard problem. Therefore, search-based approaches are required to find such (near) optimal test sequences. Results The experimental analysis compares the search-based technique with a greedy algorithm on a set of 12 hierarchical concurrent models of programs extracted from the literature. Our proposed search-based approaches (GTSG and ACOts) are able to generate test sequences by finding the shortest valid path to achieve full class (state) and transition coverage. Conclusion The extended classification tree is useful for generating of test sequences. Moreover, the experimental analysis reveals that our search-based approaches are better than the greedy deterministic approach, especially in the most complex instances. All presented algorithms are actually integrated into a professional tool for functional testing.Spanish Ministry of Economy and Competitiveness and FEDER under contract TIN2011-28194 and fellowship BES-2012-055967. Project 8.06/5.47.4142 in collaboration with the VSB-Tech. Univ. of Ostrava, Universidad de Málaga, Andalucía Tech. and EU Grant ICT-257574 (FITTEST project)

Integrasi Pareto Fitness, Multiple-Population Dan Temporary Population Pada Algoritma Genetika Untuk Pembangkitan Data Tes Pada Pengujian Perangkat Lunak

Pengujian perangkat lunak memerlukan biaya yang mahal dan sering kali lebih dari 50% biaya keseluruhan dalam pengembangan perangkat lunak digunakan dalam tahapan ini. Untuk mengurangi biaya proses pengujian perangkat lunak secara otomatis dapat digunakan. Hal yang sangat penting dalam pengujian perangkat lunak secara otomatis adalah proses menghasilkan data tes. Pengujian secara otomatis yang paling efektif dalam menekan biaya adalah pengujian branch coverage. Salah satu metode yang banyak digunakan dan memiliki kinerja baik adalah algoritma genetika (AG). Salah satu permasalahan AG dalam menghasilkan data tes adalah ketiga target cabang dipilih memungkinkan tidak ada satupun individu yang memenuhi kriteria. Hal ini akan menyebabkan proses pencarian data tes memakan waktu lebih lama. Oleh karena itu di dalam penelitian ini diusulkan integrasi pareto fitness, multiple-population dan temporary population di dalam proses pencarian data tes dengan menggunakan AG (AG-PFMPTP). Multiple-population diusulkan untuk menghindari premature convergence. Kemudian pareto fitness dan temporary population digunakan untuk mencari beberapa data tes sekaligus, kemudian mengevaluasinya dan memasukkan ke dalam archive temporary population. Dari hasil pengujian yang telah dilakukan rata-rata generasi metode AG-PFMPTP secara signifikan lebih sedikit dalam menghasilkan data tes yang dibutuhkan dibandingkan metode AG standar ataupun AG dengan multiple-population (AG-MP) pada semua benchmark program yang digunakan. Hal tersebut menunjukkan metode yang diusulkan lebih cepat dalam mencari data tes yang dibutuhka

Using memetic algorithm for robustness testing of contract-based software models

Graph Transformation System (GTS) can formally specify the behavioral aspects of complex systems through graph-based contracts. Test suite generation under normal conditions from GTS specifications is a task well-suited to evolutionary algorithms such as Genetic and Particle Swarm Optimization (PSO) metaheuristics. However, testing the vulnerabilities of a system under unexpected events such as invalid inputs is essential. Furthermore, the mentioned global search algorithms tend to make big jumps in the system’s state-space that are not concentrated on particular test goals. In this paper, we extend the HGAPSO approach into a cost-aware Memetic Algorithm (MA) by making small local changes through a proposed local search operator to optimize coverage score and testing costs. Moreover, we test GTS specifications not only under normal events but also under unexpected situations. So, three coverage-based testing strategies are investigated, including normal testing, robustness testing, and a hybrid strategy. The effectiveness of the proposed test generation algorithm and the testing strategies are evaluated through a type of mutation analysis at the model-level. Our experimental results show that (1) the hybrid testing strategy outperforms normal and robustness testing strategies in terms of fault-detection capability, (2) the robustness testing is the most cost-efficient strategy, and (3) the proposed MA with the hybrid testing strategy outperforms the state-of-the-art global search algorithms