Enhanced non-dominated sorting genetic algorithm for test case optimization

Due to inevitable software changes, regression testing has become a crucial phase in software development process. Many software testers and researchers agreed that regression testing process consumes more time and cost during software development. Test case optimization has become one of the best solutions to overcome problems in regression testing. Test case optimization is focusing on reducing number of test cases in the test suite that may reduce the overall testing time, cost and effort of software testers. It considers multiple objectives and provides several numbers of optimal solution based on objectives of the testing. Therefore, this research aims at developing an alternative solution of test case optimization technique using NSGA II with fitness scaling as an additional function. Fitness scaling function is applied in NSGA II to eliminate pre-mature convergence among set of solution in the evolution of offspring in NSGA II which may produce more efficient fitness value. This research focuses on regression testing optimization by implementing weight of test cases and fault detection rate per test case as its objective function for optimization purposes. The proposed technique is applied to the GUI-based testing case study. The result shows that Pareto front produced by enhanced NSGA II give more wider set of solution that contains more alternatives and provide better trade-off among solutions. The evaluation shows that enhanced NSGA II perform better compared to conventional NSGA II by increasing the percentage of the reduced test cases with 25% and yield lower fault detection loss with 1.64% which indicating that set of reduced test cases using enhanced NSGA II is able to maintain the fault detection capability in the system under test

Pembentukan Data Uji Menggunakan Algoritma Optimisasi Koloni Semut dan Pendekatan Teknik Pengujian Kotak Abu-Abu

Pengujian perangkat lunak dapat menjadi cara yang efektif untuk memperbaiki kualitas serta ketahanan perangkat lunak. Secara umum pengujian perangkat lunak dapat dilakukan dengan teknik kotak putih (white box) dan kotak hitam (black box). Teknik pengujian kotak abu-abu (gray box) dikenal sebagai teknik penguijan yang menggunakan pendekatan teknik pengujian kotak putih dan kotak hitam. Teknikpengujian kotak abu-abu memperluas kriteria cakupan logika dari teknik pengujian kotak putih dan menemukan semua kemungkinannya dari model desain perangkat lunak sepertipada teknik pengujian kotak hitam. Teknik pengujian kotak abu-abu menggunakan data seperti: UML Diagram, Model Arsitektur perangkat lunak, ataupun Finite State Machine Diagram (State Model) untuk membentuk kasus pengujian. Selain itu pengujian perangkat lunak dapat dilakukan dengan teknik Soft Computing dan Hard Computing. Teknik Hard Computing sulit diimplementasikan pada problematika yang ada saat ini. Sehinga teknik Soft Computing dapat menjadi alternatif yang digunakan dalam pengujian perangkat lunak. Teknik Soft Computing merupakan teknik yang lebih berfokus pada menginterpretasikan perilaku sistem dari pada hasil presisi. Teknik Soft Computing biasanya berdasarkan teknik logika fuzzy, jaringan saraf tiruan ataupun pengambilan keputusan berdasarkan nilai distribusi kemungkinan. Teknik pengujian perangkat lunak yang mengunakanteknil Soft Computing denganmetode pengambilan sampel berdasarkan nilai kemungkinan dikenal dengan teknik pengujian statistika. Berdasarkan penelitian terkait, algoritma Ant Colony Optimization (ACO) atau optimisasi koloni semut merupakan algoritma yang digunakan pada teknik pengujian statistika untuk membentuk data uji dengan kemampuan yang lebih baik dibandingkan algoritma lain seperti: Simulated Annealing (SA) serta algoritma genetika. Selain itu ACO juga memiliki hasil yang sebanding dengan algoritma Particle Swarm Optimization (PSO) atau optimisasi kawanan partikel. ACO diimplementasikan pada kode program perangkat lunak yang diuji untuk membentuk data uji berdasarkan nilai kemungkinan terbesar random data uji dari domain terpilih. Pemilihan data uji merupakan faktor utama yang menentukan keberhasilan dari suatu pengujian perangkat lunak. Sehingga pemilihan teknik yang tepat dapat membantu menunjang keberhasilan dalam pengujian perangkat lunak.Pada penelitian ini, ACO diimplementasikan berdasarkan teknik pengujian kotak abu-abu menggunakan diagram UML State Machine. Pembentukan data uji yang berkualitas adalah berdasarkan kecukupan kriteria percabangan yang dapat ditelusuri. Tujuan dari penelitian ini adalah untuk mendapatkan hasil perbandingan pembentukan data uji dengan teknik pengujian kotak abu-abu menggunakan diagram UML State Machine dan teknik pengujian struktur kotak putih menggunakan kode program. Hasil penelitian ini diharapkan mampu memberikan gambaran kualitas data uji yang dibentuk dari masing-masing teknik ========================================================================================================Software testing can be an effective way to improve software quality and reliability. In general, software testing techniques can be classified into White Box and Black Box. Gray Box testing technique is known as testing technique which is used both White Box and Black Box techniques. It extends the logical coverage criteria of white box method and finds all the possibility from the design model which is like black box method. Gray Box technique uses data such as: UML Diagram, Software Architecture Model, or Finite State Machine Diagram (State Model) to generate test cases. The other classification of software testing technique is Hard Computing technique and Soft Computing technique. Hard Computing technique is difficult to be implemented in today problems. And Soft Computing technique can be an alternative way which can be used in software testing. Soft Computing technique focuses on system behavior interpretation than precision result. Soft Computing technique is based on fuzzy logic, neural network, or decision making by probability distribution. Software testing which is used Soft Computing technique and its sampling method based on probability distribution, is known as statistical testing. Based on research, Ant Colony Optimization (ACO) Algorithm is algorithm which is used in statistical testing to generate test data, and its result better than another algorithm such as: Simulated Annealing (SA), and Genetic algorithm. Besides, its result is comparable with Paricle Swarm Optimization (PSO) algorithm. ACO was implemented in program code of software under test to generate test data based on the highest probability value of random test data from domain chosen. Test data selection is main factor which determines the software testing success. In this research, ACO was implemented based on Gray Box testing using UML State Machine Diagram. The quality of test data generated is based on branch coverage criteria. This research aims to get comparison result between Gray Box testing using UML State Machine Diagram and structural White Box Testing using program code in generating test data. The result of this research is expected to give description about the quality of test data generated from each technique

ADAPTIVE SAMPLING METHODS FOR TESTING AUTONOMOUS SYSTEMS

In this dissertation, I propose a software-in-the-loop testing architecture that uses adaptive sampling to generate test suites for intelligent systems based upon identifying transitions in high-level mission criteria. Simulation-based testing depends on the ability to intelligently create test-cases that reveal the greatest information about the performance of the system in the fewest number of runs. To this end, I focus on the discovery and analysis of performance boundaries. Locations in the testing space where a small change in the test configuration leads to large changes in the vehicle's behavior. These boundaries can be used to characterize the regions of stable performance and identify the critical factors that affect autonomous decision making software. By creating meta-models which predict the locations of these boundaries we can efficiently query the system and find informative test scenarios. These algorithms form the backbone of the Range Adversarial Planning Tool (RAPT): a software system used at naval testing facilities to identify the environmental triggers that will cause faults in the safety behavior of unmanned underwater vehicles (UUVs). This system was used to develop UUV field tests which were validated on a hardware platform at the Keyport Naval Testing Facility. The development of test cases from simulation to deployment in the field required new analytical tools. Tools that were capable of handling uncertainty in the vehicle's performance, and the ability to handle large datasets with high-dimensional outputs. This approach has also been applied to the generation of self-righting plans for unmanned ground vehicles (UGVs) using topological transition graphs. In order to create these graphs, I had to develop a set of manifold sampling and clustering algorithms which could identify paths through stable regions of the configuration space. Finally, I introduce an imitation learning approach for generating surrogate models of the target system's control policy. These surrogate agents can be used in place of the true autonomy to enable faster than real-time simulations. These novel tools for experimental design and behavioral modeling provide a new way of analyzing the performance of robotic and intelligent systems, and provide a designer with actionable feedback