CTJ: Input-Output Based Relation Combinatorial Testing Strategy Using Jaya Algorithm

            ويكاد يكون من المستحيل اختبار كل مجموعة من المدخلات نظرًا لأن تنفيذ حالات الاختبار يتطلب وقتا طويلا للغاية. الأختبار الاندماجي هو السبيل لتخطي عقبات الاختبار الشامل من خلال أختبار كل قيم المدخلات لكل المعاملات المركبة المتعددة طرق الترتيب.   يمكن تقسيم الاختبار التجميعي إلى ثلاثة أنواع هي تفاعل القوة الموحد ، والتفاعل المتغير والقوة ، والعلاقة القائمة على المدخلات والمخرجات . ان الطريقة الاخيرة الانفة الذكر تختزل الفحص الاندماجي الى مجموعة ضمن اختيار الشخص الفاحص. معظم الابحاث في الاختبار الاندماجي طبقت في تفاعل القوة الموحدة وقوة التفاعل المتغيرة ، ومع ذلك ، هناك اهتمام قليل جدا بالعلاقة بين المدخلات والمخرجات. لذا تم اقتراح خوارزمية جايا في هذا البحث  كخوارزمية مثلي لانشاء جدول الفحص الاندماجي باستراتيجية تعتمد على العلاقة بين المدخلات والمخرجات. نتيجة تطبيق خوارزمية جايا في الاختبار الاندماجي القائم على المدخلات والمخرجات مقبولة لأنها تنتج العدد الأمثل تقريبًا لحالات الاختبار في نطاق زمني مقبول.Software testing is a vital part of the software development life cycle. In many cases, the system under test has more than one input making the testing efforts for every exhaustive combination impossible (i.e. the time of execution of the test case can be outrageously long). Combinatorial testing offers an alternative to exhaustive testing via considering the interaction of input values for every t-way combination between parameters. Combinatorial testing can be divided into three types which are uniform strength interaction, variable strength interaction and input-output based relation (IOR). IOR combinatorial testing only tests for the important combinations selected by the tester. Most of the researches in combinatorial testing applied the uniform and the variable interaction strength, however, there is still a lack of work addressing IOR. In this paper, a Jaya algorithm is proposed as an optimization algorithm engine to construct a test list based on IOR in the proposed combinatorial test list generator strategy into a tool called CTJ. The result of applying the Jaya algorithm in input-output based combinatorial testing is acceptable since it produces a nearly optimum number of test cases in a satisfactory time range

PWiseHA: Application of Harmony Search Algorithm for Test Suites Generation using Pairwise Techniques

Pairwise testing is an approach that tests every possible combinations of values of parameters. In this approach, number of all combinations are selected to ensure all possible pairs of parameter values are included in the final test suite. Generating test cases is the most active research area in pairwise testing, but the generation process of the efficient test suite with minimum size can be considered as one of optimization problem. In this research paper we articulate the problem of finding a pairwise final test suite as a search problem and the application of harmony search algorithm to solve it. Also, in this research paper, we developed a pairwise software testing tool called PWiseHA that will generate test cases using harmony search algorithm and this PWiseHA is well optimized. Finally, the result obtained from PWiseHA shows a competitive results if matched with the result of existing pairwise testing tools. PWiseHA is still in prototype form, an obvious starting point for future work

An experimental study of hyper-heuristic selection and acceptance mechanism for combinatorial t-way test suite generation

Recently, many meta-heuristic algorithms have been proposed to serve as the basis of a t -way test generation strategy (where t indicates the interaction strength) including Genetic Algorithms (GA), Ant Colony Optimization (ACO), Simulated Annealing (SA), Cuckoo Search (CS), Particle Swarm Optimization (PSO), and Harmony Search (HS). Although useful, metaheuristic algorithms that make up these strategies often require specific domain knowledge in order to allow effective tuning before good quality solutions can be obtained. Hyperheuristics provide an alternative methodology to meta-heuristics which permit adaptive selection and/or generation of meta-heuristics automatically during the search process. This paper describes our experience with four hyper-heuristic selection and acceptance mechanisms namely Exponential Monte Carlo with counter (EMCQ), Choice Function (CF), Improvement Selection Rules (ISR), and newly developed Fuzzy Inference Selection (FIS),using the t -way test generation problem as a case study. Based on the experimental results, we offer insights on why each strategy differs in terms of its performance

GALP: A hybrid artificial intelligence algorithm for generating covering array

Today, there are a lot of useful algorithms for covering array (CA) generation, one of the branches of combinatorial testing. The major CA challenge is the generation of an array with the minimum number of test cases (efficiency) in an appropriate run-time (performance), for large systems. CA generation strategies are classified into several categories: computational and meta-heuristic, to name the most important ones. Generally, computational strategies have high performance and yield poor results in terms of efficiency, in contrast, meta-heuristic strategies have good efficiency and lower performance. Among the strategies available, some are efficient strategies but suffer from low performance; conversely, some others have good performance, but is not such efficient. In general, there is not a strategy that enjoys both above-mentioned metrics. In this paper, it is tried to combine the genetic algorithm and the Augmented Lagrangian Particle Swarm Optimization with Fractional Order Velocity to produce the appropriate test suite in terms of efficiency and performance. Also, a simple and effective minimizing function is employed to increase efficiency. The evaluation results show that the proposed strategy outperforms the existing approaches in terms of both efficiency and performance

DEO: A Dynamic Event Order Strategy for t-way Sequence Covering Array Test Data Generation

الخلاصة:               تعد مصفوفة التغطية المتسلسلة (SCA) من مجالات البحث النشطة في السنوات الأخيرة. بخلاف مصفوفة التغطية الاعتيادية (CA) ، يختلف ترتيب تسلسل العوامل في عملية إنشاء حالة الاختبار. تقوم هذه الورقة بمراجعة أحدث الاستراتيجيات في  الأعمال السابقة، حيث أن العثور على الحد الأدنى لحجم مجموعة الاختبار يعتبر مشكلة .NP-Hard بالإضافة إلى ذلك ، تتمتع معظم الاستراتيجيات الحالية الخاصة بتوليد SCA بترتيب عالٍ من التعقيد نظرًا لتوليد جميع التفاعلات التوافقية من خلال تبني أسلوب اختبار واحد في كل مرة. يعد الحد من التعقيد من خلال تبني عامل واحد في وقت واحد لتوليد SCA عملية صعبة. بالإضافة إلى ذلك، يوفر هذا الحد من التعقيد دعما للحصول على قوة تغطية أعلى. وبمواجهة هذا التحدي، تقترح هذه الورقة استراتيجية SCA جديدة تسمى ترتيب الاحداث الديناميكةDynamic Event Order (DEO) ، والتي يتم فيها إنشاء حالة الاختبار باستخدام عامل واحد في وقت واحد. يقدم هذا البحث  تفاصيل DEO مع مثال خطوة بخطوة لإظهار السلوك وإظهار صحة الاستراتيجية المقترحة. بالإضافة إلى ذلك، تقوم هذه الورقة بإجراء مقارنة مع الاستراتيجيات الحسابية الحالية. توضح النتائج العملية أن استراتيجية DEO المقترحة تتفوق على الاستراتيجيات الحالية من حيث الحد الأدنى لحجم الاختبار في معظم الحالات. علاوة على ذلك، تزداد أهمية DEO مع زيادة عدد التتابعات و / أو زيادة قوة التغطية. حيث نجحت استراتيجية DEO المقترحة في إنشاء SCAs حتى .t = 7 أخيرًا، نجحت إستراتيجية DEO في إيجاد حدود عليا جديدة ل .SCA في الواقع، أن الإستراتيجية المقترحة تعد قاعدة بحثية لتنفيذ وتطوير خوارزميات مستقبلية باعتماد فكرة العامل واحد في الوقت الواحد المنفذة.Sequence covering array (SCA) generation is an active research area in recent years. Unlike the sequence-less covering arrays (CA), the order of sequence varies in the test case generation process. This paper reviews the state-of-the-art of the SCA strategies, earlier works reported that finding a minimal size of a test suite is considered as an NP-Hard problem. In addition, most of the existing strategies for SCA generation have a high order of complexity due to the generation of all combinatorial interactions by adopting one-test-at-a-time fashion. Reducing the complexity by adopting one-parameter- at-a-time for SCA generation is a challenging process. In addition, this reduction facilitates the supporting for a higher strength of coverage. Motivated by such challenge, this paper proposes a novel SCA strategy called Dynamic Event Order (DEO), in which the test case generation is done using one-parameter-at-a-time fashion. The details of the DEO are presented with a step-by-step example to demonstrate the behavior and show the correctness of the proposed strategy. In addition, this paper makes a comparison with existing computational strategies. The practical results demonstrate that the proposed DEO strategy outperforms the existing strategies in term of minimal test size in most cases. Moreover, the significance of the DEO increases as the number of sequences increases and/ or the strength of coverage increases. Furthermore, the proposed DEO strategy succeeds to generate SCAs up to t=7. Finally, the DEO strategy succeeds to find new upper bounds for SCA. In fact, the proposed strategy can act as a research vehicle for variants future implementation

A Survey of Constrained Combinatorial Testing

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques

A hybrid kidney algorithm strategy for combinatorial interaction testing problem

Combinatorial Interaction Testing (CIT) generates a sampled test case set (Final Test Suite (FTS)) instead of all possible test cases. Generating the FTS with the optimum size is a computational optimization problem (COP) as well as a Non-deterministic Polynomial hard (NP-hard) problem. Recent studies have implemented hybrid metaheuristic algorithms as the basis for CIT strategy. However, the existing hybrid metaheuristic-based CIT strategies generate a competitive FTS size, there is no single CIT strategy can overcome others existing in all cases. In addition, the hybrid metaheuristic-based CIT strategies require more execution time than their own original algorithm-based strategies. Kidney Algorithm (KA) is a recent metaheuristic algorithm and has high efficiency and performance in solving different optimization problems against most of the state-of-the-art of metaheuristic algorithms. However, KA has limitations in the exploitation and exploration processes as well as the balancing control process is needed to be improved. These shortages cause KA to fail easily into the local optimum. This study proposes a low-level hybridization of KA with the mutation operator and improve the filtration process in KA to form a recently Hybrid Kidney Algorithm (HKA). HKA addresses the limitations in KA by improving the algorithm's exploration and exploitation processes by hybridizing KA with mutation operator, and improve the balancing control process by enhancing the filtration process in KA. HKA improves the efficiency in terms of generating an optimum FTS size and enhances the performance in terms of the execution time. HKA has been adopted into the CIT strategy as HKA based CIT Strategy (HKAS) to generate the most optimum FTS size. The results of HKAS shows that HKAS can generate the optimum FTS size in more than 67% of the benchmarking experiments as well as contributes by 34 new optimum size of FTS. HKAS also has better efficiency and performance than KAS. HKAS is the first hybrid metaheuristic-based CIT strategy that generates an optimum FTS size with less execution time than the original algorithm-based CIT strategy. Apart from supporting different CIT features: uniform/VS CIT, IOR CIT as well as the interaction strength up to 6, this study also introduces another recently variant of KA which are Improved KA (IKA) and Mutation KA (MKA) as well as new CIT strategies which are IKA-based (IKAS) and MKA-based (MKAS)

An orchestrated survey of available algorithms and tools for Combinatorial Testing

For functional testing based on the input domain of a functionality, parameters and their values are identified and a test suite is generated using a criterion exercising combinations of those parameters and values. Since software systems are large, resulting in large numbers of parameters and values, a technique based on combinatorics called Combinatorial Testing (CT) is used to automate the process of creating those combinations. CT is typically performed with the help of combinatorial objects called Covering Arrays. The goal of the present work is to determine available algorithms/tools for generating a combinatorial test suite. We tried to be as complete as possible by using a precise protocol for selecting papers describing those algorithms/tools. The 75 algorithms/tools we identified are then categorized on the basis of different comparison criteria, including: the test suite generation technique, the support for selection (combination) criteria, mixed covering array, the strength of coverage, and the support for constraints between parameters. Results can be of interest to researchers or software companies who are looking for a CT algorithm/tool suitable for their needs

Construction of Prioritized T-Way Test Suite Using Bi-Objective Dragonfly Algorithm

Software testing is important for ensuring the reliability of software systems. In software testing, effective test case generation is essential as an alternative to exhaustive testing. For improving the software testing technology, the t-way testing technique combined with metaheuristic algorithm has been great to analyze a large number of combinations for getting optimal solutions. However, most of the existing t-way strategies consider test case weights while generating test suites. Priority of test cases hasn’t been fully considered in previous works, but in practice, it’s frequently necessary to distinguish between high-priority and low-priority test cases. Therefore, the significance of test case prioritization is quite high. For this reason, this paper has proposed a t-way strategy that implements an adaptive Dragonfly Algorithm (DA) to construct prioritized t-way test suites. Both test case weight and test case priority have equal significance during test suite generation in this strategy. We have designed and implemented a Bi-objective Dragonfly Algorithm (BDA) for prioritized t-way test suite generation, and the two objectives are test case weight and test case priority. The test results demonstrate that BDA performs competitively against existing t-way strategies in terms of test suite size, and in addition, BDA generates prioritized test suites.©2022 Authors. Published by IEEE. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/fi=vertaisarvioitu|en=peerReviewed