Designing A Combinatorial Java Unit Testing Tool.

Software testing relates to the process of executing a program or system with the intent of finding errors. Covering as much as 40 to 50 percent of the development costs and resources, software testing is an integral part of the software development lifecycle

CTJ: Input-Output Based Relation Combinatorial Testing Strategy Using Jaya Algorithm

            ويكاد يكون من المستحيل اختبار كل مجموعة من المدخلات نظرًا لأن تنفيذ حالات الاختبار يتطلب وقتا طويلا للغاية. الأختبار الاندماجي هو السبيل لتخطي عقبات الاختبار الشامل من خلال أختبار كل قيم المدخلات لكل المعاملات المركبة المتعددة طرق الترتيب.   يمكن تقسيم الاختبار التجميعي إلى ثلاثة أنواع هي تفاعل القوة الموحد ، والتفاعل المتغير والقوة ، والعلاقة القائمة على المدخلات والمخرجات . ان الطريقة الاخيرة الانفة الذكر تختزل الفحص الاندماجي الى مجموعة ضمن اختيار الشخص الفاحص. معظم الابحاث في الاختبار الاندماجي طبقت في تفاعل القوة الموحدة وقوة التفاعل المتغيرة ، ومع ذلك ، هناك اهتمام قليل جدا بالعلاقة بين المدخلات والمخرجات. لذا تم اقتراح خوارزمية جايا في هذا البحث  كخوارزمية مثلي لانشاء جدول الفحص الاندماجي باستراتيجية تعتمد على العلاقة بين المدخلات والمخرجات. نتيجة تطبيق خوارزمية جايا في الاختبار الاندماجي القائم على المدخلات والمخرجات مقبولة لأنها تنتج العدد الأمثل تقريبًا لحالات الاختبار في نطاق زمني مقبول.Software testing is a vital part of the software development life cycle. In many cases, the system under test has more than one input making the testing efforts for every exhaustive combination impossible (i.e. the time of execution of the test case can be outrageously long). Combinatorial testing offers an alternative to exhaustive testing via considering the interaction of input values for every t-way combination between parameters. Combinatorial testing can be divided into three types which are uniform strength interaction, variable strength interaction and input-output based relation (IOR). IOR combinatorial testing only tests for the important combinations selected by the tester. Most of the researches in combinatorial testing applied the uniform and the variable interaction strength, however, there is still a lack of work addressing IOR. In this paper, a Jaya algorithm is proposed as an optimization algorithm engine to construct a test list based on IOR in the proposed combinatorial test list generator strategy into a tool called CTJ. The result of applying the Jaya algorithm in input-output based combinatorial testing is acceptable since it produces a nearly optimum number of test cases in a satisfactory time range

Constructing interaction test suites with greedy algorithms

Combinatorial approaches to testing are used in several fields, and have recently gained momentum in the field of software testing through software interaction testing. One-test-at-a-time greedy algorithms are used to automatically construct such test suites. This paper discusses basic criteria of why greedy algorithms have been appropriate for this test gen-eration problem in the past and then expands upon how greedy algorithms can be utilized to address test suite pri-oritization

Artificial Bee Colony Algorithm for Pairwise Test Generation

Our dependence on software applications has become dramatic in many activities of our daily life as they help to increase the efficiency of our tasks. These software applications have many sets of input values, parameters, software/hardware environments and system conditions, which need to be tested to ensure software reliability and quality. However, the whole comprehensive software testing is virtually not possible due to marketing pressure and resource constraints. In an attempt to solve this problem, there has been a development of a number of sampling and pairwise strategies in the literature. In this paper, we evaluated and proposed a pairwise strategy named Pairwise Artificial Bee Colony algorithm (PABC). According to the benchmarking results, the PABC strategies outdo some existing strategies to generate a test case in many of the system configurations taken into consideration. In a case where PABC is not at its optimal stage or its best performance, the experiments of a test case are effectively competitive. PABC progresses as a means to achieve the effective use of the artificial bee colony algorithm for pairwise testing reduction

A Survey of Constrained Combinatorial Testing

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques

An orchestrated survey of available algorithms and tools for Combinatorial Testing

For functional testing based on the input domain of a functionality, parameters and their values are identified and a test suite is generated using a criterion exercising combinations of those parameters and values. Since software systems are large, resulting in large numbers of parameters and values, a technique based on combinatorics called Combinatorial Testing (CT) is used to automate the process of creating those combinations. CT is typically performed with the help of combinatorial objects called Covering Arrays. The goal of the present work is to determine available algorithms/tools for generating a combinatorial test suite. We tried to be as complete as possible by using a precise protocol for selecting papers describing those algorithms/tools. The 75 algorithms/tools we identified are then categorized on the basis of different comparison criteria, including: the test suite generation technique, the support for selection (combination) criteria, mixed covering array, the strength of coverage, and the support for constraints between parameters. Results can be of interest to researchers or software companies who are looking for a CT algorithm/tool suitable for their needs

Verificación de Covering Arrays utilizando Supercomputación y computación Grid

En esta tesis se presentan un algorimo secuencial, un algoritmo paralelo y uno algoritmo grid para verificar si una matriz es un covering array (CA). Los algoritmos fueros probados usando un benchmak de CAs de fuerza variable. La conclusión principal de este trabajo radica en la identificación de las fortalezas y debilidades de los algoritmos.Avila George, H. (2010). Verificación de Covering Arrays utilizando Supercomputación y computación Grid. http://hdl.handle.net/10251/14486Archivo delegad

The Effectiveness of <i>t</i>-Way Test Data Generation

Modern society is increasingly dependent on the correct functioning of software and increasingly so in areas that are considered safety related or safety critical. Therefore, there is an increasing need to be able to verify and validate that the software is in fact correct and will perform its intended function. Many approaches to this problem have been proposed; however, none seems likely to supplant the role of testing in the near future. If we accept that there is, and will be, a continuing need to be able to test software then the question becomes one of how can this be done effectively, both in terms of ability to detect errors and in terms of cost. One avenue of research that offers prospects of improving both of these aspects is the automatic generation of test data. There has recently been a large amount of work conducted in this area. One particularly promising direction has been the application of ideas from the field of experimental design and in particular, the field of t-way adequate factorial designs. The area however, is not without issues; there is evidence that the technique is capable of detecting errors but that evidence is not unequivocal. Moreover, as with almost all work in the area of automatic test generation, there has been very little comparative work comparing the technique with other test data generation techniques. Worse, there has been effectively no work done that compares any automatic test data generation technique with the effectiveness of tests generated by humans. Another major issue with the technique is the number of tests that applying the technique can result in. This implies that there is a need for an automated oracle if the technique is to be successfully applied. The flaw with this is of course that in most situations the oracle is the human that is conducting the tests, a point often ignored in testing research. The work presented here addresses both of these points. To do this I have used a code base taken from an industrial engine control system that has an existing set of high quality unit tests developed by hand. To complement this, several other techniques for automatically generating test data have been applied, namely random testing, random experimental designs and a technique for generating single factor experiments. To address the issue of being able to compare the error detection ability of all of the sets of test vectors, rather than the usual effectiveness surrogates of code coverage I have used mutation analysis on the code base to directly measure the ability of each set of test vectors to discover common coding errors. The results presented here show that test data generation techniques based on t-way factorial designs are at least as effective as handgenerated tests and superior to random testing and the factor experimental technique. The oracle problem associated with the factorial design techniques was addressed using a test set minimisation approach. The mutation tool monitored which vectors could “kill” which code mutants. After a subset of the test vectors had been run, the most effective vectors were retained and the rest discarded. Likewise, mutants that were killed were removed from further consideration and the process repeated. Experimental results show that this minimisation procedure is effective at reducing computational overhead and is capable of producing final sets of test vectors that are comparable in size with the sets of hand-generated tests and so amenable to final hand checking

Approche à contraintes pour la sélection de Covering Array

Aujourd'hui, les éditeurs logiciels ne conçoivent, développent et ne maintiennent plus leur offre logicielle avec comme cible un client unique. Au contraire, les offres logicielles sont conçues pour cibler plusieurs entités. Par conséquent, ces applications doivent s'intégrer dans des environnements différents et s'adapter aux besoins des clients. Ainsi, les produits logiciels développés ne sont plus des programmes uniques, mais des familles de produits. Les systèmes configurables facilitent la création de ces familles de produits. Grâce à eux il est possible de créer un produit logiciel en sélectionnant les fonctionnalités qui seront intégrées. Cependant, la validation de ces systèmes est une tâche complexe. Un système configurable peut générer plusieurs millions de configurations possibles. Il ne s'agit donc plus de valider un seul et unique produit, mais un ensemble de produits. Cet important nombre de configurations est un problème pour les personneschargées de la validation. Nous proposons trois contributions qui visent à mieux répondre aux problématiques liées à la variabilité lors des projets de test: une présentation détaillée de deux projets de test industriels faisant face à des problématiques de variabilité issus de deux entreprises : Cisco et Orange; une méthode originale basée sur les techniques de programmation par contraintes pour extraire des configurations de test qui respectent le critère Pairwise à partir d'un modèle explicite de la variabilité; une comparaison de cette approche par rapport aux techniques de l'état de l'art et une étude de l'application de cette technique de test sur deux projets de tests industriels.Nowadays, software companies develop and maintain their software for several clients. Consequently, these applications have to be integrated in heterogenous context and adapt to the user requriements. All these products are sharing commonalities but also differ in certain point due to business specific constraints. Configurable systems facilitate the creation of these product families. With them it is possible to create a software product by selecting the features that will be integrated, thus, the creation of a product is greatly simplified. However, the validation of these systems is a complex task. A configurable system can generate millions of possible configurations. Thus, validation process doesn't consist in validating a single product but in validating a set of products. This large number of configurations is a problem for those responsible of the validation. In this thesis we propose three contributions that aim to solve issues raised by variability during test projects : A detailled presentation of two industrial test projects coping tat variaibility issues; an original methodology based on constraint programming techniques to select test configurations that respect pairwise criteria from a feature model; an exhaustive comparison of this approach with the existing approches and a detailled study of the application of a such techniques on the two industrials projects.RENNES1-Bibl. électronique (352382106) / SudocSudocFranceF

A Deterministic Density Algorithm for Pairwise Interaction Coverage

Pairwise coverage of factors affecting software has been proposed to screen for potential errors. Techniques to generate test suites for pairwise coverage are evaluated according to many criteria. A small number of tests is a main criterion, as this dictates the time for test execution. Randomness has been exploited to search for small test suites, but variation occurs in the test suite produced. A worst-case guarantee on test suite size is desired; repeatable generation is often necessary. The time to construct the test suite is also important. Finally, testers must be able to include certain tests, and to exclude others