A transformation-based approach to testing concurrent programs using UML activity diagrams

UML activity diagrams are widely used to model concurrent interaction among multiple objects. In this paper, we propose a transformation-based approach to generating scenario-oriented test cases for applications modeled by UML activity diagrams. Using a set of transformation rules, the proposed approach first transforms a UML activity diagram specification into an intermediate representation, from which it then constructs test scenarios with respect to the given concurrency coverage criteria. The approach then finally derives a set of test cases for the constructed test scenarios. The approach resolves the difficulties associated with fork and join concurrency in the UML activity diagram, and enables control over the number of the resulting test cases. We further implemented a tool to automate the proposed approach, and studied its feasibility and effectiveness using a case study. Experimental results show that the approach can generate test cases on demand to satisfy a given concurrency coverage criterion, and can detect up to 76.5% of seeded faults when a weak coverage criterion is used. With the approach, testers can not only schedule the software test process earlier, but can also better allocate the testing resources for testing concurrent applications

Multi-objective construction of an entire adequate test suite for an EFSM

In this paper we propose a method and a tool to generate test suites from extended finite state machines, accounting for multiple (potentially conflicting) objectives. We aim at maximizing coverage and feasibility of a test suite while minimizing similarity between its test cases and minimizing overall cost. Therefore, we define a multi-objective genetic algorithm that searches for optimal test suites based on four objective functions. In doing so, we create an entire test suite at once as opposed to test cases one at a time. Our approach is evaluated on two different case studies, showing interesting initial results

Pembentukan Data Uji Menggunakan Algoritma Optimisasi Koloni Semut dan Pendekatan Teknik Pengujian Kotak Abu-Abu

Pengujian perangkat lunak dapat menjadi cara yang efektif untuk memperbaiki kualitas serta ketahanan perangkat lunak. Secara umum pengujian perangkat lunak dapat dilakukan dengan teknik kotak putih (white box) dan kotak hitam (black box). Teknik pengujian kotak abu-abu (gray box) dikenal sebagai teknik penguijan yang menggunakan pendekatan teknik pengujian kotak putih dan kotak hitam. Teknikpengujian kotak abu-abu memperluas kriteria cakupan logika dari teknik pengujian kotak putih dan menemukan semua kemungkinannya dari model desain perangkat lunak sepertipada teknik pengujian kotak hitam. Teknik pengujian kotak abu-abu menggunakan data seperti: UML Diagram, Model Arsitektur perangkat lunak, ataupun Finite State Machine Diagram (State Model) untuk membentuk kasus pengujian. Selain itu pengujian perangkat lunak dapat dilakukan dengan teknik Soft Computing dan Hard Computing. Teknik Hard Computing sulit diimplementasikan pada problematika yang ada saat ini. Sehinga teknik Soft Computing dapat menjadi alternatif yang digunakan dalam pengujian perangkat lunak. Teknik Soft Computing merupakan teknik yang lebih berfokus pada menginterpretasikan perilaku sistem dari pada hasil presisi. Teknik Soft Computing biasanya berdasarkan teknik logika fuzzy, jaringan saraf tiruan ataupun pengambilan keputusan berdasarkan nilai distribusi kemungkinan. Teknik pengujian perangkat lunak yang mengunakanteknil Soft Computing denganmetode pengambilan sampel berdasarkan nilai kemungkinan dikenal dengan teknik pengujian statistika. Berdasarkan penelitian terkait, algoritma Ant Colony Optimization (ACO) atau optimisasi koloni semut merupakan algoritma yang digunakan pada teknik pengujian statistika untuk membentuk data uji dengan kemampuan yang lebih baik dibandingkan algoritma lain seperti: Simulated Annealing (SA) serta algoritma genetika. Selain itu ACO juga memiliki hasil yang sebanding dengan algoritma Particle Swarm Optimization (PSO) atau optimisasi kawanan partikel. ACO diimplementasikan pada kode program perangkat lunak yang diuji untuk membentuk data uji berdasarkan nilai kemungkinan terbesar random data uji dari domain terpilih. Pemilihan data uji merupakan faktor utama yang menentukan keberhasilan dari suatu pengujian perangkat lunak. Sehingga pemilihan teknik yang tepat dapat membantu menunjang keberhasilan dalam pengujian perangkat lunak.Pada penelitian ini, ACO diimplementasikan berdasarkan teknik pengujian kotak abu-abu menggunakan diagram UML State Machine. Pembentukan data uji yang berkualitas adalah berdasarkan kecukupan kriteria percabangan yang dapat ditelusuri. Tujuan dari penelitian ini adalah untuk mendapatkan hasil perbandingan pembentukan data uji dengan teknik pengujian kotak abu-abu menggunakan diagram UML State Machine dan teknik pengujian struktur kotak putih menggunakan kode program. Hasil penelitian ini diharapkan mampu memberikan gambaran kualitas data uji yang dibentuk dari masing-masing teknik ========================================================================================================Software testing can be an effective way to improve software quality and reliability. In general, software testing techniques can be classified into White Box and Black Box. Gray Box testing technique is known as testing technique which is used both White Box and Black Box techniques. It extends the logical coverage criteria of white box method and finds all the possibility from the design model which is like black box method. Gray Box technique uses data such as: UML Diagram, Software Architecture Model, or Finite State Machine Diagram (State Model) to generate test cases. The other classification of software testing technique is Hard Computing technique and Soft Computing technique. Hard Computing technique is difficult to be implemented in today problems. And Soft Computing technique can be an alternative way which can be used in software testing. Soft Computing technique focuses on system behavior interpretation than precision result. Soft Computing technique is based on fuzzy logic, neural network, or decision making by probability distribution. Software testing which is used Soft Computing technique and its sampling method based on probability distribution, is known as statistical testing. Based on research, Ant Colony Optimization (ACO) Algorithm is algorithm which is used in statistical testing to generate test data, and its result better than another algorithm such as: Simulated Annealing (SA), and Genetic algorithm. Besides, its result is comparable with Paricle Swarm Optimization (PSO) algorithm. ACO was implemented in program code of software under test to generate test data based on the highest probability value of random test data from domain chosen. Test data selection is main factor which determines the software testing success. In this research, ACO was implemented based on Gray Box testing using UML State Machine Diagram. The quality of test data generated is based on branch coverage criteria. This research aims to get comparison result between Gray Box testing using UML State Machine Diagram and structural White Box Testing using program code in generating test data. The result of this research is expected to give description about the quality of test data generated from each technique

Object Constraint Language Based Test Case Optimisation

Software testing, a pivotal phase in the Software Develop- ment Life Cycle (SDLC), ensures the correctness and perfor- mance of the corresponding software system. Model-based testing (MBT) is a method to validate whether the software system or specification satisfies the pre-defined requirements through design models. However, with modern systems ex- panding in complexity, testing has become a labour-intensive and unpredictable process within the SDLC. Therefore, many test case optimisation (TCO) techniques have been proposed to make the testing process more manageable. But these ap- proaches predominantly focus on code-based strategies, leav- ing systems expressed in Object Constraint Language (OCL) underserved. OCL is a part of the Unified Modeling Lan- guage (UML) standard and is a type of declarative language used to describe system specification by pre- and post- con- ditions. Initially, OCL has been proposed as a constraint language to add more details to the UML model, but along- side the development of OCL itself, there are more and more systems whose specifications are expressed in OCL.This thesis aims to systematically investigate the feasibil- ity of applying TCO techniques to the OCL-defined systems, with an emphasis on test case prioritisation (TCP) and test case minimisation (TCM) processes. A systematic literature review for the directly related topic, UML-based test case generation, is conducted in this thesis. Also, we adapted a set of test case optimisation algorithms and compared the performance between these algorithms under the context of OCL. Moreover, we modified one metric for the TCP evalu- ation process, which made the metric more suitable for the MBT and mutation testing environment. Furthermore, we introduce a full set of mutation operators and corresponding classifications to the OCL standard library, offering practical guidance for optimisation processes.The proposed TCO processes are validated and evalu- ated through four real-world systems expressed in OCL with different complexities. The experiment results demonstrate that for the TCM process, the size of the minimised test suite is reduced from 33.33% to 81.8% without losing any fault de- tection ability. For the TCP process, leveraging the modified evaluation metric, the improvements are up to 50%, indicat- ing that the prioritised test suite can detect system defects earlier when compared to the original one. Evaluating based on the considerations of effectiveness, efficiency and stability, we suggest the NSGA-II for the TCM process and the genetic algorithm for the TCP process. When combining TCP and TCM processes, the TCM process consistently increases the efficiency of the TCP process by reducing the search space for the prioritisation process

Model based test suite minimization using metaheuristics

Software testing is one of the most widely used methods for quality assurance and fault detection purposes. However, it is one of the most expensive, tedious and time consuming activities in software development life cycle. Code-based and specification-based testing has been going on for almost four decades. Model-based testing (MBT) is a relatively new approach to software testing where the software models as opposed to other artifacts (i.e. source code) are used as primary source of test cases. Models are simplified representation of a software system and are cheaper to execute than the original or deployed system. The main objective of the research presented in this thesis is the development of a framework for improving the efficiency and effectiveness of test suites generated from UML models. It focuses on three activities: transformation of Activity Diagram (AD) model into Colored Petri Net (CPN) model, generation and evaluation of AD based test suite and optimization of AD based test suite. Unified Modeling Language (UML) is a de facto standard for software system analysis and design. UML models can be categorized into structural and behavioral models. AD is a behavioral type of UML model and since major revision in UML version 2.x it has a new Petri Nets like semantics. It has wide application scope including embedded, workflow and web-service systems. For this reason this thesis concentrates on AD models. Informal semantics of UML generally and AD specially is a major challenge in the development of UML based verification and validation tools. One solution to this challenge is transforming a UML model into an executable formal model. In the thesis, a three step transformation methodology is proposed for resolving ambiguities in an AD model and then transforming it into a CPN representation which is a well known formal language with extensive tool support. Test case generation is one of the most critical and labor intensive activities in testing processes. The flow oriented semantic of AD suits modeling both sequential and concurrent systems. The thesis presented a novel technique to generate test cases from AD using a stochastic algorithm. In order to determine if the generated test suite is adequate, two test suite adequacy analysis techniques based on structural coverage and mutation have been proposed. In terms of structural coverage, two separate coverage criteria are also proposed to evaluate the adequacy of the test suite from both perspectives, sequential and concurrent. Mutation analysis is a fault-based technique to determine if the test suite is adequate for detecting particular types of faults. Four categories of mutation operators are defined to seed specific faults into the mutant model. Another focus of thesis is to improve the test suite efficiency without compromising its effectiveness. One way of achieving this is identifying and removing the redundant test cases. It has been shown that the test suite minimization by removing redundant test cases is a combinatorial optimization problem. An evolutionary computation based test suite minimization technique is developed to address the test suite minimization problem and its performance is empirically compared with other well known heuristic algorithms. Additionally, statistical analysis is performed to characterize the fitness landscape of test suite minimization problems. The proposed test suite minimization solution is extended to include multi-objective minimization. As the redundancy is contextual, different criteria and their combination can significantly change the solution test suite. Therefore, the last part of the thesis describes an investigation into multi-objective test suite minimization and optimization algorithms. The proposed framework is demonstrated and evaluated using prototype tools and case study models. Empirical results have shown that the techniques developed within the framework are effective in model based test suite generation and optimizatio

Generation of Software Test Data from the Design Specification Using Heuristic Techniques. Exploring the UML State Machine Diagrams and GA Based Heuristic Techniques in the Automated Generation of Software Test Data and Test Code.

Software testing is a tedious and very expensive undertaking. Automatic test data generation is, therefore, proposed in this research to help testers reduce their work as well as ascertain software quality. The concept of test driven development (TDD) has become increasingly popular during the past several years. According to TDD, test data should be prepared before the beginning of code implementation. Therefore, this research asserts that the test data should be generated from the software design documents which are normally created prior to software code implementation. Among such design documents, the UML state machine diagrams are selected as a platform for the proposed automated test data generation mechanism. Such diagrams are selected because they show behaviours of a single object in the system. The genetic algorithm (GA) based approach has been developed and applied in the process of searching for the right amount of quality test data. Finally, the generated test data have been used together with UML class diagrams for JUnit test code generation. The GA-based test data generation methods have been enhanced to take care of parallel path and loop problems of the UML state machines. In addition the proposed GA-based approach is also targeted to solve the diagrams with parameterised triggers. As a result, the proposed framework generates test data from the basic state machine diagram and the basic class diagram without any additional nonstandard information, while most other approaches require additional information or the generation of test data from other formal languages. The transition coverage values for the introduced approach here are also high; therefore, the generated test data can cover most of the behaviour of the system.EU Asia-Link project TH/Asia Link/004(91712) East-West and CAM

Fail-Safe Test Generation of Safety Critical Systems

This dissertation introduces a technique for testing proper failure mitigation in safety critical systems. Unlike other approaches which integrate behavioral and failure models, and then generate tests from the integrated model, we build safety mitigation tests from an existing behavioral test suite, using an explicit mitigation model for which we generate mitigation paths which are then woven at selected failure points into the original test suite to create failure-mitigation tests (safety mitigation test)