A methodology of testing high-level Petri nets

Petri nets have been extensively used in the modelling and analysis of concurrent and distributed systems. The verification and validation of Petri nets are of particular importance in the development of concurrent and distributed systems. As a complement to formal analysis techniques, testing has been proven to be effective in detecting system errors and is easy to apply. An open problem is how to test Petri nets systematically, effectively and efficiently. An approach to solve this problem is to develop test criteria so that test adequacy can be measured objectively and test cases can be generated efficiently, even automatically. In this paper, we present a methodology of testing high-level Petri nets based on our general theory of testing concurrent software systems. Four types of testing strategies are investigated, which include state-oriented testing, transition-oriented testing, flow-oriented testing and specification-oriented testing. For each strategy, a set of schemes toobserve and record testing results and a set of coverage criteria to measure test adequacy are defined. The subsumption relationships and extraction relationships among the proposed testing methods are systematically investigated and formally proved

Applying adaptive test cases to nondeterministic implementations

The testing of a state-based system involves the application of sequences of inputs and the observation of the resultant input/output sequences (traces). These traces can result from preset input sequences or adaptive test cases in which the choice of the next input depends on the trace that has observed up to that input. Adaptive test cases are used in a number of areas including protocol conformance testing and adaptivity forms the basis of the standardised test language TTCN. Suppose that we apply adaptive test case ° to the system under test (SUT) and observe the trace ¯¾. If the SUT is deterministic and we apply ° again, after resetting the SUT, then we will observe ¯¾ again. Further, if we have another adaptive test case °0 where a prefix ¯¾0 of ¯¾ is a possible response to °0 then we know that the application of °0 must lead to ¯¾0. Thus, for a deterministic SUT the response of the SUT to an adaptive test case °0 might be deduced from the response of the SUT to another adaptive test case. This observation can be used to reduce the cost of testing: we only apply adaptive test case °0 if we cannot deduce the response to °0 from the set of observations. While many systems are deterministic, nondeterminism is becoming increasingly common. Nondeterminism in the SUT is typically a consequence of limits in the ability to observe the SUT. For example, it could be a result of information hiding, real time properties, or of different possible interleavings in a concurrent system (see, for example. This paper investigates the case where the SUT is nondeterministic. We consider the situation in which a set O of traces has been observed in testing and we are considering applying an adaptive test case °. In general we cannot expect to be able to deduce the response of a nondeterministic SUT to an adaptive test case ° since there may be more than one possible response. Instead we consider the question of how we can decide whether the application of ° could lead to a trace that has not been observed. A solution to this would allow us to reduce the cost of testing: if all possible responses of the SUT to ° have already been observed then we do not have to apply ° in testing and thus reduce the cost of test execution. This paper considers three cases. Section 3 considers the case where we can apply a fairness assumption. Section 4 weakens this assumption to us having a lower bound p on the probability of observing alternative responses of the SUT to any input and in any state. Section 5 then considers the general case

Моделирование дискретных управляющих систем с параллелизмом поведения

Objectives. The problem of functional verification of control devices with respect to their design specification is considered. When solving the problems of implementing and testing of discrete systems, one has to deal with the presence of parallelism in the behavior of interacting control objects, which is also displayed in the assignment for designing control systems. The aim of the work is to develop a method for simulating descriptions of such systems, which allows their behavior testing dynamically on the area limited by their possible functioning.Methods. The paper considers a class of control systems with parallelism of the processes occurring in them, which permits linearization of their execution. To specify the behavior of such control systems, it is proposed to use the PRALU language of parallel control algorithms, which is based on Petri nets and which allows to order events occurring during the device operation. An object-oriented approach to simulation of the description of the control algorithm at the transaction level is proposed. For this purpose, a TLM (Transaction-Level Modeling) model has been developed for describing the devices with behavior parallelism in PRALU language. The transaction level model describes a system as a set of interacting processes that run in parallel and determine the behavior of the system over time.Results. The key concepts of the TLM model for simulating the descriptions of control algorithms in the PRALU language are defined: data structure, transactions, processes, and a barrier mechanism for synchronization of parallel processes. A method is proposed for transforming the description of an algorithm in the language into a TLM model, which is based on the representation of language operations as compositions of elementary operations that are performed sequentially. The set of these operations forms the basis for the algorithmic decomposition of a parallel algorithm in PRALU language into intermediate language program that is executed strictly sequentially. Translators of this program into the Verilog and C languages have been developed, the results of their compilation are simulators of the behavior of control system.Conclusion. The proposed simulation method can be used to create a test bench for functional verification of the circuit implementation of control devices with behavior parallelism. In this case, test sequences for verifying the circuit implementation can be generated dynamically – in the process of simulating the description of the algorithm in the PRALU language directly the control device or system, which include the control algorithm and the algorithms of controlled objects behavior.Цели. Рассматривается задача функциональной верификации устройств управления относительно спецификации на их проектирование. При решении задач реализации и тестирования дискретных систем приходится иметь дело с наличием параллелизма в поведении взаимодействующих объектов управления, что отображается также и в задании на проектирование устройств управления ими. Цель исследования заключается в разработке метода имитационного моделирования описаний дискретных систем, который позволяет динамически тестировать поведение таких систем на области, ограниченной их возможным функционированием.Методы. В работе рассматривается класс систем управления с параллелизмом происходящих в них процессов, позволяющим линеаризовать их выполнение. Для задания спецификации таких систем управления предлагается использовать язык ПРАЛУ параллельных алгоритмов управления, в основе которого лежат сети Петри и который позволяет упорядочивать во времени события, происходящие в процессе работы устройства. Предлагается объектно-ориентированный подход к моделированию описания алгоритма управления на уровне транзакций. Для этого разработана модель TLM (Transaction-Level Modeling) описаний на языке ПРАЛУ устройств с параллелизмом поведения. Модель уровня транзакций описывает систему набором взаимодействующих процессов, которые выполняются параллельно и определяют ее поведение во времени.Результаты. Определены ключевые понятия модели TLM для моделирования описаний алгоритмов управления на языке ПРАЛУ: структура данных, транзакции, процессы и барьерной механизм синхронизации параллельно выполняющихся процессов. Предложен метод преобразования описания алгоритма на языке ПРАЛУ в модель TLM, который основан на представлении операций языка в виде композиций элементарных операций, выполняющихся последовательно. Набор таких операций составляет базис алгоритмического разложения параллельного алгоритма на языке ПРАЛУ в программу на промежуточном языке, которая выполняется строго последовательно. Разработаны трансляторы этой программы на языки Verilog и C, результаты их компиляции представляют симуляторы поведения системы управления.Заключение. Предложенный метод имитационного моделирования может быть использован при создании испытательного стенда для функциональной верификации схемной реализации устройств управления с параллелизмом поведения. При этом тестовые последовательности для верификации схемной реализации могут генерироваться динамически – в процессе моделирования описания алгоритма на языке ПРАЛУ непосредственно устройства управления или системы, включающей алгоритм управления и алгоритмы поведения управляемых объектов

Model-based testing of PLC programs with appropriate conformance relations

International audienceNumerous theoretical results have been obtained in the field of conformance testing, a very promising formal technique to improve dependability of critical systems. Nevertheless, developing on this basis PLC test techniques that produce correct conformance verdicts requires to take into account the real technological features of PLC. This paper proposes conformance relations that meet this objective. Examples illustrate the benefits of the contribution

Test Modeling of Dynamic Variable Systems using Feature Petri Nets

In order to generate substantial market impact, mobile applications must be able to run on multiple platforms. Hence, software engineers face a multitude of technologies and system versions resulting in static variability. Furthermore, due to the dependence on sensors and connectivity, mobile software has to adapt its behavior accordingly at runtime resulting in dynamic variability. However, software engineers need to assure quality of a mobile application even with this large amount of variability—in our approach by the use of model-based testing (i.e., the generation of test cases from models). Recent concepts of test metamodels cannot efficiently handle dynamic variability. To overcome this problem, we propose a process for creating black-box test models based on dynamic feature Petri nets, which allow the description of configuration-dependent behavior and reconfiguration. We use feature models to define variability in the system under test. Furthermore, we illustrate our approach by introducing an example translator application

Using transfer-resource graph for software-based verification of system-on-chip

Copyright © 2006 IEEEThe verification of system-on-chip is challenging due to its high level of integration. Multiple components in a system can behave concurrently and compete for resources. Hence, for simulation-based verification, we need a methodology that allows one to automatically generate test cases for testing concurrent and resource-competing behaviors.We introduce the use of a transferresource graph (TRG) as the model for test generation. From a high abstraction level, TRG is able to model the parallelism between heterogeneous interaction forms in a system. We show how TRG is used in generating test cases of resource competitions and how these test cases are structured in event-driven test programs. For coverage, TRG can be converted to a Petri net, allowing one to measure the completeness of concurrency in simulation.Xiaoxi Xu and Cheng-Chew Li

Test Case Generation and Optimization of Object-Oriented Software using UML Behavioral Models

Testing guarantees the quality of software to be developed in terms of presence of bugs or errors. Testing can be separated into two categories such as White Box and Black Box testing. White box testing is done through detail analysis of program structure where as black box methodology deals with specification and design document i.e. without program details. Thus black box testing methodology holds major advantages, as tester can generate the test cases before code is developed, using specification and design document. Off the late, Object-Oriented program have changed the scenario of software development industry in terms of software development and its supporting technology. The object-oriented features like inheritance and encapsulation has made it easy and suitable confined to design. The inheritance feature encourages to re-use the developed components where as the encapsulation conceals the details from others. And other features of object-oriented program like polymorphism, data abstraction and modularity have increased its richness. However these features have increased the job of software tester. Special attraction are needed to look into these features while testing is carried out

Model based test suite minimization using metaheuristics

Software testing is one of the most widely used methods for quality assurance and fault detection purposes. However, it is one of the most expensive, tedious and time consuming activities in software development life cycle. Code-based and specification-based testing has been going on for almost four decades. Model-based testing (MBT) is a relatively new approach to software testing where the software models as opposed to other artifacts (i.e. source code) are used as primary source of test cases. Models are simplified representation of a software system and are cheaper to execute than the original or deployed system. The main objective of the research presented in this thesis is the development of a framework for improving the efficiency and effectiveness of test suites generated from UML models. It focuses on three activities: transformation of Activity Diagram (AD) model into Colored Petri Net (CPN) model, generation and evaluation of AD based test suite and optimization of AD based test suite. Unified Modeling Language (UML) is a de facto standard for software system analysis and design. UML models can be categorized into structural and behavioral models. AD is a behavioral type of UML model and since major revision in UML version 2.x it has a new Petri Nets like semantics. It has wide application scope including embedded, workflow and web-service systems. For this reason this thesis concentrates on AD models. Informal semantics of UML generally and AD specially is a major challenge in the development of UML based verification and validation tools. One solution to this challenge is transforming a UML model into an executable formal model. In the thesis, a three step transformation methodology is proposed for resolving ambiguities in an AD model and then transforming it into a CPN representation which is a well known formal language with extensive tool support. Test case generation is one of the most critical and labor intensive activities in testing processes. The flow oriented semantic of AD suits modeling both sequential and concurrent systems. The thesis presented a novel technique to generate test cases from AD using a stochastic algorithm. In order to determine if the generated test suite is adequate, two test suite adequacy analysis techniques based on structural coverage and mutation have been proposed. In terms of structural coverage, two separate coverage criteria are also proposed to evaluate the adequacy of the test suite from both perspectives, sequential and concurrent. Mutation analysis is a fault-based technique to determine if the test suite is adequate for detecting particular types of faults. Four categories of mutation operators are defined to seed specific faults into the mutant model. Another focus of thesis is to improve the test suite efficiency without compromising its effectiveness. One way of achieving this is identifying and removing the redundant test cases. It has been shown that the test suite minimization by removing redundant test cases is a combinatorial optimization problem. An evolutionary computation based test suite minimization technique is developed to address the test suite minimization problem and its performance is empirically compared with other well known heuristic algorithms. Additionally, statistical analysis is performed to characterize the fitness landscape of test suite minimization problems. The proposed test suite minimization solution is extended to include multi-objective minimization. As the redundancy is contextual, different criteria and their combination can significantly change the solution test suite. Therefore, the last part of the thesis describes an investigation into multi-objective test suite minimization and optimization algorithms. The proposed framework is demonstrated and evaluated using prototype tools and case study models. Empirical results have shown that the techniques developed within the framework are effective in model based test suite generation and optimizatio