A subset of precise UML for Model-based Testing

This paper presents an original model-based testing approach that takes a UML behavioural view of the system under test and automatically generates test cases and executable test scripts according to model coverage criteria. This approach is embedded in the LEIRIOS Test Designer tool and is currently deployed in domains such as Enterprise IT and electronic transaction applications. This model-based testing approach makes it possible to automatically produce the traceability matrix from requirements to test cases as part of the test generation process. This paper defines the subset of UML used for model-based testing and illustrates it using a small example

Towards Symbolic Model-Based Mutation Testing: Combining Reachability and Refinement Checking

Model-based mutation testing uses altered test models to derive test cases that are able to reveal whether a modelled fault has been implemented. This requires conformance checking between the original and the mutated model. This paper presents an approach for symbolic conformance checking of action systems, which are well-suited to specify reactive systems. We also consider nondeterminism in our models. Hence, we do not check for equivalence, but for refinement. We encode the transition relation as well as the conformance relation as a constraint satisfaction problem and use a constraint solver in our reachability and refinement checking algorithms. Explicit conformance checking techniques often face state space explosion. First experimental evaluations show that our approach has potential to outperform explicit conformance checkers.Comment: In Proceedings MBT 2012, arXiv:1202.582

Test generation from P systems using model checking

This paper presents some testing approaches based on model checking and using different testing criteria. First, test sets are built from different Kripke structure representations. Second, various rule coverage criteria for transitional, non-deterministic, cell-like P systems, are considered in order to generate adequate test sets. Rule based coverage criteria (simple rule coverage, context-dependent rule coverage and variants) are defined and, for each criterion, a set of LTL (Linear Temporal Logic) formulas is provided. A codification of a P system as a Kripke structure and the sets of LTL properties are used in test generation: for each criterion, test cases are obtained from the counterexamples of the associated LTL formulas, which are automatically generated from the Kripke structure codification of the P system. The method is illustrated with an implementation using a specific model checker, NuSMV. (C) 2010 Elsevier Inc. All rights reserved

Test Input Generation for Red-Black Trees using Abstraction

We consider the problem of test input generation for code that manipulates complex data structures. Test inputs are sequences of method calls from the data structure interface. We describe test input generation techniques that rely on state matching to avoid generation of redundant tests. Exhaustive techniques use explicit state model checking to explore all the possible test sequences up to predefined input sizes. Lossy techniques rely on abstraction mappings to compute and store abstract versions of the concrete states; they explore under-approximations of all the possible test sequences. We have implemented the techniques on top of the Java PathFinder model checker and we evaluate them using a Java implementation of red-black trees

Applicability of MIL-HDBK-516B to Certifying Autonomous Decision-Making Air Vehicle Systems

Airworthiness certification of military aircraft is accomplished by the developing military service. Air Force programs use the qualitative criteria outlined in MIL-HDBK-516B, “ASC/ EN Airworthiness Certification Criteria Expanded Version of MIL-HDBK-516B” (September 26, 2005) to aid the development of program-specific airworthiness criteria. The generalized criteria in this document are used to construct the specific criterion and associated artifacts — evidence of compliance — as the basis for making an airworthiness determination. This paper describes the process of transitioning from qualitative to specific criteria, and then examines the applicability of the existing guidance in MIL-HDBK-516B to autonomous decision-making adaptive air vehicle systems. Recommendations are made for future research and criteria expansion. An integrated approach that uses the most promising emerging and existing design, analysis, and validation and verification techniques is proposed as a means to develop the artifacts for certification coverage of autonomous adaptive unmanned air vehicle systems

Automatic test selection based on CEFSM specifications

Mutation analysis is a fault based testing method used initially for code based software testing. In this paper, this method is applied to formal specifications and used for automatic conformance test selection. This paper defines formally a set of mutation operators for CEFSM (Communicating Extended Finite State Machine) systems to enable the automated creation of mutant specifications. Mutants of a specification are used as selection criteria to pick out adequate test cases. Two different algorithms are proposed for the generation and selection of efficient test suites. Additionally, the operators and algorithms provide the basis of an automatic tool developed at the Budapest University of Technology and Economics. We present the results of an empirical study on the well-known INRES protocol acquired using the tool

A Comprehensive Safety Engineering Approach for Software-Intensive Systems Based on STPA

Formal verification and testing are complementary approaches which are used in the development process to verify the functional correctness of software. However, the correctness of software cannot ensure the safe operation of safety-critical software systems. The software must be verified against its safety requirements which are identified by safety analysis, to ensure that potential hazardous causes cannot occur. The complexity of software makes defining appropriate software safety requirements with traditional safety analysis techniques difficult. STPA (Systems-Theoretic Processes Analysis) is a unique safety analysis approach that has been developed to identify system hazards, including the software-related hazards. This paper presents a comprehensive safety engineering approach based on STPA, including software testing and model checking approaches for the purpose of developing safe software. The proposed approach can be embedded within a defined software engineering process or applied to existing software systems, allow software and safety engineers integrate the analysis of software risks with their verification. The application of the proposed approach is illustrated with an automotive software controller