Validating Architectural Feature Descriptions using LOTOS

The phases of the ANISE project (Architectural Notions In Service Engineering) are briefly explained with reference to the work reported here. An outline strategy is given for translating ANISE descriptions to LOTOS (Language of Temporal Ordering Specification), thus providing a formal basis. It is shown how modular ANISE descriptions of features can be defined and then merged. Potential feature interactions can be identified statically through structural overlaps. A scenario language is introduced to express validation tests for features in a modular fashion, and a number of examples are given. Scenarios are automatically translated to LOTOS and analysed through LOTOS simulation. This allows features to be validated in isolation, and dynamically in combination with other features. The design of the translation and validation tools is discussed, showing typical results when investigating feature descriptions. The paper concludes with a guide to extending the approach for new features

Creating telecommunication services based on object-oriented frameworks and SDL

This paper describes the tools and techniques being applied in the TINA Open Service Creation Architecture (TOSCA) project to develop object-oriented models of distributed telecommunication services in SDL. The paper also describes the way in which Tree and Tabular Combined Notation (TTCN) test cases are derived from these models and subsequently executed against the CORBA-based implementations of these services through a TTCN/CORBA gateway

Design and implementation of a TTCN to C translator

The conformance testing of a protocol implementation, may be logically divided into, the specification of the abstract test suite (ATS) from a formal descnption of the protocol, and the subsequent derivation of the executable test suite (ETS) from the ATS specification. Our concern here is with the latter step, in particular, the automatic derivation of an ATS expressed in the Tree and Tabular Combined Notation (TTCN) to an executable C language equivalent. This process is currently a manual one, and as a consequence is error prone, time consuming, often repetitive and not necessarily consistent. To overcome these problems, there exists the real need for a computer aided, and if possible, fully automatic solution. This study descnbes the design and implementation of a fully working TTCN subset to C language translator, which takes a TTCN ATS and produces an equivalent ETS, with a minimal amount of manual intervention. The methodology used is logically divided into three stages direct TTCN to C language mappings, implementation issues, including the generation of additional code to drive the above mappings, and test system implementation issues. The system was tested using parts of an ETSI ISDN LAPD ATS and the results showed considerable time savings against a similar manual implementation. In conclusion, suggestions are provided to the further development of the TTCN to C translator system, and discussion is given to the apphcation of this tool to a complete conformance testing system

Test case verification by model checking

Verification of a test case for testing the conformance of protocol implementations against the formal description of the protocol involves verifying three aspects of the test case: expected input/output test behavior, test verdicts, and the test purpose. We model the safety and liveness properties of a test case using branching time temporal logic. There are four types of safety properties: transmission safety, reception safety, synchronization safety, and verdict safety. We model a test purpose as a liveness property and give a set of notations to formally specify a test purpose. All these properties expressed as temporal formulas are verified using model checking on an extended state machine graph representing the composed behavior of a test case and protocol specification. This methodology is shown to be effective in finding errors in manually developed conformance test suites. © 1993 Kluwer Academic Publishers

Automated Testing: Requirements Propagation via Model Transformation in Embedded Software

Testing is the most common activity to validate software systems and plays a key role in the software development process. In general, the software testing phase takes around 40-70% of the effort, time and cost. This area has been well researched over a long period of time. Unfortunately, while many researchers have found methods of reducing time and cost during the testing process, there are still a number of important related issues such as generating test cases from UCM scenarios and validate them need to be researched. As a result, ensuring that an embedded software behaves correctly is non-trivial, especially when testing with limited resources and seeking compliance with safety-critical software standard. It thus becomes imperative to adopt an approach or methodology based on tools and best engineering practices to improve the testing process. This research addresses the problem of testing embedded software with limited resources by the following. First, a reverse-engineering technique is exercised on legacy software tests aims to discover feasible transformation from test layer to test requirement layer. The feasibility of transforming the legacy test cases into an abstract model is shown, along with a forward engineering process to regenerate the test cases in selected test language. Second, a new model-driven testing technique based on different granularity level (MDTGL) to generate test cases is introduced. The new approach uses models in order to manage the complexity of the system under test (SUT). Automatic model transformation is applied to automate test case development which is a tedious, error-prone, and recurrent software development task. Third, the model transformations that automated the development of test cases in the MDTGL methodology are validated in comparison with industrial testing process using embedded software specification. To enable the validation, a set of timed and functional requirement is introduced. Two case studies are run on an embedded system to generate test cases. The effectiveness of two testing approaches are determined and contrasted according to the generation of test cases and the correctness of the generated workflow. Compared to several techniques, our new approach generated useful and effective test cases with much less resources in terms of time and labor work. Finally, to enhance the applicability of MDTGL, the methodology is extended with the creation of a trace model that records traceability links among generated testing artifacts. The traceability links, often mandated by software development standards, enable the support for visualizing traceability, model-based coverage analysis and result evaluation

Verification of protocol conformance test cases using reachability analysis

A methodology is presented to verify manually written test cases against the formal specification of a protocol. Initially, a protocol and a test case are modeled as nondeterministic finite state machines and test case verification is viewed as a reachability analysis problem. An existing reachability analysis algorithm, based on the well-known perturbation technique, is modified to take nondeterminism in protocols and special test case features (timeouts and OTHERWISE events) into account. Correctness aspects of the reachability algorithm are proved. The notion of a synchronization error manifesting in a test case due to the nondeterministic nature of a protocol specification is studied. To verify data flow aspects of test cases, we extend our technique by modeling the test case and protocol specification as extended finite state machines. A test case from a proprietary test suite for the transport protocol Class 2 is taken as an example and is shown to contain several design errors. © 1992

JTorX: Exploring Model-Based Testing

The overall goal of the work described in this thesis is: ``To design a flexible tool for state-of-the-art model-based derivation and automatic application of black-box tests for reactive systems, usable both for education and outside an academic context.'' From this goal, we derive functional and non-functional design requirements. The core of the thesis is a discussion of the design, in which we show how the functional requirements are fulfilled. In addition, we provide evidence to validate the non-functional requirements, in the form of case studies and responses to a tool user questionnaire. We describe the overall architecture of our tool, and discuss three usage scenarios which are necessary to fulfill the functional requirements: random on-line testing, guided on-line testing, and off-line test derivation and execution. With on-line testing, test derivation and test execution takes place in an integrated manner: a next test step is only derived when it is necessary for execution. With random testing, during test derivation a random walk through the model is done. With guided testing, during test derivation additional (guidance) information is used, to guide the derivation through specific paths in the model. With off-line testing, test derivation and test execution take place as separate activities. In our architecture we identify two major components: a test derivation engine, which synthesizes test primitives from a given model and from optional test guidance information, and a test execution engine, which contains the functionality to connect the test tool to the system under test. We refer to this latter functionality as the ``adapter''. In the description of the test derivation engine, we look at the same three usage scenarios, and we discuss support for visualization, and for dealing with divergence in the model. In the description of the test execution engine, we discuss three example adapter instances, and then generalise this to a general adapter design. We conclude with a description of extensions to deal with symbolic treatment of data and time