Environment Behavior Models for Scenario Generation and Testing Automation

In Proceedings of the First International Workshop on Advances in Model-Based Software Testing (A-MOST'05), the 27th International Conference on Software Engineering ICSE’05, May 15-16, 2005, St. Louis, USAThis paper suggests an approach to automatic scenario generation from environment models for testing of real-time reactive systems. The behavior of the system is defined as a set of events (event trace) with two basic relations: precedence and inclusion. The attributed event grammar (AEG) specifies possible event traces and provides a uniform approach for automatically generating, executing, and analyzing test cases. The environment model includes a description of hazardous states in which the system may arrive and makes it possible to gather statistics for system safety assessment. The approach is supported by a generator that creates test cases from the AEG models. We demonstrate the approach with case studies of prototypes for the safety-critical computer-assisted resuscitation algorithm (CARA) software for a casualty intravenous fluid infusion pump and the Paderborn Shuttle System

A subset of precise UML for Model-based Testing

This paper presents an original model-based testing approach that takes a UML behavioural view of the system under test and automatically generates test cases and executable test scripts according to model coverage criteria. This approach is embedded in the LEIRIOS Test Designer tool and is currently deployed in domains such as Enterprise IT and electronic transaction applications. This model-based testing approach makes it possible to automatically produce the traceability matrix from requirements to test cases as part of the test generation process. This paper defines the subset of UML used for model-based testing and illustrates it using a small example

Software integration testing based on communication coverage criteria and partial model generation

This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to define communication-based coverage criteria that refer to send- and receive-statements and communication paths. The proposed method enables testers to focus during component integration on such parts of the specification, e.g. behaviour specifications or Markovian usage models, that are involved in the communication between components to be integrated. A more specific application area of this approach is the integration of test-models, e.g. a transmission gear can be tested based on separated models for the driver behaviour, the engine condition, and the mechanical and hydraulical transmission states. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage or generate test sequences that achieve the goal. Given the generated test sequences we derive a partial test-model of the components from which the test sequences are derived. The partial model can be used to drive further testing and can also be used as the basis for producing additional partial models in incremental integration testing. While the process of deriving the test sequences could suffer from a combinatorial explosion, the effort required to generate the partial model is polynomial in the number of test sequences and their length. Thus, where it is not feasible to produce test sequences that achieve a given type of coverage it is still possible to produce a partial model on the basis of test sequences generated to achieve some other criterion. As a result, the process of generating a partial model has the potential to scale to large industrial software systems. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers. A potential additional benefit of the approach is that it provides a visual description of the state-based testing of distributed systems, which may be beneficial in other contexts such as education and comprehension

Artificial table testing dynamically adaptive systems

Dynamically Adaptive Systems (DAS) are systems that modify their behavior and structure in response to changes in their surrounding environment. Critical mission systems increasingly incorporate adaptation and response to the environment; examples include disaster relief and space exploration systems. These systems can be decomposed in two parts: the adaptation policy that specifies how the system must react according to the environmental changes and the set of possible variants to reconfigure the system. A major challenge for testing these systems is the combinatorial explosions of variants and envi-ronment conditions to which the system must react. In this paper we focus on testing the adaption policy and propose a strategy for the selection of envi-ronmental variations that can reveal faults in the policy. Artificial Shaking Table Testing (ASTT) is a strategy inspired by shaking table testing (STT), a technique widely used in civil engineering to evaluate building's structural re-sistance to seismic events. ASTT makes use of artificial earthquakes that simu-late violent changes in the environmental conditions and stresses the system adaptation capability. We model the generation of artificial earthquakes as a search problem in which the goal is to optimize different types of envi-ronmental variations

Test adequacy assessment using test-defect coverage analytic model

Software testing is an essential activity in software development process that has been widely used as a means of achieving software reliability and quality. The emergence of incremental development in its various forms required a different approach to determining the readiness of the software for release. This approach needs to determine how reliable the software is likely to be based on planned tests, not defect growth and decline as typically shown in reliability growth models. A combination of information from a number of sources into an easily understood dashboard is expected to provide both qualitative and quantitative analyses of test and defect coverage properties. Hence, Test-Defect Coverage Analytic Model (TDCAM) is proposed which combines test and defect coverage information presented in a dashboard to help deciding whether there are enough tests planned. A case study has been conducted to demonstrate the usage of the proposed model. The visual representations and results gained from the case study show the benefits of TDCAM in assisting practitioners making informed test adequacy-related decisions

Software product line testing - a systematic mapping study

Context: Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped. Objective: This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research. Method: A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type. Results: A majority of the papers are of proposal research types (64 %). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority. Conclusions: More validation and evaluation research is needed to provide a better foundation for SPL testing

Constructing interaction test suites with greedy algorithms

Combinatorial approaches to testing are used in several fields, and have recently gained momentum in the field of software testing through software interaction testing. One-test-at-a-time greedy algorithms are used to automatically construct such test suites. This paper discusses basic criteria of why greedy algorithms have been appropriate for this test gen-eration problem in the past and then expands upon how greedy algorithms can be utilized to address test suite pri-oritization

New Directions in Software Quality Assurance Automation

Abstract. A formalism is suggested for specifying environment behavior models for software test scenario generation based on attributed event grammars. The environment model may contain descriptions of the events triggered by the software outputs and of the hazardous states in which the system could arrive, thus providing a framework for specifying properties of software behavior within the given environment. The behavior of the system can be rendered as an event set with two partial ordering relations: precedence and inclusion (event trace). This formalism may be used as a basis for automation tools for test generation, test result monitoring and verification, for experiments to gather statistics about software safety, and for evaluating of dependencies of system's behavior on environment parameters. The monitoring activities can be implemented within a uniform framework as computations over event traces

Análisis de Enfoques de Model Based Testing para Pruebas Funcionales orientados a Aplicaciones Web

En los últimos años las aplicaciones web han ido incrementando en número y a la vez en complejidad debido a la incorporación de nueva tecnología. Esto ha repercutido en un aumento de complejidad de la fase de pruebas dentro del ciclo de vida del desarrollo de software, la cual nos permite asegurar la calidad del producto desarrollado. Esta fase representa un mayor costo y esfuerzo. Con otro tipo de aplicaciones no se le asignaba el tiempo ni esfuerzo necesario. Sin embargo, debido al impacto que puede tener una aplicación web mal probada durante la puesta en marcha de la aplicación, han surgido diversas investigaciones en técnicas para la simplificación de la fase de pruebas. Una de estas técnicas es model based testing, que mediante la representación del comportamiento esperado de la aplicación, genera automáticamente los casos de prueba, incluso permite la ejecución automática de los mismos y su evaluación. El presente trabajo presenta una revisión analítica de los enfoques en model based testing para aplicaciones web orientados a pruebas funcionales, identificando para ello los enfoques existentes dentro de este contexto y realizando un esquema de caracterización para el análisis de las principales características, herramientas y documentación disponible para la aplicación de los enfoques.Universidad de Sevilla. Master Universitario en Ingeniería y Tecnología del Softwar