Testes incrementais em um desenvolvimento guiado por testes baseados em modelo

Orientador: Eliane MartinsDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: O desenvolvimento de sistemas pode ser realizado seguindo diversos modelos de processo. Os métodos ágeis propõem realizar implementações iterativas e incrementais e testes antecipados, buscando uma validação antecipada do sistema. Algumas técnicas ágeis adicionam a característica de um desenvolvimento de sistema baseado em testes, como as técnicas de Desenvolvimento Baseado em Teste (do inglês Test Driven Development (TDD)) e Desenvolvimento Baseado em Comportamento (do inglês Behaviour Driven Development (BDD)). Recentemente algumas técnicas propõem a união de técnicas ágeis de desenvolvimento baseado em testes com técnicas consolidadas da área de testes, com o objetivo principal de auxiliar na etapa de criação de testes, que serão utilizados para guiar o desenvolvimento do sistema. Um exemplo é a técnica de Desenvolvimento Guiado por Testes Baseados em Modelo (do inglês Model-Based Test Driven Development (MBTDD)) que une os conceitos de Testes Baseados em Modelo (do inglês Model-Based Testing (MBT)) e Desenvolvimento Baseado em Teste (TDD). Portanto em MBTDD, testes são derivados de modelos que representam os comportamentos esperados do sistema, e baseado nesses testes, o desenvolvimento iterativo e incremental ocorre. Entretanto quando lidamos com processos iterativos e incrementais, surgem problemas decorrente da evolução do sistema, como por exemplo: como reutilizar os artefatos de testes, e como selecionar os testes relevantes para a codificação da nova versão do sistema. Nesse contexto, este trabalho explora um processo no qual o desenvolvimento ágil de sistema é guiado por testes baseados em modelos, com o enfoque no auxílio do reúso dos artefatos de testes e no processo de identificação de testes relevantes para o desenvolvimento de uma nova versão do sistema. Para tanto, características do processo de MBTDD são unidas com características de uma técnica que busca o reúso de artefatos de testes baseado em princípios de testes de regressão, denominada Testes de Regressão SPL Baseados em Modelo Delta (do inglês Delta-Oriented Model-Based SPL Regression Testing). Para realizar a avaliação da solução proposta, ela foi aplicada em exemplos existentes e comparada com a abordagem no qual nenhum caso de teste é reutilizadoAbstract: Systems can be developed following different process models. Agile methods propose iterative and incremental implementations and anticipating tests, in order to anticipate system validation. Some agile techniques add the characteristic of development based on tests, like in Test Driven Development (TDD) and Behaviour Driven Development (BDD). Recently some techniques proposed joining the agile techniques of development based on tests with techniques consolidated in the field of testing, with the main purpose of aiding in the test creation stage, which are used to guide the development of the system. An example is Model-Based Test Driven Development (MBTDD) which joins the concepts of Model-Based Testing (MBT) and Test Driven Development (TDD). Therefore in MBTDD, tests are derived from models that represent the expected behaviour of the system, and based on those tests, iterative and incremental development is performed. However, when iterative and incremental processes are used, problems appear as the consequence of the evolution of the system, such as: how to reuse the test artefacts, and how to select the relevant tests for implementing the new version of the system. In this context, this work proposes a process in which the agile development of a system is guided by model based tests, focusing on helping with the reuse of test artefacts and on the process of identifying tests relevant to development. To achieve this goal, MBTDD process characteristics are joined with characteristics from a technique that aims to find reusability of test artefacts based on principles of regression tests, called Delta-Oriented Model-Based SPL Regression Testing. To evaluate the proposed solution, it was applied to existing examples and compared to the approach without any test case reuseMestradoCiência da ComputaçãoMestra em Ciência da Computação151647/2013-5CNP

Property-Based Testing - The ProTest Project

The ProTest project is an FP7 STREP on property based testing. The purpose of the project is to develop software engineering approaches to improve reliability of service-oriented networks; support fault-finding and diagnosis based on specified properties of the system. And to do so we will build automated tools that will generate and run tests, monitor execution at run-time, and log events for analysis. The Erlang / Open Telecom Platform has been chosen as our initial implementation vehicle due to its robustness and reliability within the telecoms sector. It is noted for its success in the ATM telecoms switches by Ericsson, one of the project partners, as well as for multiple other uses such as in facebook, yahoo etc. In this paper we provide an overview of the project goals, as well as detailing initial progress in developing property based testing techniques and tools for the concurrent functional programming language Erlang

Bridging the Gap Between Requirements and Model Analysis : Evaluation on Ten Cyber-Physical Challenge Problems

Formal verfication and simulation are powerful tools to validate requirements against complex systems. [Problem] Requirements are developed in early stages of the software lifecycle and are typically written in ambiguous natural language. There is a gap between such requirements and formal notations that can be used by verification tools, and lack of support for proper association of requirements with software artifacts for verification. [Principal idea] We propose to write requirements in an intuitive, structured natural language with formal semantics, and to support formalization and model/code verification as a smooth, well-integrated process. [Contribution] We have developed an end-to-end, open source requirements analysis framework that checks Simulink models against requirements written in structured natural language. Our framework is built in the Formal Requirements Elicitation Tool (fret); we use fret's requirements language named fretish, and formalization of fretish requirements in temporal logics. Our proposed framework contributes the following features: 1) automatic extraction of Simulink model information and association of fretish requirements with target model signals and components; 2) translation of temporal logic formulas into synchronous dataflow cocospec specifications as well as Simulink monitors, to be used by verification tools; we establish correctness of our translation through extensive automated testing; 3) interpretation of counterexamples produced by verification tools back at requirements level. These features support a tight integration and feedback loop between high level requirements and their analysis. We demonstrate our approach on a major case study: the Ten Lockheed Martin Cyber-Physical, aerospace-inspired challenge problems

Automating Test Case Generation for Android Applications using Model-based Testing

Testing of mobile applications (apps) has its quirks as numerous events are required to be tested. Mobile apps testing, being an evolving domain, carries certain challenges that should be accounted for in the overall testing process. Since smartphone apps are moderate in size so we consider that model-based testing (MBT) using state machines and statecharts could be a promising option for ensuring maximum coverage and completeness of test cases. Using model-based testing approach, we can automate the tedious phase of test case generation, which not only saves time of the overall testing process but also minimizes defects and ensures maximum test case coverage and completeness. In this paper, we explore and model the most critical modules of the mobile app for generating test cases to ascertain the efficiency and impact of using model-based testing. Test cases for the targeted model of the application under test were generated on a real device. The experimental results indicate that our framework reduced the time required to execute all the generated test cases by 50%. Experimental setup and results are reported herein

Selection of UML Models for Test Case Generation: A Discussion on Techniques to Generate Test Cases

Software development life cycle (SDLC) depicts the distinct phases a software needs to go through, starting off with the requirement engineering phase through the testing phase. Requirement engineering and testing hold a very important place in the development of the software. Testing plays such a crucial part in SDLC that more than development some projects spend more time on testing. This helps in finding bugs and ensure that a quality product is shipped. Test case generation is the toughest step in the testing process. It involves a lot of effort to find errors in the code. To eliminate the tedious process of finding errors in the code and improve the efficiency of the software, an innovative approach called Model-Based Testing (MBT) has been evolved. In the MBT approach, testing begins at the design phase and is thus helpful in identifying the faults early. Unified Modeling Language (UML) has been used to generate test cases using MBT. There has been a lot of research and proposals made for MBT using different UML diagrams. But the problem with these approaches is that a single UML diagram like activity diagram or sequence diagram is not enough to generate test cases in all the scenarios. There might be situation where multiple models might be used to generate test case. This paper discusses about different existing MBT methodologies used for testing and discusses the guidelines for choosing an efficient UML diagram to generate test cases. It also discusses about the way of improving the models by using multiple UML models or by extending the models. Keywords: Software testing, Model-Based Testing (MBT), Unified Modeling Language (UML)

A Constraint-Solving Approach for Achieving Minimal-Reset Transition Coverage of Smartcard Behaviour

Smartcards are security critical devices requiring a high assurance verification approach. Although formal techniques can be used at design or even at development stages, such systems have to undergo a traditional hardware-in-the-loop testing phase. This phase is subject to two key requirements: achieving exhaustive transition coverage of the behavior of the system under test, and minimizing the testing time. In this context, testing time is highly bound to a specific hardware reset operation. Model-based testing is the adequate approach given the availability of a precise model of the system behavior and its ability to produce high quality coverage while optimizing some cost criterion. %l'argument n'est pas convainquant.This paper presents an original algorithm addressing this problem by reformulating it as an integer programming problem to make a graph Eulerian. The associated cost criterion captures both the number of resets and the total length of the test suite, as an auxiliary objective. The algorithm ensures transition coverage. An implementation of the algorithm was developed, benchmarked, and integrated into an industrial smartcard testing framework. A validation case study from this domain is also presented. The approach can of course be applied to any other domains with similar reset-related testing constraints

Reverse Engineering and Testing of Rich Internet Applications

The World Wide Web experiences a continuous and constant evolution, where new initiatives, standards, approaches and technologies are continuously proposed for developing more effective and higher quality Web applications. To satisfy the growing request of the market for Web applications, new technologies, frameworks, tools and environments that allow to develop Web and mobile applications with the least effort and in very short time have been introduced in the last years. These new technologies have made possible the dawn of a new generation of Web applications, named Rich Internet Applications (RIAs), that offer greater usability and interactivity than traditional ones. This evolution has been accompanied by some drawbacks that are mostly due to the lack of applying well-known software engineering practices and approaches. As a consequence, new research questions and challenges have emerged in the field of web and mobile applications maintenance and testing. The research activity described in this thesis has addressed some of these topics with the specific aim of proposing new and effective solutions to the problems of modelling, reverse engineering, comprehending, re-documenting and testing existing RIAs. Due to the growing relevance of mobile applications in the renewed Web scenarios, the problem of testing mobile applications developed for the Android operating system has been addressed too, in an attempt of exploring and proposing new techniques of testing automation for these type of applications

Automatic generation of user interfaces from rigorous domain and use case models

Tese de doutoramento. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 201

Test Sequences for Web Service Composition using CPN model

Web service composition is most mature and effective way to realize the rapidly changing requirements of business in service-oriented solutions. Testing the compositions of web services is complex, due to their distributed nature and asynchronous behaviour. Colored Petri Nets (CPNs) provide a framework for the design, specification, validation and verification of systems. In this paper the CPN model used for composition design verification is reused for test design purpose. We propose an on-the-fly algorithm that generates a test suite that covers all possible paths without redundancy.  The prioritization of test sequences, test suite size and redundancy reduction are also focused. The proposed technique was applied to air line reservation system and the generated test sequences were evaluated against three coverage criteria; Decision Coverage, Input Output Coverage and Transition Coverage. Keywords— CPN, MBT, web service composition testing, test case generatio