Automated Testing of WS-BPEL Service Compositions: A Scenario-Oriented Approach

Nowadays, Service Oriented Architecture (SOA) has become one mainstream paradigm for developing distributed applications. As the basic unit in SOA, Web services can be composed to construct complex applications. The quality of Web services and their compositions is critical to the success of SOA applications. Testing, as a major quality assurance technique, is confronted with new challenges in the context of service compositions. In this paper, we propose a scenario-oriented testing approach that can automatically generate test cases for service compositions. Our approach is particularly focused on the service compositions specified by Business Process Execution Language for Web Services (WS-BPEL), a widely recognized executable service composition language. In the approach, a WS-BPEL service composition is first abstracted into a graph model; test scenarios are then derived from the model; finally, test cases are generated according to different scenarios. We also developed a prototype tool implementing the proposed approach, and an empirical study was conducted to demonstrate the applicability and effectiveness of our approach. The experimental results show that the automatic scenario-oriented testing approach is effective in detecting many types of faults seeded in the service compositions

Distribution pattern-driven development of service architectures

Distributed systems are being constructed by composing a number of discrete components. This practice is particularly prevalent within the Web service domain in the form of service process orchestration and choreography. Often, enterprise systems are built from many existing discrete applications such as legacy applications exposed using Web service interfaces. There are a number of architectural configurations or distribution patterns, which express how a composed system is to be deployed in a distributed environment. However, the amount of code required to realise these distribution patterns is considerable. In this paper, we propose a distribution pattern-driven approach to service composition and architecting. We develop, based on a catalog of patterns, a UML-compliant framework, which takes existing Web service interfaces as its input and generates executable Web service compositions based on a distribution pattern chosen by the software architect

Quality metrics for mutation testing with applications to WS-BPEL compositions

Mutation testing is a successful testing technique based on fault injection. However, it can be very costly, and several cost-reduction techniques for reducing the number of mutants have been proposed in the literature. Cost reduction can be aided by an analysis of mutation operators, but this requires the definition of specialized metrics. Several metrics have been proposed before, although their effectiveness and relative merits are not easy to assess. A step ahead in the evaluation of mutation-reduction techniques would be a better metric to determine objectively the quality of a set of mutants with respect to a given test suite. This work introduces such a metric, which is naturally extended to mutation operators and may be used to reduce the number of mutants, particularly of equivalent mutants. Finally, a firm mutation analysis tool for WS-BPEL service compositions is presented, and experimental results obtained by comparing different metrics on several compositions are presented

Sistema de teste auto-adaptativo baseado em modelo para SOA dinâmico

Orientadores: Eliane Martins, Andrea CeccarelliDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Arquitetura orientada a serviços (SOA) é um padrão de design popular para implemen- tação de serviços web devido à interoperabilidade, escalabilidade e reuso de soluções de software que promove. Os serviços que usam essa arquitetura precisam operar em um am- biente altamente dinâmico, entretanto quanto mais a complexidade desses serviços cresce menos os métodos tradicionais de validação se mostram viáveis. Aplicações baseadas em arquitetura orientada a serviços podem evoluir e mudar du- rante a execução. Por conta disso testes offline não asseguram completamente o compor- tamento correto de um sistema em tempo de execução. Por essa razão, a necessidade de tecnicas diferentes para validar o comportamento adequado de uma aplicação SOA durante o seu ciclo de vida são necessárias, por isso testes online executados durante o funcionamento serão usados nesse projeto. O objetivo do projeto é de aplicar técnicas de testes baseados em modelos para gerar e executar casos de testes relevantes em aplicações SOA durante seu tempo de execu- ção. Para alcançar esse objetivo uma estrura de teste online autoadaptativa baseada em modelos foi idealizada. Testes baseados em modelos podem ser gerados de maneira offline ou online. Nos testes offline, os casos de teste são gerados antes do sistema entrar em execução. Já nos testes online, os casos de teste são gerados e aplicados concomitantemente, e as saídas produzidas pela aplicação em teste definem o próximo passo a ser realizado. Quando uma evolução é detectada em um serviço monitorado uma atualização no modelo da aplicação alvo é executada, seguido pela geração e execução de casos de testes online. Mais precisamente, quatro componentes foram integrados em um circuito autoadap- tativo: um serviço de monitoramento, um serviço de criação de modelos, um serviço de geração de casos de teste baseado em modelos e um serviço de teste. As caracteristicas da estrutura de teste foram testadas em três cenários que foram executados em uma aplicação SOA orquestrada por BPEL, chamada jSeduite. Este trabalho é um esforço para entender as restrições e limitações de teste de soft- ware para aplicações SOA, e apresenta análises e soluções para alguns dos problemas encontrados durante a pesquisaAbstract: Service Oriented Architecture (SOA) is a popular design pattern to build web services be- cause of the interoperability, scalability, and reuse of software solutions that it promotes. The services using this architecture need to operate in a highly dynamic environment, but as the complexity of these services grows, traditional validation processes become less feasible. SOA applications can evolve and change during their execution, and offline tests do not completely assure the correct behavior of the system during its execution. There- fore there is a need of techniques to validate the proper behaviour of SOA applications during the SOA lifecycle. Because of that, in this project online testing will be used. The project goal is to employ model-based testing techniques to generate and execute relevant test cases to SOA applications during runtime. In order to achieve this goal a self-adaptive model-based online testing framework was designed. Tests based on models can be generated offline and online. Offline test are generated before the system execution. Online tests are generated and performed concomitantly, and the output produced by the application under test defines the next step to be performed. when our solution detects that a monitored service evolves, the model of the target service is updated, and online test case generation and execution is performed. More specifically, four components were integrated in a self-adaptive loop: a mon- itoring service, a model generator service, a model based testing service and a testing platform. The testing framework had its features tested in three scenarios that were performed in a SOA application orchestrated by BPEL, called jSeduite. This work is an effort to understand the constraints and limitations of the software testing on SOA applications, and present analysis and solutions to some of the problems found during the researchMestradoCiência da ComputaçãoMestre em Ciência da ComputaçãoCAPE

An empirical study on mutation testing of WS-BPEL programs

Nowadays, applications are increasingly deployed as Web services in the globally distributed cloud computing environment. Multiple services are normally composed to fulfill complex functionalities. Business Process Execution Language for Web Services (WS-BPEL) is an XML-based service composition language that is used to define a complex business process by orchestrating multiple services. Compared with traditional applications, WS-BPEL programs pose many new challenges to the quality assurance, especially testing, of service compositions. A number of techniques have been proposed for testing WS-BPEL programs, but only a few studies have been conducted to systematically evaluate the effectiveness of these techniques. Mutation testing has been widely acknowledged as not only a testing method in its own right but also a popular technique for measuring the fault-detection effectiveness of other testing methods. Several previous studies have proposed a family of mutation operators for generating mutants by seeding various faults into WS-BPEL programs. In this study, we conduct a series of empirical studies to evaluate the applicability and effectiveness of various mutation operators for WS-BPEL programs. The experimental results provide insightful and comprehensive guidance for mutation testing of WS-BPEL programs in practice. In particular, our work is the systematic study in the selection of effective mutation operators specifically for WS-BPEL programs

How validation can help in testing business processes orchestrating web services

Validation and testing are important in developing correct and fault free SOA-basedsystems. BPEL is a high level language that makes it possible to implement business processes asan orchestration of web services. In general, the testing requires much more test scenarios than thevalidation. However, in the case of BPEL processes, which have very simple and well structuredimplementation, test scenarios limited to the validation may also be efficient. The paper describes anexperiment that aims at answering a question whether or not the validation test scenarios are alsoadequate for testing an implementation of BPEL processes. The experiment employs a Software FaultInjector for BPEL Processes that is able to inject faults when the test scenarios are running. Theresults of the experiment seem very promising. Hence, it seems that validation tests might give astrong support for testing

An Integrated Methodology for Creating Composed Web/Grid Services

This thesis presents an approach to design, specify, validate, verify, implement, and evaluate composed web/grid services. Web and grid services can be composed to create new services with complex behaviours. The BPEL (Business Process Execution Language) standard was created to enable the orchestration of web services, but there have also been investigation of its use for grid services. BPEL specifies the implementation of service composition but has no formal semantics; implementations are in practice checked by testing. Formal methods are used in general to define an abstract model of system behaviour that allows simulation and reasoning about properties. The approach can detect and reduce potentially costly errors at design time. CRESS (Communication Representation Employing Systematic Specification) is a domainindependent, graphical, abstract notation, and integrated toolset for developing composite web service. The original version of CRESS had automated support for formal specification in LOTOS (Language Of Temporal Ordering Specification), executing formal validation with MUSTARD (Multiple-Use Scenario Testing and Refusal Description), and implementing in BPEL4WS as the early version of BPEL standard. This thesis work has extended CRESS and its integrated tools to design, specify, validate, verify, implement, and evaluate composed web/grid services. The work has extended the CRESS notation to support a wider range of service compositions, and has applied it to grid services as a new domain. The thesis presents two new tools, CLOVE (CRESS Language-Oriented Verification Environment) and MINT (MUSTARD Interpreter), to respectively support formal verification and implementation testing. New work has also extended CRESS to automate implementation of composed services using the more recent BPEL standard WS-BPEL 2.0