Model-Based Testing for Composite Web Services in Cloud Brokerage Scenarios

Cloud brokerage is an enabling technology allowing various services to be merged together for providing optimum quality of service for the end-users. Within this collection of composed services, testing is a challenging task which brokers have to take on to ensure quality of service. Most Software-as-a-Service (SaaS) testing has focused on high-level test generation from the functional specification of individual services, with little research into how to achieve sufficient test coverage of composite services. This paper explores the use of model-based testing to achieve testing of composite services, when two individual web services are tested and combined. Two example web services – a login service and a simple shopping service – are combined to give a more realistic shopping cart service. This paper focuses on the test coverage required for testing the component services individually and their composition. The paper highlights the problems of service composition testing, requiring a reworking of the combined specification and regeneration of the tests, rather than a simple composition of the test suites; and concludes by arguing that more work needs to be done in this area

Contract-based test generation for data flow of business processes using constraint programming

The verification of the properties of a business process (BP) has become a significant research topic in recent years. In the early stages of development, the BP model (e.g. BPMN, EPC), the BP contract (task contract, regulations and laws, business rules), and the test objectives (requirements) are the only elements available. In order to support the modellers, automatic tools must be provided in order to check whether their business processes are in line with the BP contract. This paper proposes a new business process called the automatic test-case generator to automate the generation of test cases and verify that a BP has the intended functionality (semantic conformance). This generator is analysed, designed and implemented by taking into account the following tasks: Annotation of the BP model with the business process contract, calculation of the various data flow paths, transformation of these data flow paths into SSA form, and a modelling of a constraint satisfaction problem (constraint programming) of the BP contract for all data flow paths. The execution of this business process generates the test cases automatically.Junta de Andalucía P08-TIC-04095Ministerio de Ciencia e Innovación TIN2009-1371

A verified and optimized Stream X-Machine testing method, with application to cloud service certification

The Stream X-Machine (SXM) testing method provides strong and repeatable guarantees of functional correctness, up to a specification. These qualities make the method attractive for software certification, especially in the domain of brokered cloud services, where arbitrage seeks to substitute functionally equivalent services from alternative providers. However, practical obstacles include: the difficulty in providing a correct specification, the translation of abstract paths into feasible concrete tests, and the large size of generated test suites. We describe a novel SXM verification and testing method, which automatically checks specifications for completeness and determinism, prior to generating complete test suites with full grounding information. Three optimisation steps achieve up to a ten-fold reduction in the size of the test suite, removing infeasible and redundant tests. The method is backed by a set of tools to validate and verify the SXM spec-ification, generate technology-agnostic test suites and ground these in SOAP, REST or rich-client service implementations. The method was initially validated using seven specifications, three cloud platforms and five grounding strategies

IN2GESOFT: Innovation and Integration of Methods for the Development and Quantitative Management of Software Projects TIN2004-06689-C03

This coordinated project intends to introduce new methods in software engineering project management, integrating different quantitative and qualitative technologies in the management processes. The underlying goal to all three subprojects participants is the generation of information adapted for the efficient performance in the directing of the project. The topics that are investigated are related to the capture of decisions in dynam ical environments and complex systems, software testing and the analysis of the manage ment strategies for the process assessment of the software in its different phases of the production. The project sets up a methodological, conceptual framework and supporting tools that facilitate the decision making in the software project management. This allows us to eval uate the risk and uncertainty associated to different alternatives of management before leading them to action. Thus, it is necessary to define a taxonomy of software models so that they reflect the current reality of the projects. Since the software testing is one of the most critical and costly processes directed to guarantee the quality and reliability of the software, we undertake the research on the automation of the process of software testing by means of the development of new technologies test case generation, mainly based in metaheuristic and model checking techniques in the domains of database and internet applications. The software system developed will allow the integration of these technologies, and the management information needed, from the first phases of the cycle of life in the construction of a software product up to the last ones such as regression tests and maintenance. The set of technologies that we investigate include the use of statistical analysis and of experimental design for obtaining metrics in the phase of analysis, the application of the bayesian nets to the decision processes, the application of the standards of process eval uation and quality models, the utilization of metaheuristics algorithms and technologies of prediction to optimize resources, the technologies of visualization to construct control dashboards, hybrid models for the simulation of processes and others

Automated Realistic Test Input Generation and Cost Reduction in Service-centric System Testing

Service-centric System Testing (ScST) is more challenging than testing traditional software due to the complexity of service technologies and the limitations that are imposed by the SOA environment. One of the most important problems in ScST is the problem of realistic test data generation. Realistic test data is often generated manually or using an existing source, thus it is hard to automate and laborious to generate. One of the limitations that makes ScST challenging is the cost associated with invoking services during testing process. This thesis aims to provide solutions to the aforementioned problems, automated realistic input generation and cost reduction in ScST. To address automation in realistic test data generation, the concept of Service-centric Test Data Generation (ScTDG) is presented, in which existing services used as realistic data sources. ScTDG minimises the need for tester input and dependence on existing data sources by automatically generating service compositions that can generate the required test data. In experimental analysis, our approach achieved between 93% and 100% success rates in generating realistic data while state-of-the-art automated test data generation achieved only between 2% and 34%. The thesis addresses cost concerns at test data generation level by enabling data source selection in ScTDG. Source selection in ScTDG has many dimensions such as cost, reliability and availability. This thesis formulates this problem as an optimisation problem and presents a multi-objective characterisation of service selection in ScTDG, aiming to reduce the cost of test data generation. A cost-aware pareto optimal test suite minimisation approach addressing testing cost concerns during test execution is also presented. The approach adapts traditional multi-objective minimisation approaches to ScST domain by formulating ScST concerns, such as invocation cost and test case reliability. In experimental analysis, the approach achieved reductions between 69% and 98.6% in monetary cost of service invocations during testin

Web services choreography testing using semantic service description

Web services have become popular due to their ability to integrate with and to interoperate heterogeneous applications. Several web services can be combined into a single application to meet the needs of users. In the course of web services selection, a web candidate service needs to conform to the behaviour of its client, and one way of ensuring this conformity is by testing the interaction between the web service and its user. The existing web services test approaches mainly focus on syntax-based web services description, whilst the semantic-based solutions mostly address composite process flow testing. The aim of this research is to provide an automated testing approach to support service selection during automatic web services composition using Web Service Modeling Ontology (WSMO). The research work began with understanding and analysing the existing test generation approaches for web services. Second, the weaknesses of the existing approaches were identified and addressed by utilizing the choreography transition rules of WSMO in an effort to generate a Finite State Machine (FSM). The FSM was then used to generate the working test cases. Third, a technique to generate an FSM from Abstract State Machine (ASM) was adapted to be used with WSMO. This thesis finally proposed a new testing model called the Choreography to Finite State Machine (C2FSM) to support the service selection of an automatic web service composition. It proposed new algorithms to automatically generate the test cases from the semantic description (WSMO choreography description). The proposed approach was then evaluated using the Amazon E-Commerce Web Service WSMO description. The quality of the test cases generated using the proposed approach was measured by assessing their mutation adequacy score. A total of 115 mutants were created based on 7 mutant operators. A mutation adequacy score of 0.713 was obtained. The experimental validation demonstrated a significant result in the sense that C2FSM provided an efficient and feasible solution. The result of this research could assist the service consumer agents in verifying the behaviour of the Web service in selecting appropriate services for web service composition

Sistema de teste auto-adaptativo baseado em modelo para SOA dinâmico

Orientadores: Eliane Martins, Andrea CeccarelliDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Arquitetura orientada a serviços (SOA) é um padrão de design popular para implemen- tação de serviços web devido à interoperabilidade, escalabilidade e reuso de soluções de software que promove. Os serviços que usam essa arquitetura precisam operar em um am- biente altamente dinâmico, entretanto quanto mais a complexidade desses serviços cresce menos os métodos tradicionais de validação se mostram viáveis. Aplicações baseadas em arquitetura orientada a serviços podem evoluir e mudar du- rante a execução. Por conta disso testes offline não asseguram completamente o compor- tamento correto de um sistema em tempo de execução. Por essa razão, a necessidade de tecnicas diferentes para validar o comportamento adequado de uma aplicação SOA durante o seu ciclo de vida são necessárias, por isso testes online executados durante o funcionamento serão usados nesse projeto. O objetivo do projeto é de aplicar técnicas de testes baseados em modelos para gerar e executar casos de testes relevantes em aplicações SOA durante seu tempo de execu- ção. Para alcançar esse objetivo uma estrura de teste online autoadaptativa baseada em modelos foi idealizada. Testes baseados em modelos podem ser gerados de maneira offline ou online. Nos testes offline, os casos de teste são gerados antes do sistema entrar em execução. Já nos testes online, os casos de teste são gerados e aplicados concomitantemente, e as saídas produzidas pela aplicação em teste definem o próximo passo a ser realizado. Quando uma evolução é detectada em um serviço monitorado uma atualização no modelo da aplicação alvo é executada, seguido pela geração e execução de casos de testes online. Mais precisamente, quatro componentes foram integrados em um circuito autoadap- tativo: um serviço de monitoramento, um serviço de criação de modelos, um serviço de geração de casos de teste baseado em modelos e um serviço de teste. As caracteristicas da estrutura de teste foram testadas em três cenários que foram executados em uma aplicação SOA orquestrada por BPEL, chamada jSeduite. Este trabalho é um esforço para entender as restrições e limitações de teste de soft- ware para aplicações SOA, e apresenta análises e soluções para alguns dos problemas encontrados durante a pesquisaAbstract: Service Oriented Architecture (SOA) is a popular design pattern to build web services be- cause of the interoperability, scalability, and reuse of software solutions that it promotes. The services using this architecture need to operate in a highly dynamic environment, but as the complexity of these services grows, traditional validation processes become less feasible. SOA applications can evolve and change during their execution, and offline tests do not completely assure the correct behavior of the system during its execution. There- fore there is a need of techniques to validate the proper behaviour of SOA applications during the SOA lifecycle. Because of that, in this project online testing will be used. The project goal is to employ model-based testing techniques to generate and execute relevant test cases to SOA applications during runtime. In order to achieve this goal a self-adaptive model-based online testing framework was designed. Tests based on models can be generated offline and online. Offline test are generated before the system execution. Online tests are generated and performed concomitantly, and the output produced by the application under test defines the next step to be performed. when our solution detects that a monitored service evolves, the model of the target service is updated, and online test case generation and execution is performed. More specifically, four components were integrated in a self-adaptive loop: a mon- itoring service, a model generator service, a model based testing service and a testing platform. The testing framework had its features tested in three scenarios that were performed in a SOA application orchestrated by BPEL, called jSeduite. This work is an effort to understand the constraints and limitations of the software testing on SOA applications, and present analysis and solutions to some of the problems found during the researchMestradoCiência da ComputaçãoMestre em Ciência da ComputaçãoCAPE

A model to design and verify context-aware adaptive service composition.

The introduction of mobile clients and context-aware behaviours intoWeb Service compositions may generate faults and inconsistencies. We introduce an extension of a composition model where context-awareness is made explicit and a number of correctness properties are verifiable. In particular, our extended model enables the verification of properties commonly used to validate context dependent applications. We also propose a set of algorithms to verify these properties efficiently