A subsumption hierarchy of test case prioritization for composite services

published_or_final_versio

Sistema de teste auto-adaptativo baseado em modelo para SOA dinâmico

Orientadores: Eliane Martins, Andrea CeccarelliDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Arquitetura orientada a serviços (SOA) é um padrão de design popular para implemen- tação de serviços web devido à interoperabilidade, escalabilidade e reuso de soluções de software que promove. Os serviços que usam essa arquitetura precisam operar em um am- biente altamente dinâmico, entretanto quanto mais a complexidade desses serviços cresce menos os métodos tradicionais de validação se mostram viáveis. Aplicações baseadas em arquitetura orientada a serviços podem evoluir e mudar du- rante a execução. Por conta disso testes offline não asseguram completamente o compor- tamento correto de um sistema em tempo de execução. Por essa razão, a necessidade de tecnicas diferentes para validar o comportamento adequado de uma aplicação SOA durante o seu ciclo de vida são necessárias, por isso testes online executados durante o funcionamento serão usados nesse projeto. O objetivo do projeto é de aplicar técnicas de testes baseados em modelos para gerar e executar casos de testes relevantes em aplicações SOA durante seu tempo de execu- ção. Para alcançar esse objetivo uma estrura de teste online autoadaptativa baseada em modelos foi idealizada. Testes baseados em modelos podem ser gerados de maneira offline ou online. Nos testes offline, os casos de teste são gerados antes do sistema entrar em execução. Já nos testes online, os casos de teste são gerados e aplicados concomitantemente, e as saídas produzidas pela aplicação em teste definem o próximo passo a ser realizado. Quando uma evolução é detectada em um serviço monitorado uma atualização no modelo da aplicação alvo é executada, seguido pela geração e execução de casos de testes online. Mais precisamente, quatro componentes foram integrados em um circuito autoadap- tativo: um serviço de monitoramento, um serviço de criação de modelos, um serviço de geração de casos de teste baseado em modelos e um serviço de teste. As caracteristicas da estrutura de teste foram testadas em três cenários que foram executados em uma aplicação SOA orquestrada por BPEL, chamada jSeduite. Este trabalho é um esforço para entender as restrições e limitações de teste de soft- ware para aplicações SOA, e apresenta análises e soluções para alguns dos problemas encontrados durante a pesquisaAbstract: Service Oriented Architecture (SOA) is a popular design pattern to build web services be- cause of the interoperability, scalability, and reuse of software solutions that it promotes. The services using this architecture need to operate in a highly dynamic environment, but as the complexity of these services grows, traditional validation processes become less feasible. SOA applications can evolve and change during their execution, and offline tests do not completely assure the correct behavior of the system during its execution. There- fore there is a need of techniques to validate the proper behaviour of SOA applications during the SOA lifecycle. Because of that, in this project online testing will be used. The project goal is to employ model-based testing techniques to generate and execute relevant test cases to SOA applications during runtime. In order to achieve this goal a self-adaptive model-based online testing framework was designed. Tests based on models can be generated offline and online. Offline test are generated before the system execution. Online tests are generated and performed concomitantly, and the output produced by the application under test defines the next step to be performed. when our solution detects that a monitored service evolves, the model of the target service is updated, and online test case generation and execution is performed. More specifically, four components were integrated in a self-adaptive loop: a mon- itoring service, a model generator service, a model based testing service and a testing platform. The testing framework had its features tested in three scenarios that were performed in a SOA application orchestrated by BPEL, called jSeduite. This work is an effort to understand the constraints and limitations of the software testing on SOA applications, and present analysis and solutions to some of the problems found during the researchMestradoCiência da ComputaçãoMestre em Ciência da ComputaçãoCAPE

Preemptive regression testing of workflow-based web services

published_or_final_versio

An empirical study on mutation testing of WS-BPEL programs

Nowadays, applications are increasingly deployed as Web services in the globally distributed cloud computing environment. Multiple services are normally composed to fulfill complex functionalities. Business Process Execution Language for Web Services (WS-BPEL) is an XML-based service composition language that is used to define a complex business process by orchestrating multiple services. Compared with traditional applications, WS-BPEL programs pose many new challenges to the quality assurance, especially testing, of service compositions. A number of techniques have been proposed for testing WS-BPEL programs, but only a few studies have been conducted to systematically evaluate the effectiveness of these techniques. Mutation testing has been widely acknowledged as not only a testing method in its own right but also a popular technique for measuring the fault-detection effectiveness of other testing methods. Several previous studies have proposed a family of mutation operators for generating mutants by seeding various faults into WS-BPEL programs. In this study, we conduct a series of empirical studies to evaluate the applicability and effectiveness of various mutation operators for WS-BPEL programs. The experimental results provide insightful and comprehensive guidance for mutation testing of WS-BPEL programs in practice. In particular, our work is the systematic study in the selection of effective mutation operators specifically for WS-BPEL programs

Automated Realistic Test Input Generation and Cost Reduction in Service-centric System Testing

Service-centric System Testing (ScST) is more challenging than testing traditional software due to the complexity of service technologies and the limitations that are imposed by the SOA environment. One of the most important problems in ScST is the problem of realistic test data generation. Realistic test data is often generated manually or using an existing source, thus it is hard to automate and laborious to generate. One of the limitations that makes ScST challenging is the cost associated with invoking services during testing process. This thesis aims to provide solutions to the aforementioned problems, automated realistic input generation and cost reduction in ScST. To address automation in realistic test data generation, the concept of Service-centric Test Data Generation (ScTDG) is presented, in which existing services used as realistic data sources. ScTDG minimises the need for tester input and dependence on existing data sources by automatically generating service compositions that can generate the required test data. In experimental analysis, our approach achieved between 93% and 100% success rates in generating realistic data while state-of-the-art automated test data generation achieved only between 2% and 34%. The thesis addresses cost concerns at test data generation level by enabling data source selection in ScTDG. Source selection in ScTDG has many dimensions such as cost, reliability and availability. This thesis formulates this problem as an optimisation problem and presents a multi-objective characterisation of service selection in ScTDG, aiming to reduce the cost of test data generation. A cost-aware pareto optimal test suite minimisation approach addressing testing cost concerns during test execution is also presented. The approach adapts traditional multi-objective minimisation approaches to ScST domain by formulating ScST concerns, such as invocation cost and test case reliability. In experimental analysis, the approach achieved reductions between 69% and 98.6% in monetary cost of service invocations during testin

An Adaptive Mediation Framework for Workflow Management in the Internet of Things

Tärkavad värkvõrksüsteemid koosnevad arvukast hulgast heterogeensetest füüsilistest seadmetest, mis ühenduvad Internetiga. Need seadmed suudavad pidevalt ümbritseva keskkonnaga suhelda ja osana lõppkasutaja rakendusestest edendada valdkondi nagu tark kodu, e-tervis, logistika jne. Selleks, et integreerida füüsilisi seadmeid värkvõrgu haldussüssteemidega, on töövoo haldussüsteemid kerkinud esile sobiva lahendusena. Ent töövoo haldussüsteemide rakendamine värkvõrku toob kaasa reaalajas teenuste komponeerimise väljakutseid nagu pidev teenusavastus ja -käivitus. Lisaks kerkib küsimus, kuidas piiratud resurssidega värkvõrgu seadmeid töövoo haldussüsteemidega integreerida ning kuidas töövooge värkvõrgu seadmetel käivitada. Tööülesanded (nagu pidev seadmeavastus) võivad värkvõrgus osalevatele piiratud arvutusjõudluse ja akukestvusega seadmetele nagu nutitelefonid koormavaks osutuda. Siinkohal on võimalikuks lahenduseks töö delegeerimine pilve. Käesolev magistritöö esitleb kontekstipõhist raamistikku tööülesannete vahendamiseks värkvõrgurakendustes. Antud raamistikus modelleeritakse ning käitatakse tööülesandeid kasutades töövoogusid. Raamistiku prototüübiga läbi viidud uurimus näitas, et raamistik on võimeline tuvastama, millal seadme avastusülesannete pilve delegeerimine on kuluefektiivsem. Vahel aga pole töövoo käitamistarkvara paigaldamine värkvõrgu seadmetele soovitav, arvestades energiasäästlikkust ning käituskiirust. Käesolev töö võrdles kaht tüüpi töövookäitust: a) töövoo mudeli käitamine käitusmootoriga ning b) töövoo mudelist tõlgitud programmikoodi käitamine. Lähtudes katsetest päris seadmetega, võrreldi nimetatud kahte meetodit silmas pidades süsteemiressursside- ning energiakasutust.Emerging Internet of Things (IoT) systems consist of great numbers of heterogeneous physical entities that are interconnected via the Internet. These devices can continuously interact with the surrounding environment and be used for user applications that benefit human life in domains such as assisted living, e-health, transportation etc. In order to integrate the frontend physical things with IoT management systems, Workflow Management Systems (WfMS) have gained attention as a viable option. However, applying WfMS in IoT faces real-time service composition challenges such as continuous service discovery and invocation. Another question is how to integrate resource-contained IoT devices with the WfMS and execute workflows on the IoT devices. Tasks such as continuous device discovery can be taxing for IoT-involved devices with limited processing power and battery life such as smartphones. In order to overcome this, some tasks can be delegated to a utility Cloud instance. This thesis proposes a context-based framework for task mediation in Internet of Things applications. In the framework, tasks are modelled and executed as workflows. A case study carried out with a prototype of the framework showed that the proposed framework is able to decide when it is more cost-efficient to delegate discovery tasks to the cloud. However, sometimes embedding a workflow engine in an IoT device is not beneficial considering agility and energy conservation. This thesis compared two types of workflow execution: a) execution of workflow models using an embedded workflow engine and b) execution of program code translations based on the workflow models. Based on experiments with real devices, the two methods were compared in terms of system resource and energy usage

Enhancing coverage adequacy of service compositions after runtime adaptation

Laufzeitüberwachung (engl. runtime monitoring) ist eine wichtige Qualitätssicherungs-Technik für selbstadaptive Service-Komposition. Laufzeitüberwachung überwacht den Betrieb der Service-Komposition. Zur Bestimmung der Genauigkeit von Software-Tests werden häufig Überdeckungskriterien verwendet. Überdeckungskriterien definieren Anforderungen die Software-Tests erfüllen muss. Wegen ihrer wichtigen Rolle im Software-Testen haben Forscher Überdeckungskriterien an die Laufzeitüberwachung von Service-Komposition angepasst. Die passive Art der Laufzeitüberwachung und die adaptive Art der Service-Komposition können die Genauigkeit von Software-Tests zur Laufzeit negativ beeinflussen. Dies kann jedoch die Zuversicht in der Qualität der Service-Komposition begrenzen. Um die Überdeckung selbstadaptiver Service-Komposition zur Laufzeit zu verbessern, untersucht diese Arbeit, wie die Laufzeitüberwachung und Online-Testen kombiniert werden können. Online-Testen bedeutet dass Testen parallel zu der Verwendung einer Service-Komposition erfolgt. Zunächst stellen wir einen Ansatz vor, um gültige Execution-Traces für Service-Komposition zur Laufzeit zu bestimmen. Der Ansatz berücksichtigt die Execution-Traces von Laufzeitüberwachung und (Online)-Testen. Er berücksichtigt Änderungen im Workflow und Software-Services eines Service-Komposition. Zweitens, definieren wir Überdeckungskriterien für Service-Komposition. Die Überdeckungskriterien berücksichtigen Ausführungspläne einer Service-Komposition und berücksichtigen die Überdeckung für Software-Services und die Service-Komposition. Drittens stellen wir Online-Testfälle Priorisierungs Techniken, um die Abdeckungniveau einer Service-Komposition schneller zu erreichen. Die Techniken berücksichtigen die Überdeckung einer Service-Komposition durch beide Laufzeitüberwachung und Online-Tests. Zusätzlich, berücksichtigen sie die Ausführungszeit von Testfällen und das Nutzungsmodell der Service-Komposition. Viertens stellen wir einen Rahmen für die Laufzeitüberwachung und Online-Testen von Software-Services und Service-Komposition, genannt PROSA, vor. PROSA bietet technische Unterstützung für die oben genannten Beiträge. Wir evaluieren die Beiträge anhand einer beispielhaften Service-Komposition, die häufig in dem Forschungsgebiet Service-oriented Computing eingesetzt wird.Runtime monitoring (or monitoring for short) is a key quality assurance technique for self-adaptive service compositions. Monitoring passively observes the runtime behaviour of service compositions. Coverage criteria are extensively used for assessing the adequacy (or thoroughness) of software testing. Coverage criteria specify certain requirements on software testing. The importance of coverage criteria in software testing has motivated researchers to adapt them to the monitoring of service composition. However, the passive nature of monitoring and the adaptive nature of service composition could negatively influence the adequacy of monitoring, thereby limiting the confidence in the quality of the service composition. To enhance coverage adequacy of self-adaptive service compositions at runtime, this thesis investigates how to combine runtime monitoring and online testing. Online testing means testing a service composition in parallel to its actual usage and operation. First, we introduce an approach for determining valid execution traces for service compositions at runtime. The approach considers execution traces of both monitoring and (online) testing. It considers modifications in both workflow and constituent services of a service composition. Second, we define coverage criteria for service compositions. The criteria consider execution plans of a service composition for coverage assessment and consider the coverage of an abstract service and the overall service composition. Third, we introduce online-test-case prioritization techniques to achieve a faster coverage of a service composition. The techniques employ coverage of a service composition from both monitoring and online testing, execution time of test cases, and the usage model of the service composition. Fourth, we introduce a framework for monitoring and online testing of services and service compositions called PROSA. PROSA provides technical support for the aforementioned contributions. We evaluate the contributions of this thesis using service compositions frequently used in service-oriented computing research