Automated Testing of Web Services Based on Algebraic Specifications

The testing of web services must be done in a completely automated manner when it takes place on-the-fly due to third-party services are dynamically composed to. We present an approach that uses algebraic specification to make this possible. Test data is generated from a formal specification and then used to construct and submit service requests. Test results are then extracted and checked against the specification. All these are done automatically, as required. We present ASSAT (Algebraic Specification-Based Service Automated Testing), a prototype tool that performs these tasks and demonstrate its utility by applying it to Amazon Web Services, a real-life industrial example

Identifying Test Requirements by Analyzing SLA Guarantee Terms

Service Level Agreements (SLAs) are used to specify the negotiated conditions between the provider and the consumer of services. In this paper we present a stepwise method to identify and categorize a set of test requirements that represent the potential situations that can be exercised regarding the specification of each isolated guarantee term of an SLA. This identification is addressed by means of devising a set of coverage levels that allow grading the thoroughness of the tests. The utilization of these test requirements would focus on twofold objectives: (1) the generation of a test suite that allows exercising the situations described in the test requirements and (2) the support for the derivation of a monitoring plan that checks the compliance of these requirements at runtime. The approach is illustrated over an eHealth case study

Automatic test case generation for WS-Agreements using combinatorial testing

In the scope of the applications developed under the service-based paradigm, Service Level Agreements (SLAs) are a standard mechanism used to flexibly specify the Quality of Service (QoS) that must be delivered. These agreements contain the conditions negotiated between the service provider and consumers as well as the potential penalties derived from the violation of such conditions. In this context, it is important to assure that the service based application (SBA) behaves as expected in order to avoid potential consequences like penalties or dissatisfaction between the stakeholders that have negotiated and signed the SLA. In this article we address the testing of SLAs specified using the WS-Agreement standard by means of applying testing techniques such as the Classification Tree Method and Combinatorial Testing to generate test cases. From the content of the individual terms of the SLA, we identify situations that need to be tested. We also obtain a set of constraints based on the SLA specification and the behavior of the SBA in order to guarantee the testability of the test cases. Furthermore, we define three different coverage strategies with the aim at grading the intensity of the tests. Finally, we have developed a tool named SLACT (SLA Combinatorial Testing) in order to automate the process and we have applied the whole approach to an eHealth case study

Cross-language program analysis for dynamic web applications

Web applications have become one of the most important and prevalent types of software. In modern web applications, the display of any web page is usually an interplay of multiple languages and involves code execution at different locations (the server side, the database side, and the client side). These characteristics make it hard to write and maintain web applications. Much of the existing research and tool support often deals with one single language and therefore is still limited in addressing those challenges. To fill in this gap, this dissertation is aimed at developing an infrastructure for cross-language program analysis for dynamic web applications to support creating reliable and robust web applications with higher quality and lower costs. To reach that goal, we have developed the following research components. First, to understand the client-side code that is embedded in the server-side code, we develop an output-oriented symbolic execution engine that approximates all possible outputs of a server-side program. Second, we use variability-aware parsing, a technique recently developed for parsing conditional code in software product lines, to parse those outputs into a compact tree representation (called VarDOM) that represents all possible DOM variants of a web application. Third, we leverage the VarDOM to extract semantic information from the server-side code. Specifically, we develop novel concepts, techniques, and tools (1) to build call graphs for embedded client code in different languages, (2) to compute cross-language program slices, and (3) to compute a novel test coverage criterion called output coverage that aids testers in creating effective test suites for detecting output-related bugs. The results have been demonstrated in a wide range of applications for web programs such as IDE services, fault localization, bug detection, and testing

Geração de casos de teste para aplicações Web baseados em modelo de tarefas.

O tempo de desenvolvimento de software foi reduzido com a criação de novas ferramentas e paradigmas de programação. Além disso, a necessidade por sistemas cada vez mais eficientes e inovadores, diminuiu o tempo de lançamento entre as versões. Além de entregar produtos de software em tempo hábil, assim como garantir a qualidade, segurança e corretude das aplicações Web, é imperativo a utilização de automação de testes. Diversas técnicas foram reportadas na literatura, porém, uma maneira mais eficaz para automatizar a validação dos requisitos funcionais de um sistema Web consiste em utilizar modelo de tarefas para gerar casos de teste. Mesmo assim, as soluções baseadas nessa abordagem geralmente são custosas na geração do modelo e falham na criação dos cenários. Para resolver os problemas reportados, foi desenvolvido o UsaTasker++, um sistema destinado à mutação de casos de teste para validação das regras de negócio das aplicações Web. O modelo proposto permite a geração dos cenários de teste usando uma abordagem simples e intuitiva, a partir da con guração e processamento do grafo correspondente ao modelo de tarefas. Para cada caso de teste gerado, o UsaTasker++ cria o respectivo script de teste automatizado, permitindo a sua execução e indicando os cenários com erro

Model Checking and Model-Based Testing : Improving Their Feasibility by Lazy Techniques, Parallelization, and Other Optimizations

This thesis focuses on the lightweight formal method of model-based testing for checking safety properties, and derives a new and more feasible approach. For liveness properties, dynamic testing is impossible, so feasibility is increased by specializing on an important class of properties, livelock freedom, and deriving a more feasible model checking algorithm for it. All mentioned improvements are substantiated by experiments

Evaluating Software Testing Techniques: A Systematic Mapping Study

Software testing techniques are crucial for detecting faults in software and reducing the risk of using it. As such, it is important that we have a good understanding of how to evaluate these techniques for their efficiency, scalability, applicability, and effectiveness at finding faults. This thesis enhances our understanding of testing technique evaluations by providing an overview of the state of the art in research. To accomplish this we utilize a systematic mapping study; structuring the field and identifying research gaps and publication trends. We then present a small case study demonstrating how our mapping study can be used to assist researchers in evaluating their own software testing techniques. We find that a majority of evaluations are empirical evaluations in the form of case studies and experiments, most of these evaluations are of low quality based on proper methodology guidelines, and that relatively few papers in the field discuss how testing techniques should be evaluated

Automated Realistic Test Input Generation and Cost Reduction in Service-centric System Testing

Service-centric System Testing (ScST) is more challenging than testing traditional software due to the complexity of service technologies and the limitations that are imposed by the SOA environment. One of the most important problems in ScST is the problem of realistic test data generation. Realistic test data is often generated manually or using an existing source, thus it is hard to automate and laborious to generate. One of the limitations that makes ScST challenging is the cost associated with invoking services during testing process. This thesis aims to provide solutions to the aforementioned problems, automated realistic input generation and cost reduction in ScST. To address automation in realistic test data generation, the concept of Service-centric Test Data Generation (ScTDG) is presented, in which existing services used as realistic data sources. ScTDG minimises the need for tester input and dependence on existing data sources by automatically generating service compositions that can generate the required test data. In experimental analysis, our approach achieved between 93% and 100% success rates in generating realistic data while state-of-the-art automated test data generation achieved only between 2% and 34%. The thesis addresses cost concerns at test data generation level by enabling data source selection in ScTDG. Source selection in ScTDG has many dimensions such as cost, reliability and availability. This thesis formulates this problem as an optimisation problem and presents a multi-objective characterisation of service selection in ScTDG, aiming to reduce the cost of test data generation. A cost-aware pareto optimal test suite minimisation approach addressing testing cost concerns during test execution is also presented. The approach adapts traditional multi-objective minimisation approaches to ScST domain by formulating ScST concerns, such as invocation cost and test case reliability. In experimental analysis, the approach achieved reductions between 69% and 98.6% in monetary cost of service invocations during testin

On-The-Fly Model-Based Testing of Web Services with Jambition

Increasing complexity and massive use of current web services raise multiple issues for achieving adequate service validation while sticking to time-to-market imperatives. For instance: How to automate test case generation and execution for stateful web services? How to realistically simulate web service related operation calls? How to ensure conformance to specifications? The Plastic validation framework tackles some of these issues by providing specific tools for automated model-based functional testing. Based on the Symbolic Transition System model, test cases can be generated and executed on-the-fly. This test-ing approach was applied for validating the Alarm Dispatcher eHealth service, aimed at providing health attention through mobile devices in B3G networks. In this paper we report how this modeling and testing ap-proach helped to detect failures, support conformance, and reduce drastically the testing effort spent usually in designing test cases, validating test coverage, and executing test cases in traditional testing approaches