Automating Regression Test Selection for Web Services

As Web services grow in maturity and use, so do the methods which are being used to test and maintain them. Regression Testing is a major component of most major testing systems but has only begun to be applied to Web services. The majority of the tools and techniques applying regression test to Web services are focused on test-case generation, thus ignoring the potential savings of regression test selection. Regression test selection optimizes the regression testing process by selecting a subset of all tests, while still maintaining some level of confidence about the system performing no worse than the unmodified system. A safe regression test selection technique implies that after selection, the level of confidence is as high as it would be if no tests were removed. Since safe regression test selection techniques generally involve code-based (white-box) testing, they cannot be directly applied to Web services due to their loosely-coupled, standards-based, and distributed nature. A framework which automates both the regression test selection and regression testing processes for Web services in a decentralized, end-to-end manner is proposed. As part of this approach, special consideration is given to the concurrency issues which may occur in an autonomous and decentralized system. The resulting synchronization method will be presented along with a set of algorithms which manage the regression testing and regression test selection processes throughout the system. A set of empirical results demonstrate the feasibility and benefit of the approach

Towards a Regression Test Selection Technique for Message-Based Software Integration

Regression testing is essential to ensure software quality. Regression Test-case selection is another process wherein, the testers would like to ensure that test-cases which are obsolete due to the changes in the system should not be considered for further testing. This is the Regression Test-case Selection problem. Although existing research has addressed many related problems, most of the existing regression test-case selection techniques cater to procedural systems. Being academic, they lack the scalability and detail to cater to multi-tier applications. Such techniques can be employed for procedural systems, usually mathematical applications. Enterprise applications have become complex and distributed leading to component-based architectures. Thus, inter-process communication has become a very important activity of any such system. Messaging is the most widely employed intermodule interaction mechanism. Today\u27s systems, being heavily internet dependent, are Web-Services based which utilize XML for messaging. We propose an RTS technique which is specifically targeted at enterprise applications

Towards a Regression Test Selection Technique for Message-Based Software Integration

Regression testing is essential to ensure software quality. Regression Test-case selection is another process wherein, the testers would like to ensure that test-cases which are obsolete due to the changes in the system should not be considered for further testing. This is the Regression Test-case Selection problem. Although existing research has addressed many related problems, most of the existing regression test-case selection techniques cater to procedural systems. Being academic, they lack the scalability and detail to cater to multi-tier applications. Such techniques can be employed for procedural systems, usually mathematical applications. Enterprise applications have become complex and distributed leading to component-based architectures. Thus, inter-process communication has become a very important activity of any such system. Messaging is the most widely employed intermodule interaction mechanism. Today\u27s systems, being heavily internet dependent, are Web-Services based which utilize XML for messaging. We propose an RTS technique which is specifically targeted at enterprise applications

Is XML-based test case prioritization for validating WS-BPEL evolution effective in both average and adverse scenarios?

In real life, a tester can only afford to apply one test case prioritization technique to one test suite against a service-oriented workflow application once in the regression testing of the application, even if it results in an adverse scenario such that the actual performance in the test session is far below the average. It is unclear whether the factors of test case prioritization techniques known to be significant in terms of average performance can be extrapolated to adverse scenarios. In this paper, we examine whether such a factor or technique may consistently affect the rate of fault detection in both the average and adverse scenarios. The factors studied include prioritization strategy, artifacts to provide coverage data, ordering direction of a strategy, and the use of executable and non-executable artifacts. The results show that only a minor portion of the 10 studied techniques, most of which are based on the iterative strategy, are consistently effective in both average and adverse scenarios. To the best of our know-ledge, this paper presents the first piece of empirical evidence regarding the consistency in the effectiveness of test case prioritization techniques and factors of service-oriented workflow applications between average and adverse scenarios.published_or_final_versio

Change Impact Analysis Based Regression Testing of Web Services

Reducing the effort required to make changes in web services is one of the primary goals in web service projects maintenance and evolution. Normally, functional and non-functional testing of a web service is performed by testing the operations specified in its WSDL. The regression testing is performed by identifying the changes made thereafter to the web service code and the WSDL. In this thesis, we present a tool-supported approach to perform efficient regression testing of web services. By representing a web service as a directed graph of WSDL elements, we identify and gathers the changed portions of the graph and use this information to reduce regression testing efforts. Specifically, we identify, categorize, and capture the web service testing needs in two different ways, namely, Operationalized Regression Testing of Web Service (ORTWS) and Parameterized Regression Testing of Web Service (PRTWS). Both of the approach can be combined to reduce the regression testing efforts in the web service project. The proposed approach is prototyped as a tool, named as Automatic Web Service Change Management (AWSCM), which helps in selecting the relevant test cases to construct reduced test suite from the old test suite. We present few case studies on different web service projects to demonstrate the applicability of the proposed tool. The reduction in the effort for regression testing of web service is also estimated.Comment: Master of Technology Thesis, PDPM Indian Institute of Information Technology, Design and Manufacturing Jabalpur (2014

Metamorphic testing for web services: Framework and a case study

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. Since the development and consumption of Web services are completely separated under SOA environment, the consumers are normally provided with limited knowledge of the services and thus have little information about test oracles. The lack of source code and the restricted control of Web services limit the testability of Web services. To address the prominent oracle problem when testing Web services, we propose a metamorphic testing framework for Web services taking into account the unique features of SOA. We conduct a case study where the new metamorphic testing framework is employed to test a Web service that implements the electronic payment. The results of case study show the feasibility of the framework for web services, and also the efficiency of metamorphic testing. The work presented in the paper alleviates the test oracle problem when testing Web services under SOA

A metamorphic relation-based approach to testing web services without oracles

Service Oriented Architecture (SOA) has become a major application development paradigm. As a basic unit of SOA applications, Web services significantly affect the quality of the applications constructed from them. In the context of SOA, the specification and implementation of Web services are completely separated. The lack of source code and the restricted control of Web services limit the testability of Web services, and make the oracle problem prominent. In this context, can one alleviate the test oracle problem, or effectively and efficiently test such Web services even without oracles? It is an important issue which has not been yet adequately addressed. To address the challenge of testing Web services, the authors propose a metamorphic relation-based approach to testing Web services without oracles. The proposed approach leverages so-called metamorphic relations to generate test cases and evaluate test results. To make the proposed approach practical and effective, the authors proposed a framework taking into account the unique features of SOA, and developed a prototype which partially automates the framework. Three case studies are conducted to validate the feasibility and effectiveness of the proposed approach. The work presented in the paper not only alleviates the test oracle problem of testing Web services, but also delivers an effective and efficient test technique without oracles