Automated Runtime Testing of Web Services

Service-oriented computing (SOC) is a relatively new paradigm for developing software applications through the composition of software units called services. With services, software is no longer owned but offered remotely, within or across organisational borders. Currently, the dominant technology for implementing services is that of Web services. Since service requestors do not usually have access to the implementation source code, from their perspective, services are offered as black boxes. However, requestors need to verify first that provided services are trustworthy and implemented correctly before they are integrated into their own business-critical systems. The verification and testing of remote, third-party services involve unique considerations, since testing must be performed in a blackbox manner and at runtime. Addressing the aforementioned concerns, the research work described in this thesis investigates the feasibility of testing Web services for functional correctness, especially at runtime. The aim is to introduce rigour and automation to the testing process, so that service requestors can verify Web services with correctness guarantees and with the aid of tools. Thus, formal methods are utilised to specify the functionality of Web services unambiguously, so that they are amenable to automated and systematic testing. The well-studied stream X-machine (SXM) formalism has been selected as suitable for modelling both the dynamic behavior and static data of Web services, while a proven testing method associated with SXMs is used to derive test sets that can verify the correctness of the implementations. This research concentrates on testing stateful Web services, in which the presence of state makes their behaviour more complex and more difficult to specify and test. The nature of Web service state, its effect on service behaviour, and implications on service modelling and testing, are investigated. In addition, comprehensive techniques are described for deriving a stream X-machine specification of a Web service, and for subsequently testing its implementation for equivalence to the specification. Then, a collaborative approach that makes possible third-party Web service verification and validation is proposed, in which the service provider is required to supply a SXM specification of the service functionality along with the standard WSDL description of its interface. On top of that, techniques are proposed for service providers to include information that ground the abstract SXM specification to the concrete Web service implementation. Having these descriptions available, it is possible to automate at runtime not only test set generation but also test case execution on Web services. A tool has been developed as part of this work, which extends an existing SXM-based testing tool (JSXM). The tool supports the tester activities, consisting of generation of abstract test cases from the SXM specification and their execution on the Web service under test using the supplied grounding information. Practical Web service examples are also used throughout the thesis to demonstrate the proposed techniques

Leveraging Semantic Web Service Descriptions for Validation by Automated Functional Testing

Recent years have seen the utilisation of Semantic Web Service descriptions for automating a wide range of service-related activities, with a primary focus on service discovery, composition, execution and mediation. An important area which so far has received less attention is service validation, whereby advertised services are proven to conform to required behavioural specifications. This paper proposes a method for validation of service-oriented systems through automated functional testing. The method leverages ontology-based and rule-based descriptions of service inputs, outputs, preconditions and effects (IOPE) for constructing a stateful EFSM specification. The specification is subsequently utilised for functional testing and validation using the proven Stream X-machine (SXM) testing methodology. Complete functional test sets are generated automatically at an abstract level and are then applied to concrete Web services, using test drivers created from the Web service descriptions. The testing method comes with completeness guarantees and provides a strong method for validating the behaviour of Web services

Enabling Proactive Adaptation through Just-in-time Testing of Conversational Services

Service-based applications (SBAs) will increasingly be composed of third-party services available over the Internet. Reacting to failures of those third-party services by dynamically adapting the SBAs will become a key enabler for ensuring reliability. Determining when to adapt an SBA is especially challenging in the presence of conversational (aka. stateful) services. A conversational service might fail in the middle of an invocation sequence, in which case adapting the SBA might be costly; e.g., due to the necessary state transfer to an alternative service. In this paper we propose just-in-time testing of conversational services as a novel approach to detect potential problems and to proactively trigger adaptations, thereby preventing costly compensation activities. The approach is based on a framework for online testing and a formal test-generation method which guarantees functional correctness for conversational services. The applicability of the approach is discussed with respect to its underlying assumptions and its performance. The benefits of the approach are demonstrated using a realistic example

Automating property-based testing of evolving web services

Web services are the most widely used service technology that drives the Service-Oriented Computing~(SOC) paradigm. As a result, effective testing of web services is getting increasingly important. In this paper, we present a framework and toolset for testing web services and for evolving test code in sync with the evolution of web services. Our approach to testing web services is based on the Erlang programming language and QuviQ QuickCheck, a property-based testing tool written in Erlang, and our support for test code evolution is added to Wrangler, the Erlang refactoring tool. The key components of our system include the automatic generation of initial test code, the inference of web service interface changes between versions, the provision of a number of domain specific refactorings and the automatic generation of refactoring scripts for evolving the test code. Our framework provides users with a powerful and expressive web service testing framework, while minimising users' effort in creating, maintaining and evolving the test model. The framework presented in this paper can be used by both web service providers and consumers, and can be used to test web services written in whatever language; the approach advocated here could also be adopted in other property-based testing frameworks and refactoring tools

Discovery and Selection of Certified Web Services Through Registry-Based Testing and Verification

Reliability and trust are fundamental prerequisites for the establishment of functional relationships among peers in a Collaborative Networked Organisation (CNO), especially in the context of Virtual Enterprises where economic benefits can be directly at stake. This paper presents a novel approach towards effective service discovery and selection that is no longer based on informal, ambiguous and potentially unreliable service descriptions, but on formal specifications that can be used to verify and certify the actual Web service implementations. We propose the use of Stream X-machines (SXMs) as a powerful modelling formalism for constructing the behavioural specification of a Web service, for performing verification through the generation of exhaustive test cases, and for performing validation through animation or model checking during service selection

Machine-Readable Privacy Certificates for Services

Privacy-aware processing of personal data on the web of services requires managing a number of issues arising both from the technical and the legal domain. Several approaches have been proposed to matching privacy requirements (on the clients side) and privacy guarantees (on the service provider side). Still, the assurance of effective data protection (when possible) relies on substantial human effort and exposes organizations to significant (non-)compliance risks. In this paper we put forward the idea that a privacy certification scheme producing and managing machine-readable artifacts in the form of privacy certificates can play an important role towards the solution of this problem. Digital privacy certificates represent the reasons why a privacy property holds for a service and describe the privacy measures supporting it. Also, privacy certificates can be used to automatically select services whose certificates match the client policies (privacy requirements). Our proposal relies on an evolution of the conceptual model developed in the Assert4Soa project and on a certificate format specifically tailored to represent privacy properties. To validate our approach, we present a worked-out instance showing how privacy property Retention-based unlinkability can be certified for a banking financial service.Comment: 20 pages, 6 figure

Browser-based Analysis of Web Framework Applications

Although web applications evolved to mature solutions providing sophisticated user experience, they also became complex for the same reason. Complexity primarily affects the server-side generation of dynamic pages as they are aggregated from multiple sources and as there are lots of possible processing paths depending on parameters. Browser-based tests are an adequate instrument to detect errors within generated web pages considering the server-side process and path complexity a black box. However, these tests do not detect the cause of an error which has to be located manually instead. This paper proposes to generate metadata on the paths and parts involved during server-side processing to facilitate backtracking origins of detected errors at development time. While there are several possible points of interest to observe for backtracking, this paper focuses user interface components of web frameworks.Comment: In Proceedings TAV-WEB 2010, arXiv:1009.330