An evaluation of compression and streaming techniques for efficient transfer of XML documents with Simple Object Access Protocol (SOAP)

Masteroppgave i informasjons- og kommunikasjonsteknologi 2003 - Høgskolen i Agder, GrimstadIn SOAP, the entire XML object is generated on the server before it is returned to the client. This puts unnecessary strain on server systems in terms of both memory and CPU. The objectives are to find why SOAP does not allow streaming of responses, possible solutions to the problem and outline alternative transfer methods. Furthermore, compression techniques for a streaming SOAP environment are evaluated, as well as performance of streaming versus an alternative method of data retrieval. The feasibility study concluded that SOAP itself allows streaming of responses, but the HTTP binding does not. This binding specifies the issue of a HTTP fault code in case of a SOAP processing error, meaning the processing must be completed before a HTTP code can legally be issued. One alternative to streaming is using a Request/N-Response message pattern, and dividing the data over several responses. As HTTP only supports a Request/Response message pattern, implementing this is not possible. Either the HTTP binding must be rewritten to allow streaming of responses while processing a request or HTTP must be replaced with for example DIME as the transfer protocol for SOAP to overcome these problems. Tests are set up to find the most suitable compressor technique and to verify that streaming SOAP responses utilize server resources better then alternative transfer methods. Results show that bzip2 is the most suitable compressor technique. And that streaming utilizes memory considerably more efficient, especially with multiple clients connecting

Enhanced SOAP Performance for low bandwidth environments

It is desirable that SOAP performs efficiently in environments where there are a large number of transactions. However, SOAP is based on XML and therefore inherits XML's disadvantage of having voluminous messages. Firstly, the performance of different SOAP bindings is investigated. A benchmark of different SOAP bindings in wireless environments demonstrates the unsuitability of HTTP and TCP bindings in limited bandwidth environments. UDP is recommended as an alternative transport protocol for SOAP. Secondly, the thesis examines the use of multicast in reducing the traffic caused by SOAP messages in low bandwidth environments to deal with challenges described. A novel SOAP-level multicast protocol based on the similarity of SOAP messages, called SMP (Similarity-based SOAP Multicast Protocol), is proposed. In particular, issues of traffic, network optimization, response time and scalability are investigated. Lastly, two extensions of SMP are proposed to further improve the performance of SMP. SMP's extensions are two algorithms, greedy and incremental tc-SMP, for traffic-constrained similarity-based SOAP multicast. Tc-SMP optimizes network traffic by building its own spanning trees instead of using the one built by traditional methods, such as Dijkstra's algorithm. A new client is added to a tc-SMP tree through an existing tc-SMP node that causes minimal additional traffic for that connection. Detailed analytical models and experimental evaluations of the proposed methods demonstrate that combining SOAP messages of similar content and multicasting them as aggregated messages can significantly lower total network traffic. These improvements are advantageous for Web service applications that involve a high number of simultaneous similar transactions such as stock quotes, weather and sport event reports

Optimization of Web Services for Cloud Deployment and Mobile Consumption

Research performed for this thesis indicates an impedance mismatch between prevailing approaches to development of service-oriented enterprise applications and the consumption capabilities of mobile devices. The rich semantics and strong validation mechanisms inherent in SOAP-based web services, common to large-scale enterprise development, introduce inefficiencies of network bandwidth consumption and serialization/de-serialization processing requirements. These inefficiencies may be financially burdensome when systems are migrated to a cloud-based hosting environment and both costly and non-performant when accessed from network and processor constrained mobile devices. Yet wholesale abandonment of established enterprise practice and legacy systems for the adoption of unfamiliar architectural styles is rarely practical.  This thesis proposes a series of incremental changes to enterprise web services architecture that, individually, provide measurable efficiency benefits both when served from the cloud and when consumed from mobile devices. The objective of this research is to quantify the benefits and illustrate trade-offs for each. Within a cloud deployment, selective application of HTTP compression is shown to yield performance improvements in excess of 40% with data transfer  reductions of up to 85%. Analysis identifies the characteristics of services that suffer degraded performance under compression, and illustrates how similar performance and data reduction benefits may be achieved through service augmentation with alternative message and request formats.  Thesis focus then turns to options for improving efficiency in the consumption of these services from native applications on prevailing mobile device platforms. Development and measurements performed for this thesis identify approaches for faster and more efficient processing of existing services on mobile devices and relates these to the developer effort required. Further enhancements to application performance and development simplicity are demonstrated through mobile consumption of the augmented services and formats proposed for optimized cloud deployment. Research for this thesis suggests that in both cloud and mobile sides of a distributed system, performance and financial benefits may be achieved while building upon, rather than replacing, existing services code and architectural patterns.  M.S

Analyse et amélioration de la qualité de services WEB multimédia et leurs mises en oeuvre sur ordinateur et sur FPGA

Résumé : Les services Web, issus de l’avancée technologique dans le domaine des réseaux informatiques et des dispositifs de télécommunications portables et fixes, occupent une place primordiale dans la vie quotidienne des gens. La demande croissante sur des services Web multimédia (SWM), en particulier, augmente la charge sur les réseaux d’Internet, les fournisseurs de services et les serveurs Web. Cette charge est essentiellement due au fait que les SWM de haute qualité nécessitent des débits de transfert et des tailles de paquets importants. La qualité de service (par définition, telle que vue par l’utilisateur) est influencée par plusieurs facteurs de performance, comme le temps de traitement, le délai de propagation, le temps de réponse, la résolution d’images et l’efficacité de compression. Le travail décrit dans cette thèse est motivé par la demande continuellement croissante de nouveaux SWM et le besoin de maintenir et d’améliorer la qualité de ces services. Nous nous intéressons tout d’abord à la qualité de services (QdS) des SWM lorsqu’ils sont mis en œuvre sur des ordinateurs, tels que les ordinateurs de bureau ou les portables. Nous commençons par étudier les aspects de compatibilité afin d’obtenir des SWM fonctionnant de manière satisfaisante sur différentes plate-formes. Nous étudions ensuite la QdS des SWM lorsqu’ils sont mis en œuvre selon deux approches différentes, soit le protocole SOAP et le style RESTful. Nous étudions plus particulièrement le taux de compression qui est un des facteurs influençant la QdS. Après avoir considéré sous différents angles les SWM avec mise en œuvre sur des ordinateurs, nous nous intéressons à la QdS des SWM lorsqu’ils sont mis en œuvre sur FPGA. Nous effectuons alors une étude et une mise en œuvre qui permet d’identifier les avantages à mettre en œuvre des SWM sur FPGA. Les contributions se définissent en cinq volets comme suit : 1. Nous introduisons des méthodes de création, c’est-à-dire conception et mise en œuvre, de SWM sur des plate-formes logicielles hétérogènes dans différents environnements tels que Windows, OS X et Solaris. Un objectif que nous visons est de proposer une approche permettant d’ajouter de nouveaux SWM tout en garantissant la compatibilité entre les plate-formes, dans le sens où nous identifions les options nous permettant d’offrir un ensemble riche et varié de SWM pouvant fonctionner sur les différentes plate-formes. 2. Nous identifions une liste de paramètres pertinents influençant la QdS des SWM mis en œuvre selon le protocole SOAP et selon le style REST. 3. Nous développons un environnement d’analyse pour quantifier les impacts de chaque paramètre identifié sur la QdS de SWM. Pour cela, nous considérons les SWM mis en œuvre selon le protocole SOAP et aussi selon style REST. Les QdS obtenues avec SOAP et REST sont comparées objectivement. Pour faciliter la comparaison, la même gamme d’images (dans l’analyse de SWM SOAP) a été réutilisée et les mêmes plate-formes logicielles. 4. Nous développons une procédure d’analyse qui permet de déterminer une corrélation entre la dimension d’une image et le taux de compression adéquat. Les résultats obtenus confirment cette contribution propre à cette thèse qui confirme que le taux de compression peut être optimisé lorsque les dimensions de l’image ont la propriété suivante : le rapport entre la longueur et la largeur est égal au nombre d’or connu dans la nature. Trois libraires ont été utilisées à savoir JPEG, JPEG2000 et DjVu. 5. Dans un volet complémentaire aux quatre volets précédents, qui concernent les SWM sur ordinateurs, nous étudions ainsi la conception et la mise en œuvre de SWM sur FPGA. Nous justifions l’option de FPGA en identifiant ses avantages par rapport à deux autres options : ordinateurs et ASICs. Afin de confirmer plusieurs avantages identifiés, un SWM de QdS élevée et de haute performance est créé sur FPGA, en utilisant des outils de conception gratuits, du code ouvert (open-source) et une méthode fondée uniquement sur HDL. Notre approche facilitera l’ajout d’autres modules de gestions et d’orchestration de SWM. 6. La mise à jour et l’adaptation du code open-source et de la documentation du module Ethernet IP Core pour la communication entre le FPGA et le port Ethernet sur la carte Nexys3. Ceci a pour effet de faciliter la mise en œuvre de SWM sur la carte Nexys3. // Abstract : Web services, which are the outcome of the technological advancements in IT networks and hand-held mobile devices for telecommunications, occupy an important role in our daily life. The increasing demand on multimedia Web services (MWS), in particular, augments the load on the Internet, on service providers and Web servers. This load is mainly due to the fact that the high-quality multimedia Web services necessitate high data transfer rates and considerable payload sizes. The quality of service (QoS, by definition as it is perceived by the user) is influenced by several factors, such as processing time, propagation delay, response time, image resolution and compression efficacy. The research work in this thesis is motivated by the persistent demand on new MWS, and the need to maintain and improve the QoS. Firstly, we focus on the QoS of MWS when they are implemented on desktop and laptop computers. We start with studying the compatibility aspects in order to obtain MWS functioning satisfactorily on different platforms. Secondly, we study the QoS for MWS implemented according to the SOAP protocol and the RESTful style. In particular, we study the compression rate, which is one of the pertinent factors influencing the QoS. Thirdly, after the study of MWS when implemented on computers, we proceed with the study of QoS of MWS when implemented on hardware, in particular on FPGAs. We achieved thus comprehensive study and implementations that show and compare the advantages of MWS on FPGAs. The contributions of this thesis can be resumed as follows: 1. We introduce methods of design and implementation of MWS on heterogeneous platforms, such as Windows, OS X and Solaris. One of our objectives is to propose an approach that facilitates the integration of new MWS while assuring the compatibility amongst involved platforms. This means that we identify the options that enable offering a set of rich and various MWS that can run on different platforms. 2. We determine a list of relevant parameters that influence the QoS of MWS. 3. We build an analysis environment that quantifies the impact of each parameter on the QoS of MWS implemented on both SOAP protocol and RESTful style. Both QoS for SOAP and REST are objectively compared. The analysis has been held on a large scale of different images, which produces a realistic point of view describing the behaviour of real MWS. 4. We develop an analysis procedure to determine the correlation between the aspect ratio of an image and its compression ratio. Our results confirm that the compression ratio can be improved and optimised when the aspect ratio of iiiiv an image is close to the golden ratio, which exists in nature. Three libraries of compression schemes have been used, namely: JPEG, JPEG2000 and DjVu. 5. Complementary to the four contributions mentioned above, which concern the MWS on computers, we study also the design and implementation of MWS on FPGA. This is justified by the numerous advantages that are offered by FPGAs, compared to the other technologies such as computers and ASICs. In order to highlight the advantages of implementing MWS on FPGA, we developed on FPGA a MWS of high performance and high level of QoS. To achieve our goal, we utilised freely available design utilities, open-source code and a method based only on HDL. This approach is adequate for future extensions and add-on modules for MWS orchestration