Application-level differential checkpointing for HPC applications with dynamic datasets

High-performance computing (HPC) requires resilience techniques such as checkpointing in order to tolerate failures in supercomputers. As the number of nodes and memory in supercomputers keeps on increasing, the size of checkpoint data also increases dramatically, sometimes causing an I/O bottleneck. Differential checkpointing (dCP) aims to minimize the checkpointing overhead by only writing data differences. This is typically implemented at the memory page level, sometimes complemented with hashing algorithms. However, such a technique is unable to cope with dynamic-size datasets. In this work, we present a novel dCP implementation with a new file format that allows fragmentation of protected datasets in order to support dynamic sizes. We identify dirty data blocks using hash algorithms. In order to evaluate the dCP performance, we ported the HPC applications xPic, LULESH 2.0 and Heat2D and analyze them regarding their potential of reducing I/O with dCP and how this data reduction influences the checkpoint performance. In our experiments, we achieve reductions of up to 62% of the checkpoint time.This project has received funding from the European Unions Seventh Framework Programme (FP7/2007-2013) and the Horizon 2020 (H2020) funding framework under grant agreement no. H2020-FETHPC-754304 (DEEP-EST); and from the European Unions Horizon 2020 research and innovation programme under the LEGaTO Project (legato- project.eu), grant agreement No 780681.Peer ReviewedPostprint (author's final draft

Using cooperation to improve the experience of web services consumers

Web Services (WS) are one of the most promising approaches for building loosely coupled systems. However, due to the heterogeneous and dynamic nature of the WS environment, ensuring good QoS is still non-trivial. While WS tend to scale better than tightly coupled systems, they introduce a larger communication overhead and are more susceptible to server/resource latency. Traditionally this problem has been addressed by relying on negotiated Service Level Agreement to ensure the required QoS, or the development of elaborate compensation handlers to minimize the impact of undesirable latency. This research focuses on the use of cooperation between consumers and providers as an effective means of optimizing resource utilization and consumer experiences. It introduces a novel cooperative approach to implement the cooperation between consumers and providers

Enhanced SOAP Performance for low bandwidth environments

It is desirable that SOAP performs efficiently in environments where there are a large number of transactions. However, SOAP is based on XML and therefore inherits XML's disadvantage of having voluminous messages. Firstly, the performance of different SOAP bindings is investigated. A benchmark of different SOAP bindings in wireless environments demonstrates the unsuitability of HTTP and TCP bindings in limited bandwidth environments. UDP is recommended as an alternative transport protocol for SOAP. Secondly, the thesis examines the use of multicast in reducing the traffic caused by SOAP messages in low bandwidth environments to deal with challenges described. A novel SOAP-level multicast protocol based on the similarity of SOAP messages, called SMP (Similarity-based SOAP Multicast Protocol), is proposed. In particular, issues of traffic, network optimization, response time and scalability are investigated. Lastly, two extensions of SMP are proposed to further improve the performance of SMP. SMP's extensions are two algorithms, greedy and incremental tc-SMP, for traffic-constrained similarity-based SOAP multicast. Tc-SMP optimizes network traffic by building its own spanning trees instead of using the one built by traditional methods, such as Dijkstra's algorithm. A new client is added to a tc-SMP tree through an existing tc-SMP node that causes minimal additional traffic for that connection. Detailed analytical models and experimental evaluations of the proposed methods demonstrate that combining SOAP messages of similar content and multicasting them as aggregated messages can significantly lower total network traffic. These improvements are advantageous for Web service applications that involve a high number of simultaneous similar transactions such as stock quotes, weather and sport event reports

Skalierbare Ausführung von Prozessanwendungen in dienstorientierten Umgebungen

Die Strukturierung und Nutzung von unternehmensinternen IT-Infrastrukturen auf Grundlage dienstorientierter Architekturen (SOA) und etablierter XML-Technologien ist in den vergangenen Jahren stetig gewachsen. Lag der Fokus anfänglicher SOA-Realisierungen auf der flexiblen Ausführung klassischer, unternehmensrelevanter Geschäftsprozesse, so bilden heutzutage zeitnahe Datenanalysen sowie die Überwachung von geschäftsrelevanten Ereignissen weitere wichtige Anwendungsklassen, um sowohl kurzfristig Probleme des Geschäftsablaufes zu identifizieren als auch um mittel- und langfristige Veränderungen im Markt zu erkennen und die Geschäftsprozesse des Unternehmens flexibel darauf anzupassen. Aufgrund der geschichtlich bedingten, voneinander unabhängigen Entwicklung der drei Anwendungsklassen, werden die jeweiligen Anwendungsprozesse gegenwärtig in eigenständigen Systemen modelliert und ausgeführt. Daraus resultiert jedoch eine Reihe von Nachteilen, welche diese Arbeit aufzeigt und ausführlich diskutiert. Vor diesem Hintergrund beschäftigte sich die vorliegende Arbeit mit der Ableitung einer konsolidierten Ausführungsplattform, die es ermöglicht, Prozesse aller drei Anwendungsklassen gemeinsam zu modellieren und in einer SOA-basierten Infrastruktur effizient auszuführen. Die vorliegende Arbeit adressiert die Probleme einer solchen konsolidierten Ausführungsplattform auf den drei Ebenen der Dienstkommunikation, der Prozessausführung und der optimalen Verteilung von SOA-Komponenten in einer Infrastruktur