A Peer-to-Peer Middleware Framework for Resilient Persistent Programming

The persistent programming systems of the 1980s offered a programming model that integrated computation and long-term storage. In these systems, reliable applications could be engineered without requiring the programmer to write translation code to manage the transfer of data to and from non-volatile storage. More importantly, it simplified the programmer's conceptual model of an application, and avoided the many coherency problems that result from multiple cached copies of the same information. Although technically innovative, persistent languages were not widely adopted, perhaps due in part to their closed-world model. Each persistent store was located on a single host, and there were no flexible mechanisms for communication or transfer of data between separate stores. Here we re-open the work on persistence and combine it with modern peer-to-peer techniques in order to provide support for orthogonal persistence in resilient and potentially long-running distributed applications. Our vision is of an infrastructure within which an application can be developed and distributed with minimal modification, whereupon the application becomes resilient to certain failure modes. If a node, or the connection to it, fails during execution of the application, the objects are re-instantiated from distributed replicas, without their reference holders being aware of the failure. Furthermore, we believe that this can be achieved within a spectrum of application programmer intervention, ranging from minimal to totally prescriptive, as desired. The same mechanisms encompass an orthogonally persistent programming model. We outline our approach to implementing this vision, and describe current progress.Comment: Submitted to EuroSys 200

A middleware service for fault-tolerant group communications

PhD ThesisMany distributed applications require multicast group communication services, enabling an entity to interact with a group of other entities. Providing the reliability and ordering guarantees required by group based applications is not a trivial task in distributed systems where computation and communication delays might not be known accurately. Furthermore, the approaches available to support these guarantees are diverse. The choice of approach may significantly effect the performance of an application and/or may not be suitable for some application types. Nowadays, distributed applications are frequently built as a Middleware service. The Thesis develops techniques for providing group communication support in Middleware environments. A group communication service has been designed and implemented in such a way as not to hinder the interoperability/portability of applications built using it. The service provides a variety of functions that may be tailored to suit many different types of applications. Group communication protocols are presented that ensure reliability and ordering guarantees. Furthermore, the reliability and ordering guarantees of such protocols may be tailored to suit a wide variety of applications. Mechanisms that provide a variety of approaches to inter-member and inter-group interactions that are suitable for satisfying the requirements of many different types of applications (e.g., fault- tolerant, collaborative) are also supported. The service can work over local and wide area networks (Internet).Hewlett Packard laboratories Engineering and Physical Science Research Counci

JMSGroups:JMS compliant group communication

Nowadays, computers are the indispensable part of our life. They evolve rapidly and are more and more versatile. Computer networks made the remote corners of the world just a click away. But unavoidably, any software and hardware component is subject to failure. Distributed systems spread on tens or hundreds of machines are particularly vulnerable to failures. Consequently, high availability and fault tolerance became a "must have" feature for such systems. Software fault tolerance is achieved through the technique called replication. In replication several software replicas are executed at the same time. If one or several of them fail, other still provide the service. Software replication is often implemented using group communication, which provides communication primitives with various semantics and greatly simplifies the development of highly available and fault tolerant services. However, despite tremendous advances in research and numerous prototypes, group communication stays confined to small niches and academic prototypes. In contrast, other technology, called messageoriented middleware such as the Java Message Service (JMS) is widely used in distributed systems, and has become a de-facto standard. We believe that the lack of a well-defined and easily understandable standard is the reason that hinders the deployment of group communication systems. Since JMS is a well-established technology, we propose to extend JMS adding group communication primitives to it. Foremost, this requires to extend the traditional semantics of group communication in order to take into account various features of JMS, e.g., durable/non-durable subscriptions and persistent/non-persistent messages. The resulting new group communication specification, together with the corresponding API, defines group communication primitives compatible with JMS, that we call JMSGroups. To validate the specification and API we provide a prototype implementation of JMSGroups. As such, we believe it facilitates the acceptance of group communication by a larger community and provides a powerful environment for building fault-tolerant applications

Dynamic Upgrade of Distributed Software Components

Die Aktualisierung von komplexen Telekommunikationssystemen, die sich durch die ihnen eigene Verteiltheit und hohe Kosten bei System-Nichtverfügbarkeit auszeichnen, ist ein komplizierter und fehleranfälliger Wartungsprozess. Noch stärkere Herausforderungen bergen solche Software-Aktualisierungen, die die Systemverfügbarkeit nicht beeinträchtigen sollen. Dynamic Upgrade ist eine Wartungstechnik, die das Verwalten und die Durchführung von Software-Aktualisierung automatisiert und damit den Betrieb des Systems während der Wartungszeit nicht unterbricht. In dieser Arbeit wird das Dynamic Upgrade als ein Sonderfall der Bereitstellung und Inbetriebnahme (Deployment) von Software betrachtet, in dem Teile der einen Dienst repräsentierenden Software durch neue Versionen im laufenden Betrieb ersetzt werden. Die Problemstellung des Dynamic Upgrade wird anhand einer vom Autor erarbeiteten Taxonomie erläutert, die die Entwurfsmöglichkeiten für ein System zur Unterstützung von Dynamic Upgrade hinsichtlich dreier Systemaspekte klassifiziert: Deployment, Evolution und Zuverlässigkeit (Dependability). Mit Hilfe dieser Taxonomie lassen sich auch andere Systeme zur Unterstützung von Dynamic Upgrade miteinander vergleichen. Aufbauend auf einem ausführlichen Vergleich über existierende Ansätze zur Unterstützung von Dynamic Upgrade, wird in der vorliegenden Arbeit eine Lösung entwickelt und dargestellt, die Dynamic Upgrade in verteilten komponentenbasierten Software-Systemen ermöglicht. Ausgehend von der Problemanalyse wird mit Hilfe des Unified Process ein als Deployment and Upgrade Facility bezeichnetes Modell entwickelt, das sowohl die benötigten Leistungsfähigkeiten eines Dynamic Upgrade unterstützenden Systems als auch Eigenschaften von aktualisierbaren Software-Komponenten beschreibt. Dieses Modell ist Plattform-unabhängig und einsetzbar für mehrere unterliegende Middleware-Technologien. Das Modell wird in einem Java-basierten prototypischen Rahmenwerk programmiert und um plattformspezifische Mechanismen auf der Jgroup/ARM Middleware erweitert. Das Rahmenwerk umfasst allgemeine Entwurfslösungen und ?muster, die sich für die Konstruktion einer Unterstützung für Dynamic Upgrade eignen. Es erlaubt die Kontrolle der Lebenszyklen von Aktualisierungsprozessen und ihre Koordination im Zielsystem. Darüber hinaus definiert es eine Reihe von Unterstützungsmechanismen und Algorithmen für den dynamischen Aktualisierungsprozess, der gegebenenfalls mit unterschiedlichen Zielsetzungen und unter verschiedenen Randbedingungen erfolgen soll. Insbesondere wird ein Aktualisierungsalgorithmus für replizierte Software-Komponenten dargestellt. Das entwickelte Rahmenwerk wird zwecks Plausibilitätsprüfung der dargestellten Ansätze und zur Auswertung der Auswirkungen der Dynamic Upgrade unterstützenden Mechanismen im Hinblick auf Systemperformanz in mehreren Experimenten eingesetzt. Diese quantitative Evaluierung der Experimente führt zu einer Spezifikationen eines einfachen Bewertungsmaßstabs (Benchmark), der sich zum Vergleich von Dynamic Upgrade unterstützenden Systemen eignet.Upgrading complex telecommunication software systems, characterized by their inherent distribution and a very high cost of system unavailability, is a difficult and error-prone maintenance activity. Even more challenging are such software upgrades that do not compromise the system availability. Dynamic upgrades is a technique, which automates performing and managing upgrades so that the software system remains operational during the upgrade time. In this thesis, the dynamic upgrade is considered as a special case of software deployment, in which a running service has to be replaced with its new version. The problems of dynamic upgrades are introduced using a novel taxonomy that classifies the design issues to be solved when building support for dynamic upgrade with regard to three system aspects: deployment, evolution and dependability and provides a reference to comparing other systems supporting dynamic upgrades. An extensive and thorough survey of existing approaches to dynamic upgrades follows and, furthermore, is as a starting point to designing a solution supporting dynamic upgrades in distributed component-based software systems. Derived from the problem analysis, a model called Deployment and Upgrade Facility describing the capabilities needed for managing and performing dynamic upgrades as well as properties of upgradable software components is developed using the Unified Process approach. The model is platform independent and can be used with a range of underlying middleware technologies. The model is implemented in a Java-based prototypical framework and extended with platform specific mechanisms on top of the JGroup/ARM middleware. The framework captures common design solutions and patterns for building a support for dynamic upgrade. The framework allows for controlling life-cycle and coordination of upgrade processes in the system. It also defines a number of supporting mechanisms and algorithms for the upgrade process. A special attention is drawn to an upgrade algorithm for replicated software components for achieving a synergy of replication techniques and dynamic upgrade . The developed framework is used to validate the feasibility of the approach and to measure the overhead of the mechanisms supporting dynamic upgrade with regard to the performance of the system being upgraded in a number of practical experiments. This quantitative evaluation of the experiments leads to a specification of a simple benchmark for systems supporting dynamic upgrades

Scalable Internet auctions

Current Internet based auction services rely, in general, on a centralised auction server; applications with large and geographically dispersed bidder client bases are thus supported in a centralised manner. Such an approach is fundamentally restrictive as too many users can overload the server, making the whole auction process unresponsive. Further, such an architecture can be vulnerable to server's failures, if not equipped with sufficient redundancy. In addition, bidders who are closer to the server are likely to have relatively faster access to the server than remote bidders, thereby gaining an unfair advantage. To overcome these shortcomings, this thesis investigates ways of enabling widely distributed, arbitrarily large number of auction servers to cooperate in conducting an auction. Allowing a bidder to register with anyone of the auction servers and place bids there, coupled with periodic exchange of auction information between servers forms the basis of the solution investigated to achieve scalability, responsiveness and fairness. Scalability and responsiveness are achieved since the total load is shared amongst many bidder servers; fairness is achieved since bidders are able to register with their local servers. The thesis presents the design and implementation of an hierarchically structured distributed Internet auction system. Protocols for inter-server cooperation are presented. Each server may be replicated locally to mask node failures. Performance evaluations of centralised and distributed configurations are performed to show the advantages of the distributed configuration over the centralised one.EThOS - Electronic Theses Online ServiceIranian Ministry of Science, Research and Technology : Isfahan UniversityGBUnited Kingdo

Specification of Replication Techniques, Semi-Passive Replication, and Lazy consensus*

This paper brings the following three main contributions: a hierarchy of specifications for replication techniques, semi-passive replication, and Lazy Consensus. Based on the definition of the Generic Replication problem, we difine two families of replication techniques: replication with parsimonious processing (e.g., passive replication), and replication with redundant processing (e.g., active replication). This helps relate replication techniques to each other. We define a novel replication technique with parsimonious processing, called semi-passive replication, for which we also give an algorithm. The most significant aspect of semi-passive replication is that it requires a weaker system model than existing techniques of the same family. We difine a variant of the Consensus problem, called Lazy Consensus, upon which our semi-passive replication algorithm is based. The main difference between Consensus and Lazy Consensus is a property of laziness which requires that initial values are computed only when they are actually needed

Distributed Dependancy Injection

Applications nowadays are built of objects, which collaborate in order to provide their functionality, are interconnected by default and are by no means limited to a single domain of an application, a process or a computer. In this thesis a concept of dependency injection, which enables an object to explicitly declare and require its dependencies to be provided, is distributed across domain boundaries. In support of a distributed dependency injection we provide an external tool (a container) for assembling objects and resolving their dependencies (collaborators) from across domains. We provide a model in which a group of distributed dependency injection containers connect on behalf of the applications. We provide them with a middleware solution for seamless and fault-tolerant sharing of objects/dependencies between interconnected domains. A collection of support services (i.e. the distributed object replication middleware) transparently manages replication of objects created by the dependency injection principles across multiple computers. A fresh failover is ensured by invariable consistency upon invocations. This is temporarily relaxed during degraded situations (e.g. network failures) in order to achieve availability within the isolated groups. Recovery from failures is ensured by logging and check-pointing the state of the system on a regular basis; conflicting modifications are resolved. Our proof-of-concept implementation is an add-on to .NET Remoting middleware and an extension to the Unity Container

The CORBA object group service:a service approach to object groups in CORBA

Distributed computing is one of the major trends in the computer industry. As systems become more distributed, they also become more complex and have to deal with new kinds of problems, such as partial crashes and link failures. To answer the growing demand in distributed technologies, several middleware environments have emerged during the last few years. These environments however lack support for "one-to-many" communication primitives; such primitives greatly simplify the development of several types of applications that have requirements for high availability, fault tolerance, parallel processing, or collaborative work. One-to-many interactions can be provided by group communication. It manages groups of objects and provides primitives for sending messages to all members of a group, with various reliability and ordering guarantees. A group constitutes a logical addressing facility: messages can be issued to a group without having to know the number, identity, or location of individual members. The notion of group has proven to be very useful for providing high availability through replication: a set of replicas constitutes a group, but are viewed by clients as a single entity in the system. This thesis aims at studying and proposing solutions to the problem of object group support in object-based middleware environments. It surveys and evaluates different approaches to this problem. Based on this evaluation, we propose a system model and an open architecture to add support for object groups to the CORBA middle- ware environment. In doing so, we provide the application developer with powerful group primitives in the context of a standard object-based environment. This thesis contributes to ongoing standardization efforts that aim to support fault tolerance in CORBA, using entity redundancy. The group architecture proposed in this thesis — the Object Group Service (OGS) — is based on the concept of component integration. It consists of several distinct components that provide various facilities for reliable distributed computing and that are reusable in isolation. Group support is ultimately provided by combining these components. OGS defines an object-oriented framework of CORBA components for reliable distributed systems. The OGS components include a group membership service, which keeps track of the composition of object groups, a group multicast service, which provides delivery of messages to all group members, a consensus service, which allows several CORBA objects to resolve distributed agreement problems, and a monitoring service, which provides distributed failure detection mechanisms. OGS includes support for dynamic group membership and for group multicast with various reliability and ordering guarantees. It defines interfaces for active and primary-backup replication. In addition, OGS proposes several execution styles and various levels of transparency. A prototype implementation of OGS has been realized in the context of this thesis. This implementation is available for two commercial ORBs (Orbix and VisiBroker). It relies solely on the CORBA specification, and is thus portable to any compliant ORB. Although the main theme of this thesis deals with system architecture, we have developed some original algorithms to implement group support in OGS. We analyze these algorithms and implementation choices in this dissertation, and we evaluate them in terms of efficiency. We also illustrate the use of OGS through example applications