A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

Autonomic Management of Maintenance Scheduling in Chord

This paper experimentally evaluates the effects of applying autonomic management to the scheduling of maintenance operations in a deployed Chord network, for various membership churn and workload patterns. Two versions of an autonomic management policy were compared with a static configuration. The autonomic policies varied with respect to the aggressiveness with which they responded to peer access error rates and to wasted maintenance operations. In most experiments, significant improvements due to autonomic management were observed in the performance of routing operations and the quantity of data transmitted between network members. Of the autonomic policies, the more aggressive version gave slightly better results

Hosting Byzantine Fault Tolerant Services on a Chord Ring

In this paper we demonstrate how stateful Byzantine Fault Tolerant services may be hosted on a Chord ring. The strategy presented is fourfold: firstly a replication scheme that dissociates the maintenance of replicated service state from ring recovery is developed. Secondly, clients of the ring based services are made replication aware. Thirdly, a consensus protocol is introduced that supports the serialization of updates. Finally Byzantine fault tolerant replication protocols are developed that ensure the integrity of service data hosted on the ring.Comment: Submitted to DSN 2007 Workshop on Architecting Dependable System

Achieving Robust Self-Management for Large-Scale Distributed Applications

Autonomic managers are the main architectural building blocks for constructing self-management capabilities of computing systems and applications. One of the major challenges in developing self-managing applications is robustness of management elements which form autonomic managers. We believe that transparent handling of the effects of resource churn (joins/leaves/failures) on management should be an essential feature of a platform for self-managing large-scale dynamic distributed applications, because it facilitates the development of robust autonomic managers and hence improves robustness of self-managing applications. This feature can be achieved by providing a robust management element abstraction that hides churn from the programmer. In this paper, we present a generic approach to achieve robust services that is based on finite state machine replication with dynamic reconfiguration of replica sets. We contribute a decentralized algorithm that maintains the set of nodes hosting service replicas in the presence of churn. We use this approach to implement robust management elements as robust services that can operate despite of churn. Our proposed decentralized algorithm uses peer-to-peer replica placement schemes to automate replicated state machine migration in order to tolerate churn. Our algorithm exploits lookup and failure detection facilities of a structured overlay network for managing the set of active replicas. Using the proposed approach, we can achieve a long running and highly available service, without human intervention, in the presence of resource churn. In order to validate and evaluate our approach, we have implemented a prototype that includes the proposed algorithm

Theoretical and Computational Basis for CATNETS - Annual Report Year 2

In this work the self-organising potential of the CATNETS allocation mechanism is described to provide a more comprehensive view on the research done in this project. The formal description of either the centralised and decentralised approach is presented. Furthermore the agents' bidding model is described and a comprehensive overview on how the catallactic mechanism is incorporated into the middleware and simulator environments is given. --Decentralized Market Mechanisms,Centralized Market Mechanisms,Catallaxy,Market Engineering,Simulator Integration,Prototype Integration