Small-world networks, distributed hash tables and the e-resource discovery problem

Resource discovery is one of the most important underpinning problems behind producing a scalable, robust and efficient global infrastructure for e-Science. A number of approaches to the resource discovery and management problem have been made in various computational grid environments and prototypes over the last decade. Computational resources and services in modern grid and cloud environments can be modelled as an overlay network superposed on the physical network structure of the Internet and World Wide Web. We discuss some of the main approaches to resource discovery in the context of the general properties of such an overlay network. We present some performance data and predicted properties based on algorithmic approaches such as distributed hash table resource discovery and management. We describe a prototype system and use its model to explore some of the known key graph aspects of the global resource overlay network - including small-world and scale-free properties

WebWave: Globally Load Balanced Fully Distributed Caching of Hot Published Documents

Document publication service over such a large network as the Internet challenges us to harness available server and network resources to meet fast growing demand. In this paper, we show that large-scale dynamic caching can be employed to globally minimize server idle time, and hence maximize the aggregate server throughput of the whole service. To be efficient, scalable and robust, a successful caching mechanism must have three properties: (1) maximize the global throughput of the system, (2) find cache copies without recourse to a directory service, or to a discovery protocol, and (3) be completely distributed in the sense of operating only on the basis of local information. In this paper, we develop a precise definition, which we call tree load-balance (TLB), of what it means for a mechanism to satisfy these three goals. We present an algorithm that computes TLB off-line, and a distributed protocol that induces a load distribution that converges quickly to a TLB one. Both algorithms place cache copies of immutable documents, on the routing tree that connects the cached document's home server to its clients, thus enabling requests to stumble on cache copies en route to the home server.Harvard University; The Saudi Cultural Mission to the U.S.A

Optimal Caching and Routing in Hybrid Networks

Hybrid networks consisting of MANET nodes and cellular infrastructure have been recently proposed to improve the performance of military networks. Prior work has demonstrated the benefits of in-network content caching in a wired, Internet context. We investigate the problem of developing optimal routing and caching policies in a hybrid network supporting in-network caching with the goal of minimizing overall content-access delay. Here, needed content may always be accessed at a back-end server via the cellular infrastructure; alternatively, content may also be accessed via cache-equipped "cluster" nodes within the MANET. To access content, MANET nodes must thus decide whether to route to in-MANET cluster nodes or to back-end servers via the cellular infrastructure; the in-MANET cluster nodes must additionally decide which content to cache. We model the cellular path as either i) a congestion-insensitive fixed-delay path or ii) a congestion-sensitive path modeled as an M/M/1 queue. We demonstrate that under the assumption of stationary, independent requests, it is optimal to adopt static caching (i.e., to keep a cache's content fixed over time) based on content popularity. We also show that it is optimal to route to in-MANET caches for content cached there, but to route requests for remaining content via the cellular infrastructure for the congestion-insensitive case and to split traffic between the in-MANET caches and cellular infrastructure for the congestion-sensitive case. We develop a simple distributed algorithm for the joint routing/caching problem and demonstrate its efficacy via simulation.Comment: submitted to Milcom 201

A Highly Available Cluster of Web Servers with Increased Storage Capacity

Ponencias de las Decimoséptimas Jornadas de Paralelismo de la Universidad de Castilla-La Mancha celebradas el 18,19 y 20 de septiembre de 2006 en AlbaceteWeb servers scalability has been traditionally solved by improving software elements or increasing hardware resources of the server machine. Another approach has been the usage of distributed architectures. In such architectures, usually, file al- location strategy has been either full replication or full distribution. In previous works we have showed that partial replication offers a good balance between storage capacity and reliability. It offers much higher storage capacity while reliability may be kept at an equivalent level of that from fully replicated solutions. In this paper we present the architectural details of Web cluster solutions adapted to partial replication. We also show that partial replication does not imply a penalty in performance over classical fully replicated architectures. For evaluation purposes we have used a simulation model under the OMNeT++ framework and we use mean service time as a performance comparison metric.Publicad