A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

Peer to Peer Information Retrieval: An Overview

Peer-to-peer technology is widely used for file sharing. In the past decade a number of prototype peer-to-peer information retrieval systems have been developed. Unfortunately, none of these have seen widespread real- world adoption and thus, in contrast with file sharing, information retrieval is still dominated by centralised solutions. In this paper we provide an overview of the key challenges for peer-to-peer information retrieval and the work done so far. We want to stimulate and inspire further research to overcome these challenges. This will open the door to the development and large-scale deployment of real-world peer-to-peer information retrieval systems that rival existing centralised client-server solutions in terms of scalability, performance, user satisfaction and freedom

Use of latent semantic indexing for content based searching and routing of mobile agents on P2P network

The peer-to-peer (P2P) system has a number of nodes that are connected to each other in an unstructured or a structured overlay network. One of the most important problems in a P2P system is locating of resources that are shared by various nodes. Techniques such as Flooding and Distributed Hash-Table (DHT) have been proposed to locate resources shared by various nodes. Flooding suffers from saturation as number of nodes increase, while DHT cannot handle multiple keys to define and search a resource. Various further research works including multi agent systems (MAS) have been pursued that take unstructured or structured networks as a backbone and hence inherently suffer from problems. We present the solution that is more efficient and effective for discovering shared resources on a network that is influenced by content shared by nodes. Our solution presents use of multiple agents that manage the shared information on a node and a mobile agent called Reconnaissance Agent (RA) that is responsible for querying various nodes. To reduce the search load on nodes that have unrelated content, an efficient migration route is proposed for RA that is based on cosine similarity of content shared by nodes and user query. Results show reduction in search load and traffic due to communication, and increase in recall value for locating of resources defined by multiple keys using RA that are logically similar to user query. Furthermore, the results indicate that by use of our technique the relevance of search results is higher; that is obtained by minimal traffic generation/communication and hops made by RA