10,938 research outputs found
Replica Selection in the Globus Data Grid
The Globus Data Grid architecture provides a scalable infrastructure for the
management of storage resources and data that are distributed across Grid
environments. These services are designed to support a variety of scientific
applications, ranging from high-energy physics to computational genomics, that
require access to large amounts of data (terabytes or even petabytes) with
varied quality of service requirements. By layering on a set of core services,
such as data transport, security, and replica cataloging, one can construct
various higher-level services. In this paper, we discuss the design and
implementation of a high-level replica selection service that uses information
regarding replica location and user preferences to guide selection from among
storage replica alternatives. We first present a basic replica selection
service design, then show how dynamic information collected using Globus
information service capabilities concerning storage system properties can help
improve and optimize the selection process. We demonstrate the use of Condor's
ClassAds resource description and matchmaking mechanism as an efficient tool
for representing and matching storage resource capabilities and policies
against application requirements.Comment: 8 pages, 6 figure
A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing
Data Grids have been adopted as the platform for scientific communities that
need to share, access, transport, process and manage large data collections
distributed worldwide. They combine high-end computing technologies with
high-performance networking and wide-area storage management techniques. In
this paper, we discuss the key concepts behind Data Grids and compare them with
other data sharing and distribution paradigms such as content delivery
networks, peer-to-peer networks and distributed databases. We then provide
comprehensive taxonomies that cover various aspects of architecture, data
transportation, data replication and resource allocation and scheduling.
Finally, we map the proposed taxonomy to various Data Grid systems not only to
validate the taxonomy but also to identify areas for future exploration.
Through this taxonomy, we aim to categorise existing systems to better
understand their goals and their methodology. This would help evaluate their
applicability for solving similar problems. This taxonomy also provides a "gap
analysis" of this area through which researchers can potentially identify new
issues for investigation. Finally, we hope that the proposed taxonomy and
mapping also helps to provide an easy way for new practitioners to understand
this complex area of research.Comment: 46 pages, 16 figures, Technical Repor
The Impact of Data Replicatino on Job Scheduling Performance in Hierarchical data Grid
In data-intensive applications data transfer is a primary cause of job
execution delay. Data access time depends on bandwidth. The major bottleneck to
supporting fast data access in Grids is the high latencies of Wide Area
Networks and Internet. Effective scheduling can reduce the amount of data
transferred across the internet by dispatching a job to where the needed data
are present. Another solution is to use a data replication mechanism. Objective
of dynamic replica strategies is reducing file access time which leads to
reducing job runtime. In this paper we develop a job scheduling policy and a
dynamic data replication strategy, called HRS (Hierarchical Replication
Strategy), to improve the data access efficiencies. We study our approach and
evaluate it through simulation. The results show that our algorithm has
improved 12% over the current strategies.Comment: 11 pages, 7 figure
The NorduGrid architecture and tools
The NorduGrid project designed a Grid architecture with the primary goal to
meet the requirements of production tasks of the LHC experiments. While it is
meant to be a rather generic Grid system, it puts emphasis on batch processing
suitable for problems encountered in High Energy Physics. The NorduGrid
architecture implementation uses the \globus{} as the foundation for various
components, developed by the project. While introducing new services, the
NorduGrid does not modify the Globus tools, such that the two can eventually
co-exist. The NorduGrid topology is decentralized, avoiding a single point of
failure. The NorduGrid architecture is thus a light-weight, non-invasive and
dynamic one, while robust and scalable, capable of meeting most challenging
tasks of High Energy Physics.Comment: Talk from the 2003 Computing in High Energy Physics and Nuclear
Physics (CHEP03), La Jolla, Ca, USA, March 2003, 9 pages,LaTeX, 4 figures.
PSN MOAT00
Next-Generation EU DataGrid Data Management Services
We describe the architecture and initial implementation of the
next-generation of Grid Data Management Middleware in the EU DataGrid (EDG)
project.
The new architecture stems out of our experience and the users requirements
gathered during the two years of running our initial set of Grid Data
Management Services. All of our new services are based on the Web Service
technology paradigm, very much in line with the emerging Open Grid Services
Architecture (OGSA). We have modularized our components and invested a great
amount of effort towards a secure, extensible and robust service, starting from
the design but also using a streamlined build and testing framework.
Our service components are: Replica Location Service, Replica Metadata
Service, Replica Optimization Service, Replica Subscription and high-level
replica management. The service security infrastructure is fully GSI-enabled,
hence compatible with the existing Globus Toolkit 2-based services; moreover,
it allows for fine-grained authorization mechanisms that can be adjusted
depending on the service semantics.Comment: Talk from the 2003 Computing in High Energy and Nuclear Physics
(CHEP03), La Jolla,Ca, USA, March 2003 8 pages, LaTeX, the file contains all
LaTeX sources - figures are in the directory "figures
Global Grids and Software Toolkits: A Study of Four Grid Middleware Technologies
Grid is an infrastructure that involves the integrated and collaborative use
of computers, networks, databases and scientific instruments owned and managed
by multiple organizations. Grid applications often involve large amounts of
data and/or computing resources that require secure resource sharing across
organizational boundaries. This makes Grid application management and
deployment a complex undertaking. Grid middlewares provide users with seamless
computing ability and uniform access to resources in the heterogeneous Grid
environment. Several software toolkits and systems have been developed, most of
which are results of academic research projects, all over the world. This
chapter will focus on four of these middlewares--UNICORE, Globus, Legion and
Gridbus. It also presents our implementation of a resource broker for UNICORE
as this functionality was not supported in it. A comparison of these systems on
the basis of the architecture, implementation model and several other features
is included.Comment: 19 pages, 10 figure
- …