14,751 research outputs found
A job response time prediction method for production Grid computing environments
A major obstacle to the widespread adoption of Grid Computing in both the scientific
community and industry sector is the difficulty of knowing in advance a job submission running
cost that can be used to plan a correct allocation of resources.
Traditional distributed computing solutions take advantage of homogeneous and open
environments to propose prediction methods that use a detailed analysis of the hardware and
software components. However, production Grid computing environments, which are large and
use a complex and dynamic set of resources, present a different challenge. In Grid computing
the source code of applications, programme libraries, and third-party software are not always
available. In addition, Grid security policies may not agree to run hardware or software analysis
tools to generate Grid components models.
The objective of this research is the prediction of a job response time in production Grid
computing environments. The solution is inspired by the concept of predicting future Grid
behaviours based on previous experiences learned from heterogeneous Grid workload trace
data. The research objective was selected with the aim of improving the Grid resource usability
and the administration of Grid environments. The predicted data can be used to allocate
resources in advance and inform forecasted finishing time and running costs before submission.
The proposed Grid Computing Response Time Prediction (GRTP) method implements
several internal stages where the workload traces are mined to produce a response time
prediction for a given job. In addition, the GRTP method assesses the predicted result against
the actual target jobâs response time to inference information that is used to tune the methods
setting parameters.
The GRTP method was implemented and tested using a cross-validation technique to assess
how the proposed solution generalises to independent data sets. The training set was taken from
the Grid environment DAS (Distributed ASCI Supercomputer). The two testing sets were taken
from AuverGrid and Grid5000 Grid environments
Three consecutive tests assuming stable jobs, unstable jobs, and using a job type method to
select the most appropriate prediction function were carried out. The tests offered a significant
increase in prediction performance for data mining based methods applied in Grid computing
environments. For instance, in Grid5000 the GRTP method answered 77 percent of job
prediction requests with an error of less than 10 percent. While in the same environment, the most effective and accurate method using workload traces was only able to predict 32 percent of
the cases within the same range of error.
The GRTP method was able to handle unexpected changes in resources and services which
affect the job response time trends and was able to adapt to new scenarios. The tests showed
that the proposed GRTP method is capable of predicting job response time requests and it also
improves the prediction quality when compared to other current solutions
A Mediated Definite Delegation Model allowing for Certified Grid Job Submission
Grid computing infrastructures need to provide traceability and accounting of
their users" activity and protection against misuse and privilege escalation. A
central aspect of multi-user Grid job environments is the necessary delegation
of privileges in the course of a job submission. With respect to these generic
requirements this document describes an improved handling of multi-user Grid
jobs in the ALICE ("A Large Ion Collider Experiment") Grid Services. A security
analysis of the ALICE Grid job model is presented with derived security
objectives, followed by a discussion of existing approaches of unrestricted
delegation based on X.509 proxy certificates and the Grid middleware gLExec.
Unrestricted delegation has severe security consequences and limitations, most
importantly allowing for identity theft and forgery of delegated assignments.
These limitations are discussed and formulated, both in general and with
respect to an adoption in line with multi-user Grid jobs. Based on the
architecture of the ALICE Grid Services, a new general model of mediated
definite delegation is developed and formulated, allowing a broker to assign
context-sensitive user privileges to agents. The model provides strong
accountability and long- term traceability. A prototype implementation allowing
for certified Grid jobs is presented including a potential interaction with
gLExec. The achieved improvements regarding system security, malicious job
exploitation, identity protection, and accountability are emphasized, followed
by a discussion of non- repudiation in the face of malicious Grid jobs
Towards Grid Interoperability
The Grid paradigm promises to provide global access to computing resources, data storage and experimental instruments. It also provides an elegant solution to many resource administration and provisioning problems while offering a platform for collaboration and resource sharing. Although substantial progress has been made towards these goals, nevertheless there is still a lot of work to be done until the Grid can deliver its promises. One of the central issues is the development of standards and Grid interoperability. Job execution is one of the key capabilities in all Grid environments. This is a well understood, mature area with standards and implementations. This paper describes some proof of concept experiments demonstrating the interoperability between various Grid environments
Condor services for the Global Grid:interoperability between Condor and OGSA
In order for existing grid middleware to remain viable it is important to investigate their potentialfor integration with emerging grid standards and architectural schemes. The Open Grid ServicesArchitecture (OGSA), developed by the Globus Alliance and based on standard XML-based webservices technology, was the first attempt to identify the architectural components required tomigrate towards standardized global grid service delivery. This paper presents an investigation intothe integration of Condor, a widely adopted and sophisticated high-throughput computing softwarepackage, and OGSA; with the aim of bringing Condor in line with advances in Grid computing andprovide the Grid community with a mature suite of high-throughput computing job and resourcemanagement services. This report identifies mappings between elements of the OGSA and Condorinfrastructures, potential areas of conflict, and defines a set of complementary architectural optionsby which individual Condor services can be exposed as OGSA Grid services, in order to achieve aseamless integration of Condor resources in a standardized grid environment
Large Scale In Silico Screening on Grid Infrastructures
Large-scale grid infrastructures for in silico drug discovery open
opportunities of particular interest to neglected and emerging diseases. In
2005 and 2006, we have been able to deploy large scale in silico docking within
the framework of the WISDOM initiative against Malaria and Avian Flu requiring
about 105 years of CPU on the EGEE, Auvergrid and TWGrid infrastructures. These
achievements demonstrated the relevance of large-scale grid infrastructures for
the virtual screening by molecular docking. This also allowed evaluating the
performances of the grid infrastructures and to identify specific issues raised
by large-scale deployment.Comment: 14 pages, 2 figures, 2 tables, The Third International Life Science
Grid Workshop, LSGrid 2006, Yokohama, Japan, 13-14 october 2006, to appear in
the proceeding
Distributed Computing Grid Experiences in CMS
The CMS experiment is currently developing a computing system capable of serving, processing and archiving the large number of events that will be generated when the CMS detector starts taking data. During 2004 CMS undertook a large scale data challenge to demonstrate the ability of the CMS computing system to cope with a sustained data-taking rate equivalent to 25% of startup rate. Its goals were: to run CMS event reconstruction at CERN for a sustained period at 25 Hz input rate; to distribute the data to several regional centers; and enable data access at those centers for analysis. Grid middleware was utilized to help complete all aspects of the challenge. To continue to provide scalable access from anywhere in the world to the data, CMS is developing a layer of software that uses Grid tools to gain access to data and resources, and that aims to provide physicists with a user friendly interface for submitting their analysis jobs. This paper describes the data challenge experience with Grid infrastructure and the current development of the CMS analysis system
Global Grids and Software Toolkits: A Study of Four Grid Middleware Technologies
Grid is an infrastructure that involves the integrated and collaborative use
of computers, networks, databases and scientific instruments owned and managed
by multiple organizations. Grid applications often involve large amounts of
data and/or computing resources that require secure resource sharing across
organizational boundaries. This makes Grid application management and
deployment a complex undertaking. Grid middlewares provide users with seamless
computing ability and uniform access to resources in the heterogeneous Grid
environment. Several software toolkits and systems have been developed, most of
which are results of academic research projects, all over the world. This
chapter will focus on four of these middlewares--UNICORE, Globus, Legion and
Gridbus. It also presents our implementation of a resource broker for UNICORE
as this functionality was not supported in it. A comparison of these systems on
the basis of the architecture, implementation model and several other features
is included.Comment: 19 pages, 10 figure
Grid service orchestration using the Business Process Execution Language (BPEL)
Modern scientific applications often need to be distributed across grids. Increasingly
applications rely on services, such as job submission, data transfer or data
portal services. We refer to such services as grid services. While the invocation
of grid services could be hard coded in theory, scientific users want to orchestrate
service invocations more flexibly. In enterprise applications, the orchestration of
web services is achieved using emerging orchestration standards, most notably
the Business Process Execution Language (BPEL). We describe our experience
in orchestrating scientific workflows using BPEL. We have gained this experience
during an extensive case study that orchestrates grid services for the automation of
a polymorph prediction application
- âŠ