A major obstacle to the widespread adoption of Grid Computing in both the scientific
community and industry sector is the difficulty of knowing in advance a job submission running
cost that can be used to plan a correct allocation of resources.
Traditional distributed computing solutions take advantage of homogeneous and open
environments to propose prediction methods that use a detailed analysis of the hardware and
software components. However, production Grid computing environments, which are large and
use a complex and dynamic set of resources, present a different challenge. In Grid computing
the source code of applications, programme libraries, and third-party software are not always
available. In addition, Grid security policies may not agree to run hardware or software analysis
tools to generate Grid components models.
The objective of this research is the prediction of a job response time in production Grid
computing environments. The solution is inspired by the concept of predicting future Grid
behaviours based on previous experiences learned from heterogeneous Grid workload trace
data. The research objective was selected with the aim of improving the Grid resource usability
and the administration of Grid environments. The predicted data can be used to allocate
resources in advance and inform forecasted finishing time and running costs before submission.
The proposed Grid Computing Response Time Prediction (GRTP) method implements
several internal stages where the workload traces are mined to produce a response time
prediction for a given job. In addition, the GRTP method assesses the predicted result against
the actual target job’s response time to inference information that is used to tune the methods
setting parameters.
The GRTP method was implemented and tested using a cross-validation technique to assess
how the proposed solution generalises to independent data sets. The training set was taken from
the Grid environment DAS (Distributed ASCI Supercomputer). The two testing sets were taken
from AuverGrid and Grid5000 Grid environments
Three consecutive tests assuming stable jobs, unstable jobs, and using a job type method to
select the most appropriate prediction function were carried out. The tests offered a significant
increase in prediction performance for data mining based methods applied in Grid computing
environments. For instance, in Grid5000 the GRTP method answered 77 percent of job
prediction requests with an error of less than 10 percent. While in the same environment, the most effective and accurate method using workload traces was only able to predict 32 percent of
the cases within the same range of error.
The GRTP method was able to handle unexpected changes in resources and services which
affect the job response time trends and was able to adapt to new scenarios. The tests showed
that the proposed GRTP method is capable of predicting job response time requests and it also
improves the prediction quality when compared to other current solutions
