How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications

The grid computing vision promises to provide the needed platform for a new and more demanding range of applications. For this promise to become true, a number of hurdles, including the design and deployment of adequate resource management and information services, need to be overcome. In this context, understanding the characteristics of real grid workloads is a crucial step for improving the quality of existing grid services, and in guiding the design of new solutions. Towards this goal, in this work we present the characteristics of traces of four real grid environments, namely LCG, Grid3, and TeraGrid, which are among the largest production grids currently deployed, and the DAS, which is a research grid. We focus our analysis on virtual organizations, on users, and on individual jobs characteristics. We further attempt to quantify the evolution and the performance of the grid systems from which our traces originate. Finally, given the scarcity of the information available for analysis purposes, we discuss the requirements of a new format for grid traces, and we propose the establishment of a virtual center for workload-based grid benchmarking data: the grid workloads archive

A year in the life of a large scale experimental distributed system: the Grid'5000 platform in 2008

This report presents the usage results of Grid'5000 over year 2008. Usage of the main operationnal Grid'5000 sites (Bordeaux, Lille, Lyon, Nancy, Orsay, Rennes, Sophia-Antipolis, Toulouse) is presented and analyzed

Towards ServMark, an Architecture for Testing Grid Services

Technical University of Delft - Technical Report ServMark-2006-002, July 2006Grid computing provides a natural way to aggregate resources from different administrative domains for building large scale distributed environments. The Web Services paradigm proposes a way by which virtual services can be seamlessly integrated into global-scale solutions to complex problems. While the usage of Grid technology ranges from academia and research to business world and production, two issues must be considered: that the promised functionality can be accurately quantified and that the performance can be evaluated based on well defined means. Without adequate functionality demonstrators, systems cannot be tuned or adequately configured, and Web services cannot be stressed adequately in production environment. Without performance evaluation systems, the system design and procurement processes are limp, and the performance of Web Services in production cannot be assessed. In this paper, we present ServMark, a carefully researched tool for Grid performance evaluation. While we acknowledge that a lot of ground must be covered to fulfill the requirements of a system for testing Grid environments, and Web (and Grid) Services, we believe that ServMark addresses the minimal set of critical issues

Workflow task clustering for best effort systems with

ABSTRACT Many scientific workflows are composed of fine computational granularity tasks, yet they are composed of thousands of them and are data intensive in nature, thus requiring resources such as the TeraGrid to execute efficiently. In order to improve the performance of such applications, we often employ task clustering techniques to increase the computational granularity of workflow tasks. The goal is to minimize the completion time of the workflow by reducing the impact of queue wait times. In this paper, we examine the performance impact of the clustering techniques using the Pegasus workflow management system. Experiments performed using an astronomy workflow on the NCSA TeraGrid cluster show that clustering can achieve a significant reduction in the workflow completion time (upto 97%)

How are real grids used? The analysis of four grid traces and its implications

The grid computing vision promises to provide the needed platform for a new and more demanding range of applications. For this promise to become true, a number of hurdles, including the design and deployment of adequate resource management and information services, need to be overcome. In this context, understanding the characteristics of real grid workloads is a crucial step for improving the quality of existing grid services, and in guiding the design of new solutions. Towards this goal, in this work we present the characteristics of traces of four real grid environments, namely LCG, Grid3, and TeraGrid, which are among the largest production grids currently deployed, and the DAS, which is a research grid. We focus our analysis on virtual organizations, on users, and on individual jobs characteristics. We further attempt to quantify the evolution and the performance of the grid systems from which our traces originate. Finally, given the scarcity of the information available for analysis purposes, we discuss the requirements of a new format for grid traces, and we propose the establishment of a virtual center for workload-based grid benchmarking data: the grid workloads archive