A hyper-heuristic for adaptive scheduling in computational grids

In this paper we present the design and implementation of an hyper-heuristic for efficiently scheduling independent jobs in computational grids. An efficient scheduling of jobs to grid resources depends on many parameters, among others, the characteristics of the resources and jobs (such as computing capacity, consistency of computing, workload, etc.). Moreover, these characteristics change over time due to the dynamic nature of grid environment, therefore the planning of jobs to resources should be adaptively done. Existing ad hoc scheduling methods (batch and immediate mode) have shown their efficacy for certain types of resource and job characteristics. However, as stand alone methods, they are not able to produce the best planning of jobs to resources for different types of Grid resources and job characteristics. In this work we have designed and implemented a hyper-heuristic that uses a set of ad hoc (immediate and batch mode) scheduling methods to provide the scheduling of jobs to Grid resources according to the Grid and job characteristics. The hyper-heuristic is a high level algorithm, which examines the state and characteristics of the Grid system (jobs and resources), and selects and applies the ad hoc method that yields the best planning of jobs. The resulting hyper-heuristic based scheduler can be thus used to develop network-aware applications that need efficient planning of jobs to resources. The hyper-heuristic has been tested and evaluated in a dynamic setting through a prototype of a Grid simulator. The experimental evaluation showed the usefulness of the hyper-heuristic for planning of jobs to resources as compared to planning without knowledge of the resource and job characteristics.Peer ReviewedPostprint (author's final draft

Global Grids and Software Toolkits: A Study of Four Grid Middleware Technologies

Grid is an infrastructure that involves the integrated and collaborative use of computers, networks, databases and scientific instruments owned and managed by multiple organizations. Grid applications often involve large amounts of data and/or computing resources that require secure resource sharing across organizational boundaries. This makes Grid application management and deployment a complex undertaking. Grid middlewares provide users with seamless computing ability and uniform access to resources in the heterogeneous Grid environment. Several software toolkits and systems have been developed, most of which are results of academic research projects, all over the world. This chapter will focus on four of these middlewares--UNICORE, Globus, Legion and Gridbus. It also presents our implementation of a resource broker for UNICORE as this functionality was not supported in it. A comparison of these systems on the basis of the architecture, implementation model and several other features is included.Comment: 19 pages, 10 figure

A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

A Stackelberg game for modelling asymmetric users' behavior in grid scheduling

In traditional distributed computing the users and owners of the computational resources usually belong to the same administrative domain. Therefore all users are equally entitled to use the resources. The situation is completely different in large-scale emergent distributed computing systems, such as Grid systems, where the roles of the users are asymmetric as regards their access rights and usage of resources. Further, unlike traditional distributed computing case, Grid systems introduce hierarchical levels, which are to be taken into account for optimizing the overall system's performance. In this paper we present a Stackelberg game for modelling asymmetric users' behavior in Grid scheduling scenario. We define a two-level game with a Leader at the first level and the rest of users, called Followers, at the second one. The Leader is responsible for computing a planning of his tasks, which is usually a large fraction of the total pool of tasks in the batch. The Followers try to select the best strategy for the assignments of their tasks subject to Leader's strategy. The Stackelberg game is then translated into a hierarchical optimization problem, which is solved by Genetic Algorithm (GA) on the Leader's level and by ad hoc heuristic combined with GA on the Followers' level. We have experimentally evaluated the approach through a benchmark of static instances and report computational results for resource utilization, makespan and flowtimePeer ReviewedPostprint (published version

Scheduling of Dependent Tasks Application using Random Search Technique

Since beginning of Grid computing, scheduling of dependent tasks application has attracted attention of researchers due to NP-Complete nature of the problem. In Grid environment, scheduling is deciding about assignment of tasks to available resources. Scheduling in Grid is challenging when the tasks have dependencies and resources are heterogeneous. The main objective in scheduling of dependent tasks is minimizing make-span. Due to NP-complete nature of scheduling problem, exact solutions cannot generate schedule efficiently. Therefore, researchers apply heuristic or random search techniques to get optimal or near to optimal solution of such problems. In this paper, we show how Genetic Algorithm can be used to solve dependent task scheduling problem. We describe how initial population can be generated using random assignment and height based approaches. We also present design of crossover and mutation operators to enable scheduling of dependent tasks application without violating dependency constraints. For implementation of GA based scheduling, we explore and analyze SimGrid and GridSim simulation toolkits. From results, we found that SimGrid is suitable, as it has support of SimDag API for DAG applications. We found that GA based approach can generate schedule for dependent tasks application in reasonable time while trying to minimize make-span