Chtepen, Maria

English

Ghent University Academic Bibliography

Adaptive Scheduling in GridsMaria ChtepenSupervisor(s): Bart DhoedtI. INTRODUCTIONGrid computing is a technology that has sig-nificantly gained importance in recent years, re-placing the stand-alone cluster approach. Theidea behind grid is to combine the capacity ofresources belonging to different organizationsto execute strongly computationally or storageintensive applications. Think of applicationsin the fields of high energy physics, financialsimulation and climate modeling, which canrun for weeks on an average size local clus-ter. Probably the most well-known example ofa grid application is the processing of data orig-inating from the Large Hadron Collider (LHC).LHC is the world’s largest particle accelera-tor, within which protons are moving aroundat a speed close to the speed of light, produc-ing 600 million collisions per second, result-ing in 700 megabytes of data each second. Toprocess this huge amount of data, the LHC grid[1] infrastructure was put together that con-nects over 140 scientific centers in 33 countriesaround the globe. However, using grid com-puting is not only advantageous with regard tocomputational speed-ups, but also from an eco-nomic point of view. Instead of buying largeand expensive infrastructure that is able to cap-ture peak loads, organizations can now shareresources, which obviously leads to lower hard-ware investments.Management of complex grid environmentsraises several important issues, such as secu-rity, efficient distributed data management andadvanced scheduling. The focus of our re-search lies on the latter topic. We take into ac-count grid-specific properties, such as varyingresource load and availability, complex applica-tion structure and a large number of competingusers, to optimize the assignment of grid appli-cations (jobs) to the available resources.II. ADAPTIVE SCHEDULINGFigure 1. An example of a grid infrastructure: WAN(Wide Area Network), LAN (Local Area Net-work), CR (Computational Resource), SR (Stor-age Resource), Sched (Grid Scheduler).When scheduling jobs within clusters, oneoften relies on static approaches: the systemstatus is analyzed at a particular point in time,and based on this information the final schedul-ing decision is taken. While well-suited forrelatively stable cluster environments, this ap-proach leads to significant drawbacks withingrids due to the following grid properties:• Grids are highly dynamic: strong load varia-tions on resources as well as network, and re-source failures are not uncommon. Therefore,there is a need of flexible fault-tolerance andmigration mechanisms.• Users from various organizations are compet-ing for the same resource pool: an efficient,priority-based scheduler should be provided.• Complex applications are often run on grids:a grid scheduler should be able to deal withincomplete information on job parameters andwith job dependencies.Adaptive scheduling algorithms developedin scope of this PhD take into account theabove-mentioned grid properties while decid-ing upon the best job-resource assignment.Adaptive means that the scheduling policy aswell as job schedules can be modified basedon information on grid status and jobs progresscollected at run-time . In the next subsectionsa short discussion of the proposed methods andtheir performance evaluation are given.A. Dynamic gridsTo deal with unpredictable resource fail-ures and load variations, two fault-tolerant ap-proaches were considered: job replication andjob checkpointing. Since both approaches canlead to significant system overhead, algorithmswere developed and implemented that modifythe number of job replicas and the frequencyof checkpointing depending on resource load,failure frequency, and remaining job executiontimes. Figure 2 shows that the adaptive ap-proaches significantly reduce the number ofjobs lost due to failures, compared to uncon-ditional replication approaches. At the sametime they considerably reduce system overheadby omitting redundant replicas / checkpoints,which leads to higher grid throughput [2].Figure 2. Number of jobs lost by uncon-ditional replication algorithms (Uncondition-alRL), adaptive replication algorithms (Load-DependentRL, FailureDependentRL) and adap-tive checkpointing algorithms (MeanFailureCP,CombinedFT).B. Competing usersIn grids it is often desirable that particu-lar jobs receive higher priority. Therefore, wedeveloped an algorithm that differentiates be-tween jobs based on their computational load.The algorithm defines an adaptive load bound-ary, which guarantees that as many priori-tized jobs (jobs with load less than a prede-fined value) as possible get executed. Non-prioritized jobs are processed only if this doesnot penalize the execution of prioritized jobs.C. Complex applicationsMany existing scheduling algorithms forgrids depart from the assumption that job ex-ecution times are known in advance. Unfor-tunately, for most real-world applications thisparameter is extremely hard to determine a pri-ori. In our research, we considered applicationscontaining interdependent tasks for which ex-ecution times can be estimated based on peri-odic updates on task progress. We developeda scheduling approach that reassigns tasks dy-namically, taking into account the estimates, insuch a way that application execution times arereduced at the cost of eventually slowing downexecution of incorporated tasks. Simulation re-sults have shown that our approach achieves animprovement of 60% on average [3].III. CONCLUSIONSGrid computing is a new technology, whichstill features several unsolved issues. One ofthese issues is efficient job scheduling, whichtakes into account grid-specific properties, suchas the dynamic nature of grid resources, largenumbers of competing users and the complex-ity of grid jobs. Our research addresses theseissues, introducing adaptive scheduling solu-tions.REFERENCES[1] Worldwide LHC Computing Grid Websitehttp://lcg.web.cern.ch/LCG/, CERN.[2] Maria Chtepen et al., Adaptive Task Check-pointing and Replication: Toward Efficient Fault-Tilerant Grids, IEEE Transactions on Parallel andDistributed Systems, 20(2): 180-190, February2009.[3] Maria Chtepen et al., Scheduling of DependentGrid Jobs in Absence of Exact Job Length Infor-mation In Proceedings of the 4th IEEE/IFIP Inter-national Workshop on End-to-end Virtualizationand Grid Management, Samos Island, Greece, 22- 26 September 2008.

Adaptive scheduling in grids

https://biblio.ugent.be/publication/890970/file/920504.pdf

Adaptive scheduling in grids

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography