Parallelization of Littlewood-Richardson Coefficients Computation and its Integration into the BonjourGrid Meta-Desktop Grid Middleware

International audienceThe aim of this paper is to show how to parallelize a compute intensive application in mathematics (Group Theory) for an institutional Desktop Grid platform coordinated by a meta-grid middleware named BonjourGrid. The paper is twofold: first of all, it shows how to parallelize a sequential program for a multicore CPU which participates in the computation and second it demonstrates the effort for launching multiple instances of the solutions for the mathematical problem with the BonjourGrid middleware. BonjourGrid is a fully decentralized Desktop Grid middleware. The main results of the paper are: a) an efficient multi-threaded version of a sequential program to compute Littlewood- Richardson coefficients, namely the Multi-LR program and b) a proof of concept, centered around the user needs, for the BonjourGrid middleware dedicated to coordinate multiple instances of programsfor Desktop Grids and with the help of Multi-LR. In this paper, the scientific work consists in starting from a model for the solution of a compute intensive problem in mathematics, to incorporate the concrete model into a middleware and running it on commodity PCs platform managed by an innovative meta Desktop Grid middleware

PARALLEL COMPUTING WITH P2P DESKTOP GRIDS

Tightly-coupled parallel computing is an important tool for problem solving. Structured peer-to-peer network overlays are failure-tolerant and have a low admin- istrative burden. This work seeks to unite the two. First, I present a completely decentralized algorithm for parallel job scheduling and load balancing in distributed peer-to-peer environments. This algorithm is useful for meta-scheduling across known clusters and scheduling on desktop grids. To accomplish this, I build on previous work to route jobs to appropriate resources then use the new algorithm to start parallel jobs and balance load across the grid. I also discuss what constitutes useful clusterings for this algorithm as well as inherent scaling limitations. Ultimately, I show that my algorithm performs comparably to one using centralized load balancing with global up-to-date information. The principal contribution of this work is that the parallel job scheduling is completely decentralized, which is not featured in previous work, and enables reliable ad hoc sharing of distributed resources to run parallel computations. Second, I show how clusters of computers can be found dynamically by using an existing latency prediction technique coupled with a new refinement algorithm. Several latency prediction techniques are compared experimentally. One, based on a tree metric space embedding, is found to be superior to the others. Nevertheless, I show that it is not quite accurate enough. To solve this problem, I present a refinement algorithm for producing quality clusters while still maintaining bounds for the amount of information any given node must store about other nodes. I show that clusters derived this way have scheduler performance comparable to those chosen statically with global knowledge. Lastly, I discuss previously undiscovered under-specifications in the Content Addressable Network (CAN) structured peer to peer system. In high-churn situ- ations, the CAN allows stale information and changes to the overlay structure to create routing problems. I show solutions to these two problems, as well as discuss other issues that may also disrupt a CAN

BonjourGrid as a Decentralised Job Scheduler

International audienceInstitutional desktop grid systems are attractive for running distributed applications with significant computational requirements. While the rapid increasing number of users and applications running on such systems does demonstrate the potential of desktop grid and institutional desktop grid, current implementations follow the old-fashioned master-worker paradigm. Obviously, vulnerability to failures and permanent administrative monitoring are the disadvantages of client-server architectures. To bypass this, we proposed a novel system, called BonjourGrid, able to orchestrate multiple instances of institutional desktop grid middlewares, able to remove the risk of single-source bottleneck and failure, and able to guaranty the continuity of services in a distributed manner. In this paper, we use BonjourGrid protocol, which is based on publish/subscribe paradigm, to show how to adapt it to fulfill all the requirements of a decentralised job scheduler. An evaluation proves that BonjourGrid is able to manage more than 100 applications instanciated in a concurrent way on an institutional desktop grid. Analysing the execution of 100 applications with 2110 tasks during 3 hours demonstrates the potential of BonjourGrid concept and shows that, comparing to a classical desktop grid with one central master, Bonjourgrid gives an acceptable overhead that can be explained