A Genetic Algorithm to Schedule Workflow Collections on a SOA-Grid with Communication Costs

International audienceIn this paper we study the problem of scheduling a collection of workflows, identical or not, on a SOA grid. A workflow (job) is represented by a directed acyclic graph (DAG) with typed tasks. All of the grid hosts are able to process a set of task types with unrelated processing costs and are able to transmit files through communication links for which the communication times are not negligible. The goal is to minimize the maximum completion time (makespan) of the workflows. To solve this problem we propose a genetic approach. The contributions of this paper are both the design of a Genetic Algorithm taking the communication costs into account and the performance analysis

Comparison on OpenStack and OpenNebula performance to improve multi-Cloud architecture on cosmological simulation use case

With the increasing numbers of Cloud Service Providers and the migration of the Grids to the Cloud paradigm, it is necessary to be able to leverage these new resources. Moreover, a large class of High Performance Computing (HPC) applications can run these resources without (or with minor) modifications. But using these resources come with the cost of being able to interact with these new resource providers. In this paper we introduce the design of a HPC middleware that is able to use resources coming from an environment that compose of multiple Clouds as well as classical \hpc resources. Using the \diet middleware, we are able to deploy a large-scale, distributed HPC platform that spans across a large pool of resources aggregated from different providers. Furthermore, we hide to the end users the difficulty and complexity of selecting and using these new resources even when new Cloud Service Providers are added to the pool. Finally, we validate the architecture concept through cosmological simulation RAMSES. Thus we give a comparison of 2 well-known Cloud Computing Software: OpenStack and OpenNebula.Avec l'augmentation du nombre de fournisseurs de service Cloud et la migration des applications depuis les grilles de calcul vers le Cloud, il est nécessaire de pouvoir tirer parti de ces nouvelles ressources. De plus, une large classe des applications de calcul haute performance peuvent s'exécuter sur ces ressources sans modifications (ou avec des modifications mineures). Mais utiliser ces ressources vient avec le coût d'être capable d'intéragir avec des nouveaux fournisseurs de ressources. Dans ce papier, nous introduisons la conception d'un nouveau intergiciel HPC qui permet d'utiliser les ressources qui proviennent d'un environement composé de plusieurs Clouds comme des ressources classiques. En utilisant l'intergiciel \diet, nous sommes capable de déployer une plateforme HPC distribuée et large échelle qui s'étend sur un large ensemble de ressources aggrégées entre plusieurs fournisseurs Cloud. De plus, nous cachons à l'utilisateur final la difficulté et la complexité de sélectionner et d'utiliser ces nouvelles ressources quand un nouveau fournisseur de service Cloud est ajouté dans l'ensemble. Finalement, nous validons notre concept d'architecture via une application de simulation cosmologique RAMSES. Et nous fournissons une comparaison entre 2 intergiciels de Cloud: OpenStack et OpenNebula

A Genetic Algorithm with Communication Costs to Schedule Workflows on a SOA-Grid

International audienceIn this paper we study the problem of scheduling a collection of workflows, identical or not, on a SOA (Service Oriented Architecture) grid . A workflow (job) is represented by a directed acyclic graph (DAG) with typed tasks. All of the grid hosts are able to process a set of typed tasks with unrelated processing costs and are able to transmit files through communication links for which the communication times are not negligible. The goal of our study is to minimize the maximum completion time (makespan) of the workflows. To solve this problem we propose a genetic approach. The contributions of this paper are both the design of a Genetic Algorithm taking the communication costs into account and its performance analysis

Using Virtualization and Job Folding for Batch Scheduling

International audienceIn this paper we study the problem of batch scheduling within a homogeneous cluster. In this context, the problem is that the more processors the job requires the more difficult it is to find an idle slot to run it on. As a consequence the resources are often inefficiently used as some of them remain unallocated in the final schedule. To address this issue we propose a technique called job folding that uses virtualization to reduce the number of processors allocated to a parallel job and thus allows to execute it earlier. Our goal is to optimize the resource use. In this paper we propose several heuristics based on job folding and we compare their performance with classical on-line scheduling algorithms as FCFS or backfilling. The contributions of the paper are both on the design of the job folding algorithms and on their performance analysis

Assessing new approaches to schedule a batch of identical intree-shaped workflows on a heterogeneous platform

International audienceIn this paper, we consider the makespan optimisation when scheduling a batch of identical workflows on a heterogeneous platform as a service-oriented grid or a micro-factory. A job is represented by a directed acyclic graph (DAG) with typed tasks and no fork nodes (in-tree precedence constraints). The processing resources are able to process a set of task types, each with unrelated processing cost. The objective function is to minimise the execution makespan of a batch of identical workflows while most of the works concentrate on the throughput in this case. Three algorithms are studied in this context: a classical list algorithm and two algorithms based on new approaches, a genetic algorithm and a steady-state algorithm. The contribution of this paper is both on the adaptation of these algorithms to the particular case of batches of identical workflows and on the performance analysis of these algorithms regarding the makespan. We show the benefits of their adaptation, and we show that the algorithm performance depends on the structure of the workflow, on the size of the batch and on the platform characteristics

Using a Sparse Promoting Method in Linear Programming Approximations to Schedule Parallel Jobs.

International audienceIn this paper, we tackle the well‐known problem of scheduling a collection of parallel jobs on a set of processors either in a cluster or in a multiprocessor computer. For the makespan objective, that is, the completion time of the last job, this problem has been shown to be NP‐hard, and several heuristics have already been proposed to minimize the execution time. In this paper, we consider both rigid and moldable jobs. Our main contribution is the introduction of a new approach to the scheduling problem, based on the recent discoveries in the field of compressed sensing. In the proposed approach, all possible positions and shapes of the jobs are encoded into a matrix, and the scheduling is performed by selecting the best columns under natural constraints. Thus, the solution to the new scheduling formulation is naturally sparse, and we may use appropriate relaxations to achieve the optimization task in the quickest possible way. Among many possible relaxation strategies, we choose to minimize the <i&gtℓ</i&gt<sub&gt<i&gtp</i&gt</sub&gt‐quasi‐norm for <i&gtp</i&gt∈(0,1). Minimization of the <i&gtℓ</i&gt<sub&gt<i&gtp</i&gt</sub&gt‐quasi‐norm is implemented via a successive linear programming approximation heuristic. We propose several new algorithms based on this approach, and we assess their efficiency through simulations. The experiments show that the scheme outperforms the classic Largest Task First list based algorithm for scheduling small to medium instances but needs improvements to compete on larger numbers of jobs.&nbsp

Seeding the Cloud: An Innovative Approach to Grow Trust in Cloud Based Infrastructures

International audienceComplying with security and privacy requirements of appliances such as mobile handsets, personal computers, servers for customers, enterprises and governments is mandatory to prevent from theft of sensitive data and to preserve their integrity. Nowadays, with the rising of the Cloud Computing approach in business fields, security and privacy are even more critical. The aim of this article is then to propose a way to build a secure and trustable Cloud. The idea is to spread and embed Secure Elements (SE) on each level of the Cloud in order to make a wide trusted infrastructure which complies with access control and isolation policies. This article presents therefore this new approach of trusted Cloud infrastructure based on a Network of Secure Elements (NoSE), and it illustrates this approach through different use cases