50 research outputs found

    Optimal Replica Placement in Tree Networks with QoS and Bandwidth Constraints and the Closest Allocation Policy

    Get PDF
    This paper deals with the replica placement problem on fully homogeneous tree networks known as the Replica Placement optimization problem. The client requests are known beforehand, while the number and location of the servers are to be determined. We investigate the latter problem using the Closest access policy when adding QoS and bandwidth constraints. We propose an optimal algorithm in two passes using dynamic programming

    Multi-criteria scheduling of pipeline workflows

    Get PDF
    Mapping workflow applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline graphs. Several antagonist criteria should be optimized, such as throughput and latency (or a combination). In this paper, we study the complexity of the bi-criteria mapping problem for pipeline graphs on communication homogeneous platforms. In particular, we assess the complexity of the well-known chains-to-chains problem for different-speed processors, which turns out to be NP-hard. We provide several efficient polynomial bi-criteria heuristics, and their relative performance is evaluated through extensive simulations

    Multi-kritÀres Mapping und Scheduling von Workflow-Anwendungen auf heterogenen Plattformen

    Get PDF
    The results summarized in this thesis deal with the mapping and scheduling of workflow applications on heterogeneous platforms. In this context, we focus on three different types of streaming applications: * Replica placement in tree networks * In this kind of application, clients are issuing requests to some servers and the question is where to place replicas in the network such that all requests can be processed. We discuss and compare several policies to place replicas in tree networks, subject to server capacity, Quality of Service (QoS) and bandwidth constraints. The client requests are known beforehand, while the number and location of the servers have to be determined. The standard approach in the literature is to enforce that all requests of a client be served by the closest server in the tree. We introduce and study two new policies. One major contribution of this work is to assess the impact of these new policies on the total replication cost. Another important goal is to assess the impact of server heterogeneity, both from a theoretical and a practical perspective. We establish several new complexity results, and provide several efficient polynomial heuristics for NP-complete instances of the problem. * Pipeline workflow applications * We consider workflow applications that can be expressed as linear pipeline graphs. An example for this application type is digital image processing, where images are treated in steady-state mode. Several antagonist criteria should be optimized, such as throughput and latency (or a combination) as well as latency and reliability (i.e., the probability that the computation will be successful) of the application. While simple polynomial algorithms can be found for fully homogeneous platforms, the problem becomes NP-hard when tackling heterogeneous platforms. We present an integer linear programming formulation for this latter problem. Furthermore, we provide several efficient polynomial bi-criteria heuristics, whose relative performances are evaluated through extensive simulation. As a case-study, we provide simulations and MPI experimental results for the JPEG encoder application pipeline on a cluster of workstations. * Complex streaming applications * We consider the execution of applications structured as trees of operators, i.e., the application of one or several trees of operators in steady-state to multiple data objects that are continuously updated at various locations in a network. A first goal is to provide the user with a set of processors that should be bought or rented in order to ensure that the application achieves a minimum steady-state throughput, and with the objective of minimizing platform cost. We then extend our model to multiple applications: several concurrent applications are executed at the same time in a network, and one has to ensure that all applications can reach their application throughput. Another contribution of this work is to provide complexity results for different instances of the basic problem, as well as integer linear program formulations of various problem instances. The third contribution is the design of several polynomial-time heuristics, for both application models. One of the primary objectives of the heuristics for concurrent applications is to reuse intermediate results shared by multiple applications.In meiner Dissertation beschĂ€ftige ich mich mit dem Scheduling von Workflow-Anwendungen in heterogenen Plattformen. In diesem Zusammenhang konzentriere ich mich auf drei verschiene Anwendungstypen.: * Platzierung von Replikaten in Baumnetzwerken * Dieses erste Schedulingproblem behan-delt die Platzierung von Replikaten in Baumnetzwerken. Ein Beispiel hierfĂŒr ist die Platzierung von Replikaten in verteilten Datenbanksystemen, deren Verbindungsstruktur baumartig organi-siert ist. Die Platzierung soll dabei unter mehreren Constraints (ServerkapazitĂ€ten, sowie DienstgĂŒte und BandbreitenbeschrĂ€nkungen) durchgefĂŒhrt werden. In diesem Anwendungstyp stellen Clients Anfragen an verschiedene Server. Diese Client-Anfragen sind im Voraus bekannt, wĂ€hrend Anzahl und Platzierung der Server erst ermittelt werden mĂŒssen. Die in der Literatur gĂ€ngige Strategie fordert, dass alle Anfragen eines Clients vom nĂ€chstgelegenen Server im Baum behandelt werden. Es werden zwei neue Verfahrensweisen vorgestellt und untersucht. Ein wichtiges Teilergebnis dieser Studie bewertet die Auswirkung der beiden neuen Strategien auf die globalen Replikationskosten. Ausserdem wird der Einfluss von HeterogenitĂ€t aus theore-tischer und praktischer Sicht untersucht. Es werden verschiedene KomplexitĂ€tsergebnisse erar-beitet und mehrere effiziente Polynomialzeit-Heuristiken fĂŒr NP-vollstĂ€ndige Instanzen des Problems vorgestellt. * Lineare Workflow-Anwendungen * Als nĂ€chstes werden Workflow-Anwendungen untersucht, die als lineare Graphen dargestellt werden können. Ein Beispiel dieses Applikationstyps ist die digitale Bildverarbeitung, in der Bilder mittels einer Pipeline verarbeitet werden. Es sollen ver-schieÂŹdene gegensĂ€tzliche Kriterien optimiert werden, wie zum Beispiel Durchsatz und Latenz-zeit, beziehungsweise eine Kombination der beiden, aber auch Latenzzeit und Ausfallsicherheit der Anwendung. WĂ€hrend fĂŒr vollhomogene Plattformen polynomiale Algorithmen gefunden werden können, wird das Problem NP-hart, sobald heterogene Plattformen angestrebt werden. Diese Arbeit beinhaltet eine vollstĂ€ndige KomplexitĂ€tsanalyse. FĂŒr die bisher unbekannten polynomialen Varianten des Problems werden optimale Algorithmen vorgeschlagen. Ein ganz-zahliges lineares Programm fĂŒr das bekannte „chains-on-chains“ Problem fĂŒr heterogene Plattformen wird vorgestellt. Des weiteren werden verschiedene effiziente polynomiale bi-kritĂ€re Heuristiken prĂ€sentiert, deren relative Effizienz durch umfangreiche Simulationen eruiert werden. Eine Fallstudie beschĂ€ftigt sich mit der JPEG-Encoder-Pipeline. Hierbei werden Simulationen und MPI-basierte Auswertungen auf einem Rechen-Cluster erstellt. * Komplexe Streaming-Anwendungen * Als letztes wird die AusfĂŒhrung von Anwendungen, die als Operator-BĂ€ume strukturiert sind, untersucht. Konkret bedeutet dies, dass ein oder mehrere Operator-BĂ€ume in stationĂ€rem Zustand auf mannigfaltige Datenobjekte angewendet werden, welche fortlaufend an verschiedenen Stellen im Netzwerk aktualisiert werden. Ein erstes Ziel ist, dem Benutzer eine Gruppe von Rechnern vorzuschlagen, die gekauft oder gemietet werden sollen, so dass die Anwendung einen minimalen stationĂ€ren Durchsatz erzielt und gleichzeitig Plattformkosten minimiert werden können. Anschließend wird das Modell auf mehrere Anwendungen erweitert: verschiedene nebenlĂ€ufige Anwendungen werden zeitgleich in einem Netzwerk ausgefĂŒhrt und es muss sichergestellt werden, dass alle Anwendungen ihren Durchsatz erreichen können. Beide Modelle werden aus theoretischer Sicht untersucht und eine KomplexitĂ€ts-analyse fĂŒr unterschiedliche Instanzen des Grundproblems, sowie Formulierungen als lineare Programme erstellt. FĂŒr beide Anwendungsmodelle werden verschiedene Polynomialzeit-Heuristiken prĂ€sentiert und charakterisiert. Ein Hauptziel der Heuristiken fĂŒr nebenlĂ€ufige Anwendungen ist die Wiederverwertung von Zwischenergebnissen, welche von mehreren Anwedungen geteilt werden

    Optimizing latency and reliability of pipeline workflow applications

    Get PDF
    (ENG) Mapping applications onto heterogeneous platforms is a difficult challenge, even for simple application patterns such as pipeline graphs. Theproblem is even more complex when processors are subject to failureduring the execution of the application. In this paper, we study thecomplexity of a bi-criteria mapping which aims at optimizing the la-tency (i.e., the response time) and the reliability (i.e., the probabilitythat the computation will be successful) of the application. Latency isminimized by using faster processors, while reliability is increased byreplicating computations on a set of processors. However, replicationincreases latency (additional communications, slower processors). Theapplication fails to be executed only if all the processors fail duringexecution. While simple polynomial algorithms can be found for fullyhomogeneous platforms, the problem becomes NP-hard when tacklingheterogeneous platforms. This is yet another illustration of the additional complexity added by heterogeneity.L’ordonnancement et l’allocation des applications sur plates-formes hĂ©tĂ©rogĂšnes sont des problĂšmes cruciaux, mĂȘme pour des applications simples comme des graphes en pipeline. Le problĂšme devient mĂȘme encore plus complexe quand les processeurs peuvent tomber en panne pendant l’exĂ©cution de l’application. Dans cet article, nous Ă©tudions la complexitĂ© d’une allocation bi-critĂšre qui vise Ă  optimiser la latence (i.e., le temps de rĂ©ponse) et la fiabilitĂ© (i.e., la probabilitĂ© que le calcul rĂ©ussisse)de l’application. La latence est minimisĂ©e en utilisant des processeurs rapides, tandis que la fiabilitĂ© est augmentĂ©e en rĂ©pliquant les calculs sur un ensemble de processeurs. Toutefois, la rĂ©plication augmente la latence (communications additionnelles et processeurs moins rapides). L’application Ă©choue Ă  ĂȘtre exĂ©cutĂ©e seulement si tout les processeurs Ă©chouent pendant l’exĂ©cution. Des algorithmes simples en temps polynomial peuvent ĂȘtre trouvĂ©s pour plates-formes complĂštement homogĂšnes,tandis que le problĂšme devient NP-dur quand on s’attaque aux plates-formes hĂ©tĂ©rogĂšnes. C’est encore une autre illustration de la complexitĂ© additionnelle due Ă  l’hĂ©tĂ©rogĂ©nĂ©itĂ©

    Resource Allocation Strategies for In-Network Stream Processing

    Get PDF
    In this paper we consider the operator mapping problem for in-network stream processing applications. In-network stream processing consists in applying a tree of operators in steady-state to multiple data objects that are continually updated at various locations on a network. Examples of in-network stream processing include the processing of data in a sensor network, or of continuous queries on distributed relational databases. We study the operator mapping problem in a ``constructive'' scenario, i.e., a scenario in which one builds a platform dedicated to the application buy purchasing processing servers with various costs and capabilities. The objective is to minimize the cost of the platform while ensuring that the application achieves a minimum steady-state throughput. The first contribution of this paper is the formalization of a set of relevant operator-placement problems as linear programs, and a proof that even simple versions of the problem are NP-complete. Our second contribution is the design of several polynomial time heuristics, which are evaluated via extensive simulations and compared to theoretical bounds for optimal solutions

    Optimal Energy Consumption and Throughput for Workflow Applications on Distributed Architectures.

    No full text
    In this research report we study both the throughput and the energy optimization problem for a distributed system subject to failures that executes a workflow at different speed levels. The application is modeled as a directed acyclic graph composed of typed tasks linked by dependency constraints. A continuous flow, or a great number of application instances has to be processed optimizing the collaborative system performance which implies to increase the throughput -- the number of application instances processed by time unit --~or to decrease the period~-- the time needed to output one instance of the system. The system is designed as a collaborative platform of distributed machines. Each machine collaborates with others by performing all the instances of at least one task of the DAG. The problem we tackle is to optimize the configuration of the platform. In this report we propose two polynomial algorithms that optimize the two objectives of period (i.e., throughput) and energy minimization and we prove that the proposed algorithms give optimal results. Our optimization approach is hierarchic in the sens that we either minimize the energy consumption for an optimal period or minimize the period for the optimal energy consumption

    Bi-criteria Pipeline Mappings for Parallel Image Processing

    Get PDF
    Mapping workflow applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline graphs. Several antagonistic criteria should be optimized, such as throughput and latency (or a combination). Typical applications include digital image processing, where images are processed in steady-state mode. In this paper, we study the mapping of a particular image processing application, the JPEG encoding. Mapping pipelined JPEG encoding onto parallel platforms is useful for instance for encoding Motion JPEG images. As the bi-criteria mapping problem is NP-complete, we concentrate on the evaluation and performance of polynomial heuristics

    Sheduling approach for Microfactories with setup times.

    No full text
    International audienceIn this paper we consider microfactories for manipulation and assembly. These microfactories are composed of several cells containing microrobotic systems capable of a high level of repeatability. The assembly plan of the production is a pipeline of tasks that are performed by the cells. Our aim is to manage the production flow in the case where the cells can be reconfigured to perform different task types. Each cell is in charge of several consecutive tasks. A setup time is necessary to switch from the processing of one task type to another, and multiple intermediate results may be stored temporarily in storage areas to avoid switching the task type after the processing of each product. In this context we assess the optimized use of these storage areas, called buffers, and its impact on the production throughput

    Multi-criteria Mapping and Scheduling of Workflow Applications onto Heterogeneous Platforms

    Get PDF
    The results summarized in this document deal with the mapping and scheduling of workow applications on heterogeneous platforms. In this context, we focus on three different types of streaming applications:Replica placement in tree networks - In this kind of application, clients are issuing requests to some servers and the question is where to place replicas in the network such that all requests can be processed. We discuss and compare several policies to place replicas in tree networks, subject to server capacity, Quality of Service (QoS) and bandwidth constraints. The client requests are known beforehand, while the number and location of the servers have to be determined. The standard approach in the literature is to enforce that all requests of a client be served by the closest server inthe tree. We introduce and study two new policies. One major contribution of this work is to assess the impact of these new policies on the total replication cost. Another important goal is to assess the impact of server heterogeneity, both from a theoreticaland a practical perspective. We establish several new complexity results, and provide several efficient polynomial heuristics for NP-complete instances of the problem.Pipeline workflow applications - We consider workflow applications that can be expressed as linear pipeline graphs. An example for this application type is digital image processing, where images are treated in steady-state mode. Several antagonist criteria should be optimized, such as throughput and latency (or a combination) as well as latency and reliability (i.e., the probability that the computation will be successful) of the application. While simple polynomial algorithms can be found for fully homogeneous platforms, the problem becomes NP-hard when tackling heterogeneous platforms. We present an integer linear programming formulation for this latter problem. Furthermore, we provide several efficient polynomial bi-criteria heuristics, whose relative performances are evaluated through extensive simulation. As a case-study, we provide simulations and MPI experimental results for the JPEG encoder application pipeline on a cluster of workstations.Complex streaming applications - We consider the execution of applications structured as trees of operators, i.e., the application of one or several trees of operators in steady-state to multiple data objects that are continuously updated at various locations ina network. A first goal is to provide the user with a set of processors that should be bought or rented in order to ensure that the application achieves a minimum steady-state throughput, and with the objective of minimizing platform cost. We then extend our model to multiple applications: several concurrent applications are executed at the same time in a network, and one has to ensure that all applications can reach their application throughput. Another contribution of this work is to provide complexity results for different instances of the basic problem, as well as integer linear program formulations of various problem instances. The third contribution is the design of several polynomial-time heuristics, for both application models. One of the primary objectives of the heuristics for concurrent applications is to reuse intermediate resultsshared by multiple applications.Les travaux prĂ©sentĂ©s dans cette thĂšse portent sur le placement et l'ordonnancement d'applications de flux de donnĂ©es sur des plates-formes hĂ©tĂ©rogĂšnes. Dans ce contexte, nous nous concentrons sur trois types diffĂ©rents d'applications :Placement de rĂ©pliques dans les rĂ©seaux hiĂ©rarchiques - Dans ce type d'application, plusieurs clients Ă©mettent des requĂȘtes Ă  quelques serveurs et la question est : oĂč doit-on placer des rĂ©pliques dans le rĂ©seau afin que toutes les requĂȘtes puissent ĂȘtre traitĂ©es. Nous discutons et comparons plusieurs politiques de placement de rĂ©pliques dans des rĂ©seaux hiĂ©rarchiques en respectant des contraintes de capacitĂ© de serveur, de qualitĂ©de service et de bande-passante. Les requĂȘtes des clients sont connues a priori, tandis que le nombre et la position des serveurs sont Ă  dĂ©terminer. L'approche traditionnelle dans la littĂ©rature est de forcer toutes les requĂȘtes d'un client Ă  ĂȘtre traitĂ©es par le serveur le plus proche dans le rĂ©seau hiĂ©rarchique. Nous introduisons et Ă©tudions deux nouvelles politiques. Une principale contribution de ce travail est l'Ă©valuation de l'impact de ces nouvelles politiques sur le coĂ»t total de replication. Un autre but important est d'Ă©valuer l'impact de l'hĂ©tĂ©rogĂ©nĂ©itĂ© des serveurs, d'une perspective Ă  lafois thĂ©orique et pratique. Nous Ă©tablissons plusieurs nouveaux rĂ©sultats de complexitĂ©, et nous prĂ©sentons plusieurs heuristiques efficaces en temps polynomial.Applications de flux de donnĂ©es - Nous considĂ©rons des applications de flux de donnĂ©es qui peuvent ĂȘtre exprimĂ©es comme des graphes linĂ©aires. Un exemple pour ce type d'application est le traitement numĂ©rique d'images, oĂč les images sont traitĂ©es enrĂ©gime permanent. Plusieurs critĂšres antagonistes doivent ĂȘtre optimisĂ©s, tels que le dĂ©bit et la latence (ou une combinaison) ainsi que la latence et la fiabilitĂ© (i.e. la probabilitĂ© que le calcul soit rĂ©ussi) de l'application. Bien qu'il soit possible de trouverdes algorithmes polynomiaux simples pour les plates-formes entiĂšrement homogĂšnes, le problĂšme devient NP-difficile lorsqu'on s'attaque Ă  des plates-formes hĂ©tĂ©rogĂšnes. Nous prĂ©sentons une formulation en programme linĂ©aire pour ce dernier problĂšme. Deplus nous introduisons plusieurs heuristiques bi-critĂšres efficaces en temps polynomial, dont la performance relative est Ă©valuĂ©e par des simulations extensives. Dans une Ă©tude de cas, nous prĂ©sentons des simulations et des rĂ©sultats expĂ©rimentaux (programmĂ©s en MPI) pour le graphe d'application de l'encodeur JPEG sur une grappe de calcul.Applications complexes de streaming - ConsidĂ©rons l'exĂ©cution d'applications organisĂ©es en arbres d'opĂ©rateurs, i.e. l'application en rĂ©gime permanent d'un ou plusieurs arbres d'opĂ©rateurs Ă  donnĂ©es multiples qui doivent ĂȘtre mis Ă  jour continuellement Ă  diffĂ©rents endroits du rĂ©seau. Un premier but est de fournir Ă  l'utilisateur un ensemble de processeurs qui doit ĂȘtre achetĂ© ou louĂ© pour garantir que le dĂ©bit minimum de l'application en rĂ©gime permanent soit atteint. Puis nous Ă©tendons notre modĂšle aux applications multiples : plusieurs applications concurrentes sont exĂ©cutĂ©es en mĂȘmetemps dans un rĂ©seau, et on doit assurer que toutes les applications puissent atteindre leur dĂ©bit requis. Une autre contribution de ce travail est d'apporter des rĂ©sultats de complexitĂ© pour des instances variĂ©es du problĂšme. La troisiĂšme contribution est l'Ă©laborationde plusieurs heuristiques polynomiales pour les deux modĂšles d'application. Un objectif premier des heuristiques pour applications concurrentes est la rĂ©utilisation des rĂ©sultats intermĂ©diaires qui sont partagĂ©s parmi diffĂ©rentes applications
    corecore