Optimal Replica Placement in Tree Networks with QoS and Bandwidth Constraints and the Closest Allocation Policy

This paper deals with the replica placement problem on fully homogeneous tree networks known as the Replica Placement optimization problem. The client requests are known beforehand, while the number and location of the servers are to be determined. We investigate the latter problem using the Closest access policy when adding QoS and bandwidth constraints. We propose an optimal algorithm in two passes using dynamic programming

Multi-criteria scheduling of pipeline workflows

Mapping workflow applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline graphs. Several antagonist criteria should be optimized, such as throughput and latency (or a combination). In this paper, we study the complexity of the bi-criteria mapping problem for pipeline graphs on communication homogeneous platforms. In particular, we assess the complexity of the well-known chains-to-chains problem for different-speed processors, which turns out to be NP-hard. We provide several efficient polynomial bi-criteria heuristics, and their relative performance is evaluated through extensive simulations

Optimizing latency and reliability of pipeline workflow applications

(ENG) Mapping applications onto heterogeneous platforms is a difficult challenge, even for simple application patterns such as pipeline graphs. Theproblem is even more complex when processors are subject to failureduring the execution of the application. In this paper, we study thecomplexity of a bi-criteria mapping which aims at optimizing the la-tency (i.e., the response time) and the reliability (i.e., the probabilitythat the computation will be successful) of the application. Latency isminimized by using faster processors, while reliability is increased byreplicating computations on a set of processors. However, replicationincreases latency (additional communications, slower processors). Theapplication fails to be executed only if all the processors fail duringexecution. While simple polynomial algorithms can be found for fullyhomogeneous platforms, the problem becomes NP-hard when tacklingheterogeneous platforms. This is yet another illustration of the additional complexity added by heterogeneity.L’ordonnancement et l’allocation des applications sur plates-formes hétérogènes sont des problèmes cruciaux, même pour des applications simples comme des graphes en pipeline. Le problème devient même encore plus complexe quand les processeurs peuvent tomber en panne pendant l’exécution de l’application. Dans cet article, nous étudions la complexité d’une allocation bi-critère qui vise à optimiser la latence (i.e., le temps de réponse) et la fiabilité (i.e., la probabilité que le calcul réussisse)de l’application. La latence est minimisée en utilisant des processeurs rapides, tandis que la fiabilité est augmentée en répliquant les calculs sur un ensemble de processeurs. Toutefois, la réplication augmente la latence (communications additionnelles et processeurs moins rapides). L’application échoue à être exécutée seulement si tout les processeurs échouent pendant l’exécution. Des algorithmes simples en temps polynomial peuvent être trouvés pour plates-formes complètement homogènes,tandis que le problème devient NP-dur quand on s’attaque aux plates-formes hétérogènes. C’est encore une autre illustration de la complexité additionnelle due à l’hétérogénéité

Multi-kritäres Mapping und Scheduling von Workflow-Anwendungen auf heterogenen Plattformen

The results summarized in this thesis deal with the mapping and scheduling of workflow applications on heterogeneous platforms. In this context, we focus on three different types of streaming applications: * Replica placement in tree networks * In this kind of application, clients are issuing requests to some servers and the question is where to place replicas in the network such that all requests can be processed. We discuss and compare several policies to place replicas in tree networks, subject to server capacity, Quality of Service (QoS) and bandwidth constraints. The client requests are known beforehand, while the number and location of the servers have to be determined. The standard approach in the literature is to enforce that all requests of a client be served by the closest server in the tree. We introduce and study two new policies. One major contribution of this work is to assess the impact of these new policies on the total replication cost. Another important goal is to assess the impact of server heterogeneity, both from a theoretical and a practical perspective. We establish several new complexity results, and provide several efficient polynomial heuristics for NP-complete instances of the problem. * Pipeline workflow applications * We consider workflow applications that can be expressed as linear pipeline graphs. An example for this application type is digital image processing, where images are treated in steady-state mode. Several antagonist criteria should be optimized, such as throughput and latency (or a combination) as well as latency and reliability (i.e., the probability that the computation will be successful) of the application. While simple polynomial algorithms can be found for fully homogeneous platforms, the problem becomes NP-hard when tackling heterogeneous platforms. We present an integer linear programming formulation for this latter problem. Furthermore, we provide several efficient polynomial bi-criteria heuristics, whose relative performances are evaluated through extensive simulation. As a case-study, we provide simulations and MPI experimental results for the JPEG encoder application pipeline on a cluster of workstations. * Complex streaming applications * We consider the execution of applications structured as trees of operators, i.e., the application of one or several trees of operators in steady-state to multiple data objects that are continuously updated at various locations in a network. A first goal is to provide the user with a set of processors that should be bought or rented in order to ensure that the application achieves a minimum steady-state throughput, and with the objective of minimizing platform cost. We then extend our model to multiple applications: several concurrent applications are executed at the same time in a network, and one has to ensure that all applications can reach their application throughput. Another contribution of this work is to provide complexity results for different instances of the basic problem, as well as integer linear program formulations of various problem instances. The third contribution is the design of several polynomial-time heuristics, for both application models. One of the primary objectives of the heuristics for concurrent applications is to reuse intermediate results shared by multiple applications.In meiner Dissertation beschäftige ich mich mit dem Scheduling von Workflow-Anwendungen in heterogenen Plattformen. In diesem Zusammenhang konzentriere ich mich auf drei verschiene Anwendungstypen.: * Platzierung von Replikaten in Baumnetzwerken * Dieses erste Schedulingproblem behan-delt die Platzierung von Replikaten in Baumnetzwerken. Ein Beispiel hierfür ist die Platzierung von Replikaten in verteilten Datenbanksystemen, deren Verbindungsstruktur baumartig organi-siert ist. Die Platzierung soll dabei unter mehreren Constraints (Serverkapazitäten, sowie Dienstgüte und Bandbreitenbeschränkungen) durchgeführt werden. In diesem Anwendungstyp stellen Clients Anfragen an verschiedene Server. Diese Client-Anfragen sind im Voraus bekannt, während Anzahl und Platzierung der Server erst ermittelt werden müssen. Die in der Literatur gängige Strategie fordert, dass alle Anfragen eines Clients vom nächstgelegenen Server im Baum behandelt werden. Es werden zwei neue Verfahrensweisen vorgestellt und untersucht. Ein wichtiges Teilergebnis dieser Studie bewertet die Auswirkung der beiden neuen Strategien auf die globalen Replikationskosten. Ausserdem wird der Einfluss von Heterogenität aus theore-tischer und praktischer Sicht untersucht. Es werden verschiedene Komplexitätsergebnisse erar-beitet und mehrere effiziente Polynomialzeit-Heuristiken für NP-vollständige Instanzen des Problems vorgestellt. * Lineare Workflow-Anwendungen * Als nächstes werden Workflow-Anwendungen untersucht, die als lineare Graphen dargestellt werden können. Ein Beispiel dieses Applikationstyps ist die digitale Bildverarbeitung, in der Bilder mittels einer Pipeline verarbeitet werden. Es sollen ver-schie¬dene gegensätzliche Kriterien optimiert werden, wie zum Beispiel Durchsatz und Latenz-zeit, beziehungsweise eine Kombination der beiden, aber auch Latenzzeit und Ausfallsicherheit der Anwendung. Während für vollhomogene Plattformen polynomiale Algorithmen gefunden werden können, wird das Problem NP-hart, sobald heterogene Plattformen angestrebt werden. Diese Arbeit beinhaltet eine vollständige Komplexitätsanalyse. Für die bisher unbekannten polynomialen Varianten des Problems werden optimale Algorithmen vorgeschlagen. Ein ganz-zahliges lineares Programm für das bekannte „chains-on-chains“ Problem für heterogene Plattformen wird vorgestellt. Des weiteren werden verschiedene effiziente polynomiale bi-kritäre Heuristiken präsentiert, deren relative Effizienz durch umfangreiche Simulationen eruiert werden. Eine Fallstudie beschäftigt sich mit der JPEG-Encoder-Pipeline. Hierbei werden Simulationen und MPI-basierte Auswertungen auf einem Rechen-Cluster erstellt. * Komplexe Streaming-Anwendungen * Als letztes wird die Ausführung von Anwendungen, die als Operator-Bäume strukturiert sind, untersucht. Konkret bedeutet dies, dass ein oder mehrere Operator-Bäume in stationärem Zustand auf mannigfaltige Datenobjekte angewendet werden, welche fortlaufend an verschiedenen Stellen im Netzwerk aktualisiert werden. Ein erstes Ziel ist, dem Benutzer eine Gruppe von Rechnern vorzuschlagen, die gekauft oder gemietet werden sollen, so dass die Anwendung einen minimalen stationären Durchsatz erzielt und gleichzeitig Plattformkosten minimiert werden können. Anschließend wird das Modell auf mehrere Anwendungen erweitert: verschiedene nebenläufige Anwendungen werden zeitgleich in einem Netzwerk ausgeführt und es muss sichergestellt werden, dass alle Anwendungen ihren Durchsatz erreichen können. Beide Modelle werden aus theoretischer Sicht untersucht und eine Komplexitäts-analyse für unterschiedliche Instanzen des Grundproblems, sowie Formulierungen als lineare Programme erstellt. Für beide Anwendungsmodelle werden verschiedene Polynomialzeit-Heuristiken präsentiert und charakterisiert. Ein Hauptziel der Heuristiken für nebenläufige Anwendungen ist die Wiederverwertung von Zwischenergebnissen, welche von mehreren Anwedungen geteilt werden

Resource Allocation Strategies for In-Network Stream Processing

In this paper we consider the operator mapping problem for in-network stream processing applications. In-network stream processing consists in applying a tree of operators in steady-state to multiple data objects that are continually updated at various locations on a network. Examples of in-network stream processing include the processing of data in a sensor network, or of continuous queries on distributed relational databases. We study the operator mapping problem in a ``constructive'' scenario, i.e., a scenario in which one builds a platform dedicated to the application buy purchasing processing servers with various costs and capabilities. The objective is to minimize the cost of the platform while ensuring that the application achieves a minimum steady-state throughput. The first contribution of this paper is the formalization of a set of relevant operator-placement problems as linear programs, and a proof that even simple versions of the problem are NP-complete. Our second contribution is the design of several polynomial time heuristics, which are evaluated via extensive simulations and compared to theoretical bounds for optimal solutions

Optimal Energy Consumption and Throughput for Workflow Applications on Distributed Architectures.

In this research report we study both the throughput and the energy optimization problem for a distributed system subject to failures that executes a workflow at different speed levels. The application is modeled as a directed acyclic graph composed of typed tasks linked by dependency constraints. A continuous flow, or a great number of application instances has to be processed optimizing the collaborative system performance which implies to increase the throughput -- the number of application instances processed by time unit --~or to decrease the period~-- the time needed to output one instance of the system. The system is designed as a collaborative platform of distributed machines. Each machine collaborates with others by performing all the instances of at least one task of the DAG. The problem we tackle is to optimize the configuration of the platform. In this report we propose two polynomial algorithms that optimize the two objectives of period (i.e., throughput) and energy minimization and we prove that the proposed algorithms give optimal results. Our optimization approach is hierarchic in the sens that we either minimize the energy consumption for an optimal period or minimize the period for the optimal energy consumption

Bi-criteria Pipeline Mappings for Parallel Image Processing

Mapping workflow applications onto parallel platforms is a challenging problem, even for simple application patterns such as pipeline graphs. Several antagonistic criteria should be optimized, such as throughput and latency (or a combination). Typical applications include digital image processing, where images are processed in steady-state mode. In this paper, we study the mapping of a particular image processing application, the JPEG encoding. Mapping pipelined JPEG encoding onto parallel platforms is useful for instance for encoding Motion JPEG images. As the bi-criteria mapping problem is NP-complete, we concentrate on the evaluation and performance of polynomial heuristics

Sheduling approach for Microfactories with setup times.

International audienceIn this paper we consider microfactories for manipulation and assembly. These microfactories are composed of several cells containing microrobotic systems capable of a high level of repeatability. The assembly plan of the production is a pipeline of tasks that are performed by the cells. Our aim is to manage the production flow in the case where the cells can be reconfigured to perform different task types. Each cell is in charge of several consecutive tasks. A setup time is necessary to switch from the processing of one task type to another, and multiple intermediate results may be stored temporarily in storage areas to avoid switching the task type after the processing of each product. In this context we assess the optimized use of these storage areas, called buffers, and its impact on the production throughput

A novel function for spumaretrovirus integrase: an early requirement for integrase-mediated cleavage of 2 LTR circles

Retroviral integration is central to viral persistence and pathogenesis, cancer as well as host genome evolution. However, it is unclear why integration appears essential for retrovirus production, especially given the abundance and transcriptional potential of non-integrated viral genomes. The involvement of retroviral endonuclease, also called integrase (IN), in replication steps apart from integration has been proposed, but is usually considered to be accessory. We observe here that integration of a retrovirus from the spumavirus family depends mainly on the quantity of viral DNA produced. Moreover, we found that IN directly participates to linear DNA production from 2-LTR circles by specifically cleaving the conserved palindromic sequence found at LTR-LTR junctions. These results challenge the prevailing view that integrase essential function is to catalyze retroviral DNA integration. Integrase activity upstream of this step, by controlling linear DNA production, is sufficient to explain the absolute requirement for this enzyme. The novel role of IN over 2-LTR circle junctions accounts for the pleiotropic effects observed in cells infected with IN mutants. It may explain why 1) 2-LTR circles accumulate in vivo in mutants carrying a defective IN while their linear and integrated DNA pools decrease; 2) why both LTRs are processed in a concerted manner. It also resolves the original puzzle concerning the integration of spumaretroviruses. More generally, it suggests to reassess 2-LTR circles as functional intermediates in the retrovirus cycle and to reconsider the idea that formation of the integrated provirus is an essential step of retrovirus production

Semi-infinite TASEP with a Complex Boundary Mechanism

We consider a totally asymmetric exclusion process on the positive half-line. When particles enter in the system according to a Poisson source, Liggett has computed all the limit distributions when the initial distribution has an asymptotic density. In this paper we consider systems for which particles enter at the boundary according to a complex mechanism depending on the current configuration in a finite neighborhood of the origin. For this kind of models, we prove a strong law of large numbers for the number of particles entered in the system at a given time. Our main tool is a new representation of the model as a multi-type particle system with infinitely many particle types