Asymptotically Optimal Approximation Algorithms for Coflow Scheduling

Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a {\em coflow}, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal. In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We consider two different models for coflows based on the nature of individual flows: circuits, and packets. We design constant-factor polynomial-time approximation algorithms for scheduling packet-based coflows with or without given flow paths, and circuit-based coflows with given flow paths. Furthermore, we give an $O(\log n/\log \log n)$-approximation polynomial time algorithm for scheduling circuit-based coflows where flow paths are not given (here $n$ is the number of network edges). We obtain our results by developing a general framework for coflow schedules, based on interval-indexed linear programs, which may extend to other coflow models and objective functions and may also yield improved approximation bounds for specific network scenarios. We also present an experimental evaluation of our approach for circuit-based coflows that show a performance improvement of at least 22% on average over competing heuristics.Comment: Fixed minor typo

Multiprocessor task scheduling in multistage hyrid flowshops: a genetic algorithm approach

This paper considers multiprocessor task scheduling in a multistage hybrid flow-shop environment. The objective is to minimize the make-span, that is, the completion time of all the tasks in the last stage. This problem is of practical interest in the textile and process industries. A genetic algorithm (GA) is developed to solve the problem. The GA is tested against a lower bound from the literature as well as against heuristic rules on a test bed comprising 400 problems with up to 100 jobs, 10 stages, and with up to five processors on each stage. For small problems, solutions found by the GA are compared to optimal solutions, which are obtained by total enumeration. For larger problems, optimum solutions are estimated by a statistical prediction technique. Computational results show that the GA is both effective and efficient for the current problem. Test problems are provided in a web site at www.benchmark.ibu.edu.tr/mpt-h; fsp

Scheduling Storms and Streams in the Cloud

Motivated by emerging big streaming data processing paradigms (e.g., Twitter Storm, Streaming MapReduce), we investigate the problem of scheduling graphs over a large cluster of servers. Each graph is a job, where nodes represent compute tasks and edges indicate data-flows between these compute tasks. Jobs (graphs) arrive randomly over time, and upon completion, leave the system. When a job arrives, the scheduler needs to partition the graph and distribute it over the servers to satisfy load balancing and cost considerations. Specifically, neighboring compute tasks in the graph that are mapped to different servers incur load on the network; thus a mapping of the jobs among the servers incurs a cost that is proportional to the number of "broken edges". We propose a low complexity randomized scheduling algorithm that, without service preemptions, stabilizes the system with graph arrivals/departures; more importantly, it allows a smooth trade-off between minimizing average partitioning cost and average queue lengths. Interestingly, to avoid service preemptions, our approach does not rely on a Gibbs sampler; instead, we show that the corresponding limiting invariant measure has an interpretation stemming from a loss system.Comment: 14 page

Approximation and online algorithms in scheduling and coloring

In the last three decades, approximation and online algorithms have become a major area of theoretical computer science and discrete mathematics. Scheduling and coloring problems are among the most popular ones for which approximation and online algorithms have been analyzed. On one hand, motivated by the well-known difficulty to obtain good lower bounds for the problems, it is particularly hard to prove results on the online and offline performance of algorithms. On the other hand, the theoretically oriented studies of approximation and online algorithms for scheduling and coloring have also impact on the development of better algorithms for real world applications. In the thesis we present approximation algorithms and online algorithms for a number of scheduling and labeling (coloring) problems. Our work in the first part of the thesis is devoted to scheduling problems with the average weighted completion time objective function, that is primarily motivated by some theoretical questions which were open for a number of recent years. Here we present a general method which leads to the design of polynomial time approximation schemes (PTASs), best possible approximation results. In contrast, our work in the second part of the thesis is motivated by practical applications. We consider a number of new labeling and scheduling problems which occur in the design of communication networks. Here we present and analyze efficient approximation and online algorithms. We use very simple techniques which do not require large computational resources

Algorithms and complexity analyses for some combinational optimization problems

The main focus of this dissertation is on classical combinatorial optimization problems in two important areas: scheduling and network design. In the area of scheduling, the main interest is in problems in the master-slave model. In this model, each machine is either a master machine or a slave machine. Each job is associated with a preprocessing task, a slave task and a postprocessing task that must be executed in this order. Each slave task has a dedicated slave machine. All the preprocessing and postprocessing tasks share a single master machine or the same set of master machines. A job may also have an arbitrary release time before which the preprocessing task is not available to be processed. The main objective in this dissertation is to minimize the total completion time or the makespan. Both the complexity and algorithmic issues of these problems are considered. It is shown that the problem of minimizing the total completion time is strongly NP-hard even under severe constraints. Various efficient algorithms are designed to minimize the total completion time under various scenarios. In the area of network design, the survivable network design problems are studied first. The input for this problem is an undirected graph G = (V, E), a non-negative cost for each edge, and a nonnegative connectivity requirement ruv for every (unordered) pair of vertices &ruv. The goal is to find a minimum-cost subgraph in which each pair of vertices u,v is joined by at least ruv edge (vertex)-disjoint paths. A Polynomial Time Approximation Scheme (PTAS) is designed for the problem when the graph is Euclidean and the connectivity requirement of any point is at most 2. PTASs or Quasi-PTASs are also designed for 2-edge-connectivity problem and biconnectivity problem and their variations in unweighted or weighted planar graphs. Next, the problem of constructing geometric fault-tolerant spanners with low cost and bounded maximum degree is considered. The first result shows that there is a greedy algorithm which constructs fault-tolerant spanners having asymptotically optimal bounds for both the maximum degree and the total cost at the same time. Then an efficient algorithm is developed which finds fault-tolerant spanners with asymptotically optimal bound for the maximum degree and almost optimal bound for the total cost