Asymptotically Optimal Approximation Algorithms for Coflow Scheduling

Many modern datacenter applications involve large-scale computations composed of multiple data flows that need to be completed over a shared set of distributed resources. Such a computation completes when all of its flows complete. A useful abstraction for modeling such scenarios is a {\em coflow}, which is a collection of flows (e.g., tasks, packets, data transmissions) that all share the same performance goal. In this paper, we present the first approximation algorithms for scheduling coflows over general network topologies with the objective of minimizing total weighted completion time. We consider two different models for coflows based on the nature of individual flows: circuits, and packets. We design constant-factor polynomial-time approximation algorithms for scheduling packet-based coflows with or without given flow paths, and circuit-based coflows with given flow paths. Furthermore, we give an $O(\log n/\log \log n)$-approximation polynomial time algorithm for scheduling circuit-based coflows where flow paths are not given (here $n$ is the number of network edges). We obtain our results by developing a general framework for coflow schedules, based on interval-indexed linear programs, which may extend to other coflow models and objective functions and may also yield improved approximation bounds for specific network scenarios. We also present an experimental evaluation of our approach for circuit-based coflows that show a performance improvement of at least 22% on average over competing heuristics.Comment: Fixed minor typo

Scheduling Coflows for Minimizing the Makespan in Identical Parallel Networks

With the development of technology, parallel computing applications have been commonly executed in large data centers. These parallel computing applications include the computation phase and communication phase, and work is completed by repeatedly executing these two phases. However, due to the ever-increasing computing demands, large data centers are burdened with massive communication demands. Coflow is a recently proposed networking abstraction to capture communication patterns in data-parallel computing frameworks. This paper focuses on the coflow scheduling problem in identical parallel networks, where the goal is to minimize makespan, the maximum completion time of coflows. The coflow scheduling problem in huge data center is considered one of the most significant $NP$-hard problems. In this paper, coflow can be considered as either a divisible or an indivisible case. Distinct flows in a divisible coflow can be transferred through different network cores, while those in an indivisible coflow can only be transferred through the same network core. In the divisible coflow scheduling problem, this paper proposes a $(3-\tfrac{2}{m})$-approximation algorithm, and a $(\tfrac{8}{3}-\tfrac{2}{3m})$-approximation algorithm, where $m$ is the number of network cores. In the indivisible coflow scheduling problem, this paper proposes a $(2m)$-approximation algorithm. Finally, we simulate our proposed algorithm and Weaver's [Huang \textit{et al.}, In 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 1071-1081, 2020.] and compare the performance of our algorithms with that of Weaver's

Scheduling Coflows for Minimizing the Total Weighted Completion Time in Identical Parallel Networks

Coflow is a recently proposed network abstraction to capture communication patterns in data centers. The coflow scheduling problem in large data centers is one of the most important $NP$-hard problems. Previous research on coflow scheduling focused mainly on the single-switch model. However, with recent technological developments, this single-core model is no longer sufficient. This paper considers the coflow scheduling problem in identical parallel networks. The identical parallel network is an architecture based on multiple network cores running in parallel. Coflow can be considered as divisible or indivisible. Different flows in a divisible coflow can be transmitted through different network cores. Considering the divisible coflow scheduling problem, we propose a $(6-\frac{2}{m})$-approximation algorithm with arbitrary release times, and a $(5-\frac{2}{m})$-approximation without release time, where $m$ is the number of network cores. On the other hand, when coflow is indivisible, we propose a $(7-\frac{2}{m})$-approximation algorithm with arbitrary release times, and a $(6-\frac{2}{m})$-approximation without release time