Heuristic Algorithms for Scheduling Iterative Task Computations on Distributed Memory Machines

Many partitioned scientific programs can be modeled as iterative execution of computational tasks, represented by iterative task graphs (ITGs). In this paper, we consider the symbolic scheduling of ITGs on distributed memory architectures with nonzero communication overhead without searching the entire iteration space. An ITG may or may not have dependence cycles and we propose heuristic algorithms for mapping cyclic and acyclic ITGs, which incorporate techniques of software pipelining, graph unfolding, directed acyclic graph (DAG) scheduling and load balancing. We provide an analysis for computing near-optimal unfolding factors and comparing the performance of the proposed heuristic algorithms with the optimal solutions. We also study the stability of run-time performance when weights are not estimated accurately at compile-time. Our experiments study the scheduling performance of solving several scientific computing problems and analyze the effectiveness of optimization techniques us..