Problem and Machine Sensitive Communication Optimization

Reducing communication costs can significantly improve the execution time of a parallel program. This paper presents a new approach for communication optimization in data parallel programs that is based on global data flow analysis and performance prediction. Our techniques are based on simple yet highly effective data flow equations which are solved iteratively for arbitrary control flow graphs. Previous techniques are based on fixed communication optimization strategies whose quality can be very sensible to changes of problem and machine sizes. Our algorithm is novel in that we carefully examine tradeoffs between communication latency hiding and reducing the number and volume of messages (e.g. message coalescing and aggregating) by systematically evaluating a reasonable set of promising communication placements for a given program covering several (possibly conflicting) communication guiding profit motives. P 3 T , a state-of-the-art performance estimator that carefully models prob..