Scalability study in parallel computing

An asymptotic scalability metric, called Constant-Memory-per-Processor (CMP) scalability, is presented. This metric is useful in analyzing performance of a parallel algorithm on distributed memory architectures as the number of processors grows, but the memory size per processor remains fixed. To illustrate the CMP-scalability metric, parallel Matrix Multiplication (MM), Gauss-Jordan Elimination (GJE) with partial pivoting, and Fast Fourier Transform (FFT) algorithms are considered on the hypercube and two-dimensional mesh topologies;A comparison between the asymptotic CMP-scalability and the isoefficiency-scalability metrics is performed to attain a better understanding of scalability. An analysis of the scalability of GJE and FFT on a mesh predicts that GJE is asymptotically more scalable than FFT using the isoefficiency-scalability metric, but the CMP-scalability metric predicts that FFT is asymptotically more scalable than GJE. Closer investigation reveals that both are correct, and that each metric amounts to a different planer cross-section of the multi-dimensional performance surface. Combining information from both the isoefficiency and CMP-scalability metrics we are able to show how to predict the relative change in performance of two algorithms along the fixed-processor planar cross-section and the fixed-problem size planar cross-section;Scalability metrics such as the CMP-scalability metric and isoefficiency-scalability metric indicate the asymptotic behavior as the number of processors becomes large. However, we question how useful these metrics are on a specific machine with a fixed number of processors and a fixed memory per processor. We investigate the utility of the CMP-and isoefficiency-scalability metrics by a detailed analysis of the three algorithms on a 16K processor MasPar MP-1 machine. Included in our analysis are the effects of varying the processor speed and communication speeds of the parallel computer on the accuracy of these scalability metrics

Portable high-performance programs

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.Includes bibliographical references (p. 159-169).by Matteo Frigo.Ph.D