Implementing a tool for designing portable parallel programs

The Implementation aspects of a novel parallel programming model called Cluster-M is presented in this thesis. This model provides an environment for efficiently designing highly parallel portable software. The two main components of this model are Cluster-M Specifications and Cluster-M Representations. A Cluster-M Specification consists of a number of clustering levels emphasizing computation and communication requirements of a parallel solution to a given problem. A Cluster-M Representation on the other hand, represents a multi-layered partitioning of a system graph corresponding to the topology of the target architecture. A set of basic constructs essential for writing Cluster-M Specifications using PCN are presented. Also, a. C program for generating the Cluster-M Representations is shown. Cluster-M Specifications are to be mapped onto the Representations using a proposed mapping methodology. Using Cluster-M a single software can be ported among various parallel computing systems. This thesis concentrates on the implementation of the Specifications and the Representations

Optimal cube-connected cube multiprocessors

Many CFD (computational fluid dynamics) and other scientific applications can be partitioned into subproblems. However, in general the partitioned subproblems are very large. They demand high performance computing power themselves, and the solutions of the subproblems have to be combined at each time step. The cube-connect cube (CCCube) architecture is studied. The CCCube architecture is an extended hypercube structure with each node represented as a cube. It requires fewer physical links between nodes than the hypercube, and provides the same communication support as the hypercube does on many applications. The reduced physical links can be used to enhance the bandwidth of the remaining links and, therefore, enhance the overall performance. The concept and the method to obtain optimal CCCubes, which are the CCCubes with a minimum number of links under a given total number of nodes, are proposed. The superiority of optimal CCCubes over standard hypercubes was also shown in terms of the link usage in the embedding of a binomial tree. A useful computation structure based on a semi-binomial tree for divide-and-conquer type of parallel algorithms was identified. It was shown that this structure can be implemented in optimal CCCubes without performance degradation compared with regular hypercubes. The result presented should provide a useful approach to design of scientific parallel computers

Efficient structural outlooks for vertex product networks

In this thesis, a new classification for a large set of interconnection networks, referred to as "Vertex Product Networks" (VPN), is provided and a number of related issues are discussed including the design and evaluation of efficient structural outlooks for algorithm development on this class of networks. The importance of studying the VPN can be attributed to the following two main reasons: first an unlimited number of new networks can be defined under the umbrella of the VPN, and second some known networks can be studied and analysed more deeply. Examples of the VPN include the newly proposed arrangement-star and the existing Optical Transpose Interconnection Systems (OTIS-networks). Over the past two decades many interconnection networks have been proposed in the literature, including the star, hyperstar, hypercube, arrangement, and OTIS-networks. Most existing research on these networks has focused on analysing their topological properties. Consequently, there has been relatively little work devoted to designing efficient parallel algorithms for important parallel applications. In an attempt to fill this gap, this research aims to propose efficient structural outlooks for algorithm development. These structural outlooks are based on grid and pipeline views as popular structures that support a vast body of applications that are encountered in many areas of science and engineering, including matrix computation, divide-and- conquer type of algorithms, sorting, and Fourier transforms. The proposed structural outlooks are applied to the VPN, notably the arrangement-star and OTIS-networks. In this research, we argue that the proposed arrangement-star is a viable candidate as an underlying topology for future high-speed parallel computers. Not only does the arrangement-star bring a solution to the scalability limitations from which the Abstract existing star graph suffers, but it also enables the development of parallel algorithms based on the proposed structural outlooks, such as matrix computation, linear algebra, divide-and-conquer algorithms, sorting, and Fourier transforms. Results from a performance study conducted in this thesis reveal that the proposed arrangement-star supports efficiently applications based on the grid or pipeline structural outlooks. OTIS-networks are another example of the VPN. This type of networks has the important advantage of combining both optical and electronic interconnect technology. A number of studies have recently explored the topological properties of OTIS-networks. Although there has been some work on designing parallel algorithms for image processing and sorting, hardly any work has considered the suitability of these networks for an important class of scientific problems such as matrix computation, sorting, and Fourier transforms. In this study, we present and evaluate two structural outlooks for algorithm development on OTIS-networks. The proposed structural outlooks are general in the sense that no specific factor network or problem domain is assumed. Timing models for measuring the performance of the proposed structural outlooks are provided. Through these models, the performance of various algorithms on OTIS-networks are evaluated and compared with their counterparts on conventional electronic interconnection systems. The obtained results reveal that OTIS-networks are an attractive candidate for future parallel computers due to their superior performance characteristics over networks using traditional electronic interconnects

Divide-and-conquer algorithms for multiprocessors

During the past decade there has been a tremendous surge in understanding the nature of parallel computation. A number of parallel computers are commercially available. However, there are some problems in developing application programs on these computers;This dissertation considers various issues involved in implementing parallel algorithms on Multiple Instruction Multiple Data (MIMD) machines with a bounded number of processors. Strategies for implementing divide-and-conquer algorithms on MIMD machines are proposed. Results linking time complexity, communication complexity and the complexity of divide-and-combine functions of divide-and-conquer algorithms are analyzed. An efficient criterion for partitioning a parallel program is proposed and a method for obtaining a closed form expression for time complexity of a parallel program in terms of problem size and number of processors is developed

OREGAMI: Software Tools for Mapping Parallel Computations to Parallel Architectures

22 pagesThe mapping problem in message-passing parallel processors involves the assignment of tasks in a parallel computation to processors and the routing of inter-task messages along the links of the interconnection network. We have developed a unified set of software tools called OREGAMI for automatic and guided mapping of parallel computations to parallel architectures in order to achieve portability and maximal performance from parallel systems. Our tools include a description language which enables the programmer of parallel algorithms to specify information about the static and dynamic communication behavior of the computation to be mapped. This information is used by the mapping algorithms to assign tasks to processors and to route communication in the network topology. Two key features of our system are (a) the ability to take advantage of the regularity present in both the computation structure and the interconnection network and (b) the desire to balance the user's knowledge and intuition with the computational power of efficient combinatorial algorithms

Symmetric Tori connected Torus Network

A Symmetric Tori connected Torus Network (STTN) is a 2D-torus network of multiple basic modules, in which the basic modules are 2D-torus networks that are hierarchically interconnected for higher-level networks. In this paper, we present the architecture of the STTN, addressing of node, routing of message, and evaluate the static network performance of STTN, TTN, TESH, mesh, and torus networks. It is shown that the STTN possesses several attractive features, including constant degree, small diameter, low cost, small average distance, moderate bisection width, and high fault tolerant performance than that of other conventional and hierarchical interconnection networks

Algorithmic skeletons as a method of parallel programming

A new style of abstraction for program development, based on the concept of algorithmic skeletons, has been proposed in the literature. The programmer is offered a variety of independent algorithmic skeletons each of which describe the structure of a particular style of algorithm. The appropriate skeleton is used by the system to mould the solution. Parallel programs are particularly appropriate for this technique because of their complexity. This thesis investigates algorithmic skeletons as a method of hiding the complexities of parallel programming from the user, and for guiding them towards efficient solutions. To explore this approach, this thesis describes the implementation and benchmarking of the divide and conquer and task queue paradigms as skeletons. All but one category of problem, as implemented in this thesis, scale well over eight processors. The rate of speed up tails off when there are significant communication requirements. The results show that, with some user knowledge, efficient parallel programs can be developed using this method. The evaluation explores methods for fine tuning some skeleton programs to achieve increased efficiency