Performance Analysis And Optimal Utilization Of Inter-Process Communications On A Commodity Cluster

Classical science is based on theory, observation and physical experimentation. Contemporary science is characterized by theory, observation, experimentation and numerical simulation. With the use of hardware and software we can simulate lots of phenomenon. This saves time, money and physical resources. Simulation of a certain phenomenon requires lots of computing power. Answer to these computational power needs is high performance computer. High performance computers consist of numerous processors working on same task in parallel. In the past, high performance computers were very expensive and affordable by few institutions. After Message Passing Interface library is ported to PC platform, commodity clusters can be built of inexpensive PCs and afforded by any researcher. Lots of performance analyses have been conducted on high-end supercomputers. None has been done on commodity clusters. In this thesis, experiments for six major MPI communication functions were performed on eight different configurations of clusters. Performance analyses were then conducted on the results. Based on the results, methods for optimal utilization of inter-process communications on commodity clusters were proposed

Design study of MovementSlicer : an interactive visualization of patterns and group meetings in 2D movement data

Movement data collected through GPS or other technologies is increasingly common, but is difficult to visualize due to overplotting and occlusion of movements when displayed on 2D maps. An additional challenge is the extraction of useful higher-level information (such as meetings) derived from the raw movement data. We present a design study of MovementSlicer, a tool for visualizing the places visited, and behaviors of, individual actors, and also the meetings between multiple actors. We first present a taxonomy of visualizations of movement data, and then consider tasks to support when analyzing movement data and especially meetings of multiple actors. We argue that Gantt charts have many advantages for understanding the movements and meetings of small groups of moving entities, and present the design of a Gantt chart that can nest people within locations or locations within people along the vertical axis, and show time along the horizontal axis. The rows of our Gantt chart are sorted by activity level and can be filtered using a weighted adjacency matrix showing meetings between people. Empty time intervals in the Gantt chart can be automatically folded, with smoothly animated transitions, yielding a multi-focal view. Case studies demonstrate the utility of our prototype

Techniques To Facilitate the Understanding of Inter-process Communication Traces

High Performance Computing (HPC) systems play an important role in today’s heavily digitized world, which is in a constant demand for higher speed of calculation and performance. HPC applications are used in multiple domains such as telecommunication, health, scientific research, and more. With the emergence of multi-core and cloud computing platforms, the HPC paradigm is quickly becoming the design of choice of many service providers. HPC systems are also known to be complex to debug and analyze due to the large number of processes they involve and the way these processes communicate with each other to perform specific tasks. As a result, software engineers must spend extensive amount of time understanding the complex interactions among a system’s processes. This is usually done through the analysis of execution traces generated from running the system at hand. Traces, however, are very difficult to work with due to the overwhelming size of typical traces. The objective of this research is to present a set of techniques that facilitates the understanding of the behaviour of HPC applications through the analysis of system traces. The first technique consists of building an exchange format called MTF (MPI Trace Format) for representing and exchanging traces generated from HPC applications based on the MPI (Message Passing Interface) standard, which is a de facto standard for inter-process communication for high performance computing systems. The design of MTF is validated against well-known requirements for a standard exchange format. The second technique aims to facilitate the understanding of large traces of inter-process communication by automatically extracting communication patterns that characterize their main behaviour. Two algorithms are presented. The first one permits the recognition of repeating patterns in traces of MPI (Message Passing Interaction) applications whereas the second algorithm searches if a given communication pattern occurs in a trace. Both algorithms are based on the n-gram extraction technique used in natural language processing. Finally, we developed a technique to abstract MPI traces by detecting the different execution phases in a program based on concepts from information theory. Using this approach, software engineers can examine the trace as a sequence of high-level computational phases instead of a mere flow of low-level events. The techniques presented in this thesis have been tested on traces generated from real HPC programs. The results from several case studies demonstrate the usefulness and effectiveness of our techniques