Modeling Network Contention Effects on All-to-All Operations

10 pagesOne of the most important collective communication patterns used in scientific applications is the complete exchange, also called All-to-All. Although efficient complete exchange algorithms have been studied for specific networks, general solutions like those available in well-known MPI distributions (e.g. the MPI_Alltoall operation) are strongly influenced by the congestion of network resources. In this paper we present an integrated approach to model the performance of the All-to-All collective operation. Our approach consists in identifying a contention signature that characterizes a given network environment, using it to augment a contention-free communication model. This approach allows an accurate prediction of the performance of the All-to-All operation over different network architectures with a small overhead. This approach is assessed by experimental results using three different network architectures, namely Fast Ethernet, Gigabit Ethernet and Myrinet

A job response time prediction method for production Grid computing environments

A major obstacle to the widespread adoption of Grid Computing in both the scientific community and industry sector is the difficulty of knowing in advance a job submission running cost that can be used to plan a correct allocation of resources. Traditional distributed computing solutions take advantage of homogeneous and open environments to propose prediction methods that use a detailed analysis of the hardware and software components. However, production Grid computing environments, which are large and use a complex and dynamic set of resources, present a different challenge. In Grid computing the source code of applications, programme libraries, and third-party software are not always available. In addition, Grid security policies may not agree to run hardware or software analysis tools to generate Grid components models. The objective of this research is the prediction of a job response time in production Grid computing environments. The solution is inspired by the concept of predicting future Grid behaviours based on previous experiences learned from heterogeneous Grid workload trace data. The research objective was selected with the aim of improving the Grid resource usability and the administration of Grid environments. The predicted data can be used to allocate resources in advance and inform forecasted finishing time and running costs before submission. The proposed Grid Computing Response Time Prediction (GRTP) method implements several internal stages where the workload traces are mined to produce a response time prediction for a given job. In addition, the GRTP method assesses the predicted result against the actual target job’s response time to inference information that is used to tune the methods setting parameters. The GRTP method was implemented and tested using a cross-validation technique to assess how the proposed solution generalises to independent data sets. The training set was taken from the Grid environment DAS (Distributed ASCI Supercomputer). The two testing sets were taken from AuverGrid and Grid5000 Grid environments Three consecutive tests assuming stable jobs, unstable jobs, and using a job type method to select the most appropriate prediction function were carried out. The tests offered a significant increase in prediction performance for data mining based methods applied in Grid computing environments. For instance, in Grid5000 the GRTP method answered 77 percent of job prediction requests with an error of less than 10 percent. While in the same environment, the most effective and accurate method using workload traces was only able to predict 32 percent of the cases within the same range of error. The GRTP method was able to handle unexpected changes in resources and services which affect the job response time trends and was able to adapt to new scenarios. The tests showed that the proposed GRTP method is capable of predicting job response time requests and it also improves the prediction quality when compared to other current solutions

Prédiction de Performances pour les Communications Collectives

12 pagesNational audienceDes travaux récents visent lóptimisation des opérations de communication collective dans les environnements de type grille de calcul. La solution la plus répandue est la séparation des communications internes et externes à chaque grappe, mais cela néxclut pas le découpage des communications en plusieurs couches, pratique efficace démontrée par Karonis et al. [10]. Dans les deux cas, la prédiction des performances est un facteur essentiel, soit pour le réglage fin des paramètres de communication, soit pour le calcul de la distribution et de la hiérarchie des communications. Pour cela, il est très important dávoir des modèles précis des communications collectives, lesquels seront utilisés pour prédire ces performances. Cet article décrit notre expérience sur la modélisation des opérations de communication collective. Nous présentons des modèles de communication pour différents patrons de communication collective comme un vers plusieurs, un vers plusieurs personnalisé et plusieurs vers plusieurs. Pour évaluer la précision des modèles, nous comparons les prédictions obtenues avec les résultats des expérimentations effectuées sur deux environnements réseaux différents, Fast Ethernet et Myrinet