Building a Virtual Globus Grid in a Reconfigurable Environment - A case study: Grid5000

With the continuous evolution of distributed computing grids and with the perpetuous development of the available computing resources and protocols, there is a \textit{sine qua non} requirement to pass beyond the physical design of the grids. A viable solution is offered by virtual grids, having the advantage of flexible mapping and adaptation to live in-place resources. A software image is proposed, built with the use of the Globus Toolkit, the herein document describing the construction and configuratin phases as well as the deployment protocol in a live grid - Grid5000

Flexible And Secure Access To Computing Clusters

The investigation presented in this paper was prompted by the need to provide a manageablesolution for secure access to computing clusters with a federated authentication framework.This requirement is especially important for scientists who need direct access to computingnodes in order to run their applications (e.g. chemical or medical simulations) with proprietary,open-source or custom-developed software packages. Our existing software, whichenables non-Web clients to use Shibboleth-secured services, has been extended to providedirect SSH access to cluster nodes using the Linux Pluggable Authentication Modules mechanism.This allows Shibboleth users to run the required software on clusters. Validationand performance comparison with existing SSH authentication mechanisms confirm that thepresented tools satisfy the stated requirements

Enhanced Failure Detection Mechanism in MapReduce

The popularity of MapReduce programming model has increased interest in the research community for its improvement. Among the other directions, the point of fault tolerance, concretely the failure detection issue seems to be a crucial one, but that until now has not reached its satisfying level. Motivated by this, I decided to devote my main research during this period into having a prototype system architecture of MapReduce framework with a new failure detection service, containing both analytical (theoretical) and implementation part. I am confident that this work should lead the way for further contributions in detecting failures to any NoSQL App frameworks, and cloud storage systems in general

Towards trusted volunteer grid environments

Intensive experiences show and confirm that grid environments can be considered as the most promising way to solve several kinds of problems relating either to cooperative work especially where involved collaborators are dispersed geographically or to some very greedy applications which require enough power of computing or/and storage. Such environments can be classified into two categories; first, dedicated grids where the federated computers are solely devoted to a specific work through its end. Second, Volunteer grids where federated computers are not completely devoted to a specific work but instead they can be randomly and intermittently used, at the same time, for any other purpose or they can be connected or disconnected at will by their owners without any prior notification. Each category of grids includes surely several advantages and disadvantages; nevertheless, we think that volunteer grids are very promising and more convenient especially to build a general multipurpose distributed scalable environment. Unfortunately, the big challenge of such environments is, however, security and trust. Indeed, owing to the fact that every federated computer in such an environment can randomly be used at the same time by several users or can be disconnected suddenly, several security problems will automatically arise. In this paper, we propose a novel solution based on identity federation, agent technology and the dynamic enforcement of access control policies that lead to the design and implementation of trusted volunteer grid environments.Comment: 9 Pages, IJCNC Journal 201

Time-Independent Trace Acquisition Framework -- A Grid'5000 How-to

GRID5000This manual describes step-by-step how to create a Grid'5000 appliance that comprises all the tools needed to acquire time-independent traces of the execution of an MPI application. Time-independent traces are an original way to estimate the performance of parallel applications. It allows to totally decouple the acquisition of a trace from its replay in a simulation framework. This manual also details the different acquisition scenarios allowed by this approach. Traces can be acquired in a very classical way, by folding the execution on less resources, or by scattering the execution across multiple clusters.Ce manuel décrit pas à pas la création d'une image système pour Griud'5000 comprenant tous les outils nécessaires à l'acquisition de traces de l'exécution d'une application MPI qui sont indépendantes du temps. L'utilisation de telles traces est une approche originale pour estimer les performances d'applications parallèles. Cela permet de découpler entièrement l'acquisition d'une trace de son rejeu dans un environnement de simulation. Ce manuel décrit également les différents scénarios d'acquisition rendus possibles par cette approche. Les traces peuvent être obtenues de façon classique, en repliant l'exécution sur moins de ressources, ou encore en répartissant l'exécution sur plusieurs grappes de machines

A job response time prediction method for production Grid computing environments

A major obstacle to the widespread adoption of Grid Computing in both the scientific community and industry sector is the difficulty of knowing in advance a job submission running cost that can be used to plan a correct allocation of resources. Traditional distributed computing solutions take advantage of homogeneous and open environments to propose prediction methods that use a detailed analysis of the hardware and software components. However, production Grid computing environments, which are large and use a complex and dynamic set of resources, present a different challenge. In Grid computing the source code of applications, programme libraries, and third-party software are not always available. In addition, Grid security policies may not agree to run hardware or software analysis tools to generate Grid components models. The objective of this research is the prediction of a job response time in production Grid computing environments. The solution is inspired by the concept of predicting future Grid behaviours based on previous experiences learned from heterogeneous Grid workload trace data. The research objective was selected with the aim of improving the Grid resource usability and the administration of Grid environments. The predicted data can be used to allocate resources in advance and inform forecasted finishing time and running costs before submission. The proposed Grid Computing Response Time Prediction (GRTP) method implements several internal stages where the workload traces are mined to produce a response time prediction for a given job. In addition, the GRTP method assesses the predicted result against the actual target job’s response time to inference information that is used to tune the methods setting parameters. The GRTP method was implemented and tested using a cross-validation technique to assess how the proposed solution generalises to independent data sets. The training set was taken from the Grid environment DAS (Distributed ASCI Supercomputer). The two testing sets were taken from AuverGrid and Grid5000 Grid environments Three consecutive tests assuming stable jobs, unstable jobs, and using a job type method to select the most appropriate prediction function were carried out. The tests offered a significant increase in prediction performance for data mining based methods applied in Grid computing environments. For instance, in Grid5000 the GRTP method answered 77 percent of job prediction requests with an error of less than 10 percent. While in the same environment, the most effective and accurate method using workload traces was only able to predict 32 percent of the cases within the same range of error. The GRTP method was able to handle unexpected changes in resources and services which affect the job response time trends and was able to adapt to new scenarios. The tests showed that the proposed GRTP method is capable of predicting job response time requests and it also improves the prediction quality when compared to other current solutions

Parallelization of Littlewood-Richardson Coefficients Computation and its Integration into the BonjourGrid Meta-Desktop Grid Middleware

International audienceThe aim of this paper is to show how to parallelize a compute intensive application in mathematics (Group Theory) for an institutional Desktop Grid platform coordinated by a meta-grid middleware named BonjourGrid. The paper is twofold: first of all, it shows how to parallelize a sequential program for a multicore CPU which participates in the computation and second it demonstrates the effort for launching multiple instances of the solutions for the mathematical problem with the BonjourGrid middleware. BonjourGrid is a fully decentralized Desktop Grid middleware. The main results of the paper are: a) an efficient multi-threaded version of a sequential program to compute Littlewood- Richardson coefficients, namely the Multi-LR program and b) a proof of concept, centered around the user needs, for the BonjourGrid middleware dedicated to coordinate multiple instances of programsfor Desktop Grids and with the help of Multi-LR. In this paper, the scientific work consists in starting from a model for the solution of a compute intensive problem in mathematics, to incorporate the concrete model into a middleware and running it on commodity PCs platform managed by an innovative meta Desktop Grid middleware

Combining Grid and Cloud Resources by Use of Middleware for SPMD Application

International audienceDistributed computing environments have evolved from in-house clusters to Grids and now Cloud platforms. We, as others, provide HPC benchmarks results over Amazon EC2 that show a lower performance of Cloud resources compared to private resources. So, it is not yet clear how much of impact Clouds will have in high performance computing (HPC). But hybrid Grid/Cloud computing may offer opportunities to increase overall applications performance, while benefiting from in-house computational resources extending them by Cloud ones only whenever needed. In this paper, we advocate the usage of ProActive, a well established middleware in the grid community, for mixed Grid/Cloud computing, extended with features to address Grid/Cloud issues with little or no effort for application developers. We also introduce a framework, developed in the context of the DiscoGrid project, based upon the ProActive middleware to couple HPC domain-decomposition SPMD applications in heterogeneous multi-domain environments. Performance results coupling Grid and Cloud resources for the execution of such kind of highly communicating and processing intensive applications have shown an overhead of about 15%, which is a non-negligible value, but lower enough to consider using such environments to achieve a better cost-performance trade-off than using exclusively Cloud resources