XtremWeb : une plate-forme de recherche sur le calcul global et pair à pair

ISBN 274620472XLa connection à grande échelle des ordinateurs aux réseaux de communication et les mécanismes qui permettent leur interopérabilité (standardisation des protocoles, homogénéisation des formats d'échange, systèmes de nomage efficace, sécurisation des sites) motivent des tentatives de globalisation des ressources et des données à partir de systèmes de Calcul Global, Pair à Pair ou de GRID. Ces systèmes sont les prémisses de l'émergence de systèmes d'exploitation à grande échelle. Dans cet article nous proposons d'abord de situer les systémes Pair à Pair par rapport aux autres structures de GRID en soulignant leur différences qui ont principalement une origine historique. Dans une deuxième partie nous présentons une ratio- nalisation des systémes de Calcul Global et Pair à Pair. Nous détaillons ensuite la plate-forme XtremWeb qui constitue l'une des premières tentatives de généralisation du concept de parité entre les ressources appliqué à la globalisation du processeur, de la mémoire et du stockage. Nous terminons par la présentation de quelques points durs en recherche : la sécurisation des participants et la communication inter-noeud

Use of A Network Enabled Server System for a Sparse Linear Algebra Grid Application

Solving systems of linear equations is one of the key operations in linear algebra. Many different algorithms are available in that purpose. These algorithms require a very accurate tuning to minimise runtime and memory consumption. The TLSE project provides, on one hand, a scenario-driven expert site to help users choose the right algorithm according to their problem and tune accurately this algorithm, and, on the other hand, a test-bed for experts in order to compare algorithms and define scenarios for the expert site. Both features require to run the available solvers a large number of times with many different values for the control parameters (and maybe with many different architectures). Currently, only the grid can provide enough computing power for this kind of application. The DIET middleware is the GRID backbone for TLSE. It manages the solver services and their scheduling in a scalable way.La résolution de systèmes linéaires creux est une opération clé en algèbre linéaire. Beaucoup d’algorithmes sont utilisés pour cela, qui dépendent de nombreux paramètres, afin d’offrir une robustesse, une performance et une consommation mémoire optimales. Le projet GRID-TLSE fournit d’une part, un site d’expertise basé sur l’utilisation de scénarios pour aider les utilisateurs à choisir l’algorithme qui convient le mieux à leur problème ainsi que les paramètres associés; et d’autre part, un environnement pour les experts du domaine leur permettant de comparer efficacement des algorithmes et de définir dynamiquement de nouveaux scénarios d’utilisation. Ces fonctionnalités nécessitent de pouvoir exécuter les logiciels de résolution disponibles un grand nombre de fois,avec beaucoup de valeurs différentes des paramètres de contrôle (et éventuellement sur plusieurs architectures de machines). Actuellement, seule la grille peut fournir la puissance de calcul pour ce type d’applications. L’intergiciel DIETest utilisé pour gérer la grille, les différents services, et leur ordonnancement efficace

The International Grid (iGrid): Empowering Global Research Community Networking Using High Performance International Internet Services

The Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago and Indiana University collaborated on a major research demonstration at the IEEE/ACM Supercomputing '98 (SC'98) conference in Orlando, Florida, November 7-13, 1998, to showcase the evolution and importance of global research community networking. Collaborators worked together to solve complex computational problems using advanced high-speed networks to access geographically-distributed computing, storage, and display resources. It is the collection of computing and communication resources that we refer to as the International Grid (iGrid). This paper presents an overview of the grid testbed, some of the underlying technologies used to enable distributed computing and collaborative problem solving, and descriptions of the applications. It concludes with recommendations for the future of global research community networking, based on the experiences of iGrid participants from the USA, Australia, Canada, Germany, Japan, The Netherlands, Russia, Switzerland, Singapore, and Taiwan

Volunteer computing

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001.Includes bibliographical references (p. 205-216).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.This thesis presents the idea of volunteer computing, which allows high-performance parallel computing networks to be formed easily, quickly, and inexpensively by enabling ordinary Internet users to share their computers' idle processing power without needing expert help. In recent years, projects such as SETI@home have demonstrated the great potential power of volunteer computing. In this thesis, we identify volunteer computing's further potentials, and show how these can be achieved. We present the Bayanihan system for web-based volunteer computing. Using Java applets, Bayanihan enables users to volunteer their computers by simply visiting a web page. This makes it possible to set up parallel computing networks in a matter of minutes compared to the hours, days, or weeks required by traditional NOW and metacomputing systems. At the same time, Bayanihan provides a flexible object-oriented software framework that makes it easy for programmers to write various applications, and for researchers to address issues such as adaptive parallelism, fault-tolerance, and scalability. Using Bayanihan, we develop a general-purpose runtime system and APIs, and show how volunteer computing's usefulness extends beyond solving esoteric mathematical problems to other, more practical, master-worker applications such as image rendering, distributed web-crawling, genetic algorithms, parametric analysis, and Monte Carlo simulations. By presenting a new API using the bulk synchronous parallel (BSP) model, we further show that contrary to popular belief and practice, volunteer computing need not be limited to master-worker applications, but can be used for coarse-grain message-passing programs as well. Finally, we address the new problem of maintaining reliability in the presence of malicious volunteers. We present and analyze traditional techniques such as voting, and new ones such as spot-checking, encrypted computation, and periodic obfuscation. Then, we show how these can be integrated in a new idea called credibility-based fault-tolerance, which uses probability estimates to limit and direct the use of redundancy. We validate this new idea with parallel Monte Carlo simulations, and show how it can achieve error rates several orders-of-magnitude smaller than traditional voting for the same slowdown.by Luis F.G. Sarmenta.Ph.D

Analysis Of Aircraft Arrival Delay And Airport On-time Performance

While existing grid environments cater to specific needs of a particular user community, we need to go beyond them and consider general-purpose large-scale distributed systems consisting of large collections of heterogeneous computers and communication systems shared by a large user population with very diverse requirements. Coordination, matchmaking, and resource allocation are among the essential functions of large-scale distributed systems. Although deterministic approaches for coordination, matchmaking, and resource allocation have been well studied, they are not suitable for large-scale distributed systems due to the large-scale, the autonomy, and the dynamics of the systems. We have to seek for nondeterministic solutions for large-scale distributed systems. In this dissertation we describe our work on a coordination service, a matchmaking service, and a macro-economic resource allocation model for large-scale distributed systems. The coordination service coordinates the execution of complex tasks in a dynamic environment, the matchmaking service supports finding the appropriate resources for users, and the macro-economic resource allocation model allows a broker to mediate resource providers who want to maximize their revenues and resource consumers who want to get the best resources at the lowest possible price, with some global objectives, e.g., to maximize the resource utilization of the system

Platform as a service integration for scientific computing using DIRAC

Cada día crece máis a demanda de recursos de computación requirida polos investigadores, capacidades de cálculo que coexisten co crecente volume de datos xerado actualmente. Estes investigadores están a demandar un servizo de Computación de Altas Prestacións (HPC) que permita a execución das suas simulacións dunha forma na que se deslocalicen os recursos para poder acceder aos máximos posibles, facilitandoo coa forma o máis cómoda e segura para eles. Doutra banda, as universidades están conectadas con centros de investigación con redes que pusuen unha velocidade e fiabilidade que posibilitan a execución de traballos de cálculo científico. As capacidades de computo existentes en universidades van dende aulas informáticas para usos docentes, laboratorios, etc., ata clusters de ordenadores pertencentes a grupos de investigación. Usando tecnoloxías grid e cloud estes recursos computacionais heteroxéneos poderían ser reutilizados polos investigadores para realizar simulacións, aportando unha maior cantidade de cómputo a xa existente e deslocalizando os recursos entre distintos lugares ao redor do planeta. O obxectivo desta tese é adaptar a contorna para computación distribuída DIRAC, desenvolvida para o proxecto LHCb do CERN, para o seu uso por varias comunidades de usuarios baseado nas tecnoloxías cloud e big data. Esta contorna pusuiría repositorios de software centralizados que permitan proveer o software necesario para que a través dos entornos na nube se poidan executar as aplicacións dos investigadores en calquera parte do planeta dunha forma escalable, permitindo aprobeitar tanto recursos dedicados como nondedicados. Avaliando así a execución desta plataforma para a realización de cálculos científicos. Este traballo comezará coa obtención de requisitos, para pasar despois ao proceso de integración básica. Posteriormente, optimizarase o uso do software cientifico empregado para as contornas cloud, tratando de adaptalo aos entornos virtualizados. Para iso, será necesario realizar un estudo estadístico que sexa o máis próximo posible aos entornos en producción para poder determinar e crear as infraestructuras adaptadas evitando así a perda de rendemento dentro de recursos. O seguinte caso sería utilizar as tecnoloxías virtualizadas, adaptando as arquitecturas creadas, para a creación de sistemas que permitan o envío de traballos que requiran de grandes cantidades de datos no eido do big data dunha forma distribuida

A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.Comment: 29 pages, 15 figure

Estudo sobre processamento maciçamente paralelo na internet

Orientador: Marco Aurélio Amaral HenriquesTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de ComputaçãoResumo: Este trabalho estuda a possibilidade de aproveitar o poder de processamento agregado dos computadores conectados pela Internet para resolver problemas de grande porte. O trabalho apresenta um estudo do problema tanto do ponto de vista teórico quanto prático. Desde o ponto de vista teórico estudam-se as características das aplicações paralelas que podem tirar proveito de um ambiente computacional com um grande número de computadores heterogêneos fracamente acoplados. Desde o ponto de vista prático estudam-se os problemas fundamentais a serem resolvidos para se construir um computador paralelo virtual com estas características e propõem-se soluções para alguns dos mais importantes como balanceamento de carga e tolerância a falhas. Os resultados obtidos indicam que é possível construir um computador paralelo virtual robusto, escalável e tolerante a falhas e obter bons resultados na execução de aplicações com alta razão computação/comunicaçãoAbstract: This thesis explores the possibility of using the aggregated processing power of computers connected by the Internet to solve large problems. The issue is studied both from the theoretical and practical point of views. From the theoretical perspective this work studies the characteristics that parallel applications should have to be able to exploit an environment with a large, weakly connected set of computers. From the practical perspective the thesis indicates the fundamental problems to be solved in order to construct a large parallel virtual computer, and proposes solutions to some of the most important of them, such as load balancing and fault tolerance. The results obtained so far indicate that it is possible to construct a robust, scalable and fault tolerant parallel virtual computer and use it to execute applications with high computing/communication ratioDoutoradoEngenharia de ComputaçãoDoutor em Engenharia Elétric

Using machine learning to measure the cross section of top quark pairs in the muon+jets channel at the Compact Muon Solenoid

The cross section for pp to top-antitop production at a center of mass energy of 7 TeV is measured using a data sample with integrated luminosity 36.1 inverse pb collected by the CMS detector at the LHC. The analysis is performed on a computing grid. Events with an isolated muon and three hadronic jets are analyzed using a multivariate machine learning algorithm. Kinematic variables and b tags are provided as input to the algorithm; output from the algorithm is used in a maximum likelihood fit to determine top-antitop event yield. The measured cross section is 151 +/- 15(stat.) +35/-28(syst.) +/- 6(lumi.) pb