Accounting Facilities in the European Supercomputing Grid DEISA

Account management and resource usage monitoring are essential services for production Grids. The scope of a production Grid infrastructure, the heterogeneity of resources and services, the typical community usage profiles, and the depth of integration of the resource providers regarding operational procedures and policies imply specific requirements for accounting facilities. We present the accounting facilities currently used in production in the Distributed European Infra-structure for the Supercomputing Applications (DEISA). DEISA is a consortium of leading national supercomputing centres currently deploying and operating a persistent, production quality, distributed su-percomputing environment with continental scope. The DEISA accounting facilities gather information from the site-local batch systems and the distributed DEISA user administration system, and generate XML usage records conforming to the OGF usage record specification which are then stored locally in a XML data base at each DEISA site. The distributed accounting information can be fetched by clients such as users, project supervisors, site accounting managers and DEISA supervisors. The information is made available by site-local WSRF-compliant accounting information services that allow for a fine-grained setting of access rights. Each authorized client gets a specific view on the accounting information according to one of the following roles: a) a site accounting manager imports usage records of related home-site users from all DEISA sites for longterm archiving, b) a project supervisor retrieves information to assess the resource usage by his project partners, c) a DEISA supervisor (e.g. someone overlooking the usage on behalf of the DEISA executive committee) gets a report on the global usage of DEISA resources, and d) the user who can retrieve all the accounting information related to his own jobs. The privacy and integrity of the data provided and transferred from the accounting information service running at each site is guaranteed using X.509 certificates for mutual authentication and secure communication channels

Cloud Computing and Grid Computing 360-Degree Compared

Cloud Computing has become another buzzword after Web 2.0. However, there are dozens of different definitions for Cloud Computing and there seems to be no consensus on what a Cloud is. On the other hand, Cloud Computing is not a completely new concept; it has intricate connection to the relatively new but thirteen-year established Grid Computing paradigm, and other relevant technologies such as utility computing, cluster computing, and distributed systems in general. This paper strives to compare and contrast Cloud Computing with Grid Computing from various angles and give insights into the essential characteristics of both.Comment: IEEE Grid Computing Environments (GCE08) 200

Supercomputing futures : the next sharing paradigm for HPC resources : economic model, market analysis and consequences for the Grid

À la croisée des chemins du génie informatique, de la finance et de l'économétrie, cette thèse se veut fondamentalement un exercice en ingénierie économique dont l' objectif est de contribuer un système novateur, durable et adaptatif pour le partage de resources de calcul haute-performance. Empruntant à la finance fondamentale et à l'analyse technique, le modèle proposé construit des ratios et des indices de marché à partir de statistiques transactionnelles. Cette approche, encourageant les comportements stratégiques, pave la voie à une métaphore de partage plus efficace pour la Grid, où l'échange de ressources se voit maintenant pondéré. Le concept de monnaie de Grid, un instrument beaucoup plus liquide et utilisable que le troc de resources comme telles est proposé: les Grid Credits. Bien que les indices proposés ne doivent pas être considérés comme des indicateurs absolus et contraignants, ils permettent néanmoins aux négociants de se faire une idée de la valeur au marché des différentes resources avant de se positionner. Semblable sur de multiples facettes aux bourses de commodités, le Grid Exchange, tel que présenté, permet l'échange de resources via un mécanisme de double-encan. Néanmoins, comme les resources de super-calculateurs n'ont rien de standardisé, la plate-forme permet l'échange d'ensemble de commodités, appelés requirement sets, pour les clients, et component sets, pour les fournisseurs. Formellement, ce modèle économique n'est qu'une autre instance de la théorie des jeux non-coopératifs, qui atteint éventuellement ses points d'équilibre. Suivant les règles du "libre-marché", les utilisateurs sont encouragés à spéculer, achetant, ou vendant, à leur bon vouloir, l'utilisation des différentes composantes de superordinateurs. En fin de compte, ce nouveau paradigme de partage de resources pour la Grid dresse la table à une nouvelle économie et une foule de possibilités. Investissement et positionnement stratégique, courtiers, spéculateurs et même la couverture de risque technologique sont autant d'avenues qui s'ouvrent à l'horizon de la recherche dans le domaine

3rd EGEE User Forum

We have organized this book in a sequence of chapters, each chapter associated with an application or technical theme introduced by an overview of the contents, and a summary of the main conclusions coming from the Forum for the chapter topic. The first chapter gathers all the plenary session keynote addresses, and following this there is a sequence of chapters covering the application flavoured sessions. These are followed by chapters with the flavour of Computer Science and Grid Technology. The final chapter covers the important number of practical demonstrations and posters exhibited at the Forum. Much of the work presented has a direct link to specific areas of Science, and so we have created a Science Index, presented below. In addition, at the end of this book, we provide a complete list of the institutes and countries involved in the User Forum

A service-oriented Grid environment with on-demand QoS support

Grid Computing entstand aus der Vision für eine neuartige Recheninfrastruktur, welche darauf abzielt, Rechenkapazität so einfach wie Elektrizität im Stromnetz (power grid) verfügbar zu machen. Der entsprechende Zugriff auf global verteilte Rechenressourcen versetzt Forscher rund um den Globus in die Lage, neuartige Herausforderungen aus Wissenschaft und Technik in beispiellosem Ausmaß in Angriff zu nehmen. Die rasanten Entwicklungen im Grid Computing begünstigten auch Standardisierungsprozesse in Richtung Harmonisierung durch Service-orientierte Architekturen und die Anwendung kommerzieller Web Services Technologien. In diesem Kontext ist auch die Sicherung von Qualität bzw. entsprechende Vereinbarungen über die Qualität eines Services (QoS) wichtig, da diese vor allem für komplexe Anwendungen aus sensitiven Bereichen, wie der Medizin, unumgänglich sind. Diese Dissertation versucht zur Entwicklung im Grid Computing beizutragen, indem eine Grid Umgebung mit Unterstützung für QoS vorgestellt wird. Die vorgeschlagene Grid Umgebung beinhaltet eine sichere Service-orientierte Infrastruktur, welche auf Web Services Technologien basiert, sowie bedarfsorientiert und automatisiert HPC Anwendungen als Grid Services bereitstellen kann. Die Grid Umgebung zielt auf eine kommerzielle Nutzung ab und unterstützt ein durch den Benutzer initiiertes, fallweises und dynamisches Verhandeln von Serviceverträgen (SLAs). Das Design der QoS Unterstützung ist generisch, jedoch berücksichtigt die Implementierung besonders die Anforderungen von rechenintensiven und zeitkritischen parallelen Anwendungen, bzw. Garantien f¨ur deren Ausführungszeit und Preis. Daher ist die QoS Unterstützung auf Reservierung, anwendungsspezifische Abschätzung und Preisfestsetzung von Ressourcen angewiesen. Eine entsprechende Evaluation demonstriert die Möglichkeiten und das rationale Verhalten der QoS Infrastruktur. Die Grid Infrastruktur und insbesondere die QoS Unterstützung wurde in Forschungs- und Entwicklungsprojekten der EU eingesetzt, welche verschiedene Anwendungen aus dem medizinischen und bio-medizinischen Bereich als Services zur Verfügung stellen. Die EU Projekte GEMSS und Aneurist befassen sich mit fortschrittlichen HPC Anwendungen und global verteilten Daten aus dem Gesundheitsbereich, welche durch Virtualisierungstechniken als Services angeboten werden. Die Benutzung von Gridtechnologie als Basistechnologie im Gesundheitswesen ermöglicht Forschern und Ärzten die Nutzung von Grid Services in deren Arbeitsumfeld, welche letzten Endes zu einer Verbesserung der medizinischen Versorgung führt.Grid computing emerged as a vision for a new computing infrastructure that aims to make computing resources available as easily as electric power through the power grid. Enabling seamless access to globally distributed IT resources allows dispersed users to tackle large-scale problems in science and engineering in unprecedented ways. The rapid development of Grid computing also encouraged standardization, which led to the adoption of a service-oriented paradigm and an increasing use of commercial Web services technologies. Along these lines, service-level agreements and Quality of Service are essential characteristics of the Grid and specifically mandatory for Grid-enabling complex applications from certain domains such as the health sector. This PhD thesis aims to contribute to the development of Grid technologies by proposing a Grid environment with support for Quality of Service. The proposed environment comprises a secure service-oriented Grid infrastructure based on standard Web services technologies which enables the on-demand provision of native HPC applications as Grid services in an automated way and subject to user-defined QoS constraints. The Grid environment adopts a business-oriented approach and supports a client-driven dynamic negotiation of service-level agreements on a case-by-case basis. Although the design of the QoS support is generic, the implementation emphasizes the specific requirements of compute-intensive and time-critical parallel applications, which necessitate on-demand QoS guarantees such as execution time limits and price constraints. Therefore, the QoS infrastructure relies on advance resource reservation, application-specific resource capacity estimation, and resource pricing. An experimental evaluation demonstrates the capabilities and rational behavior of the QoS infrastructure. The presented Grid infrastructure and in particular the QoS support has been successfully applied and demonstrated in EU projects for various applications from the medical and bio-medical domains. The EU projects GEMSS and Aneurist are concerned with advanced e-health applications and globally distributed data sources, which are virtualized by Grid services. Using Grid technology as enabling technology in the health domain allows medical practitioners and researchers to utilize Grid services in their clinical environment which ultimately results in improved healthcare

Evaluation of an Accounting Model for Dynamic Virtual Organizations

Accounting of Grid resource and service usage determines the central support activity for Grid systems to be adopted as a means for service-oriented computing in Dynamic Virtual Organizations (DVO). An all-embracing study of existing Grid accounting systems has revealed that these approaches focus primarily on technical precision, while they lack a foundation of appropriate economic accounting principles and the support for multi-provider scenarios or virtualization concepts. Consequently, a new, flexible, resource-based accounting model for DVOs was developed, combining technical and economic accounting by means of Activity-based Costing. Driven by a functional evaluation, this paper pursues a full-fledged evaluation of the new, generically applicable Grid accounting model. This is done for the specific environment of the Leibniz Supercomputing Centre (LRZ) in Garching, Germany. Thus, a detailed evaluation methodology and evaluation environment is outlined, leading to actual model-based cost calculations for a defined set of considered Grid services. The results gained are analyzed and respective conclusions on model applicability, optimizations, and further extensions are draw

NetJobs: A new approach to network monitoring for the Grid using Grid jobs

With grid computing, the far-fl�ung and disparate IT resources act as a single "virtual datacenter". Grid computing interfaces heterogeneous IT resources so they are available when and where we need them. Grid allows us to provision applications and allocate capacity among research and business groups that are geographically and organizationally dispersed. Building a high availability Grid is hold as the next goal to achieve: protecting against computer failures and site failures to avoid downtime of resource and honor Service Level Agreements. Network monitoring has a key role in this challenge. This work is concerning the design and the prototypal implementation of a new approach to Network monitoring for the Grid based on the usage of Grid scheduled jobs. This work was carried out within the Network Support task (SA2) of the Enabling Grids for E-sciencE (EGEE) project. This thesis is organized as follows: Chapter 1: Grid Computing From the origins of Grid Computing to the latest projects. Conceptual framework and main features characterizing many kind of popular grids will be presented. Chapter 2: The EGEE and EGI projects This chapter describes the Enabling Grids for E-sciencE (EGEE) project and the European Grid Infrastructure (EGI). EGEE project (2004-2010) was the�flagship Grid infrastructure project of the EU. The third and last two-year phase of the project (started on 1 May 2008) was financed with a total budget of around 47 million euro, with a further estimated 50 million euro worth of computing resources contributed by the partners. A total manpower of 9,000 Person Months, of which over 4,500 Person Months has been contributed by the partners from their own funding sources. At its close, EGEE represented a worldwide infrastructure of approximately to 200,000 CPU cores, collaboratively hosted by more than 300 centres around the world. By the end of the project, around 13 million jobs were executed on the EGEE grid each month. The new organization, EGI.eu, has then been created to continue the coordination and evolution of the European Grid Infrastructure (EGI) based on EGEE Grid. Chapter3: gLite Middleware Chapter three gives an overview on the gLite Grid Middleware. gLite is the middleware stack for grid computing used by the EGEE and EGI projects with in a very large variety of scientifi�c domains. Born from the collaborative efforts of more than 80 people in 12 different academic and industrial research centers as part of the EGEE Project, gLite provides a complete set of services for building a production grid infrastructure. gLite provides a framework for building grid applications tapping into the power of distributed computing and storage resources across the Internet. The gLite services are currently adopted by more than 250 Computing Centres and used by more than 15000 researchers in Europe and around the world. Chapter 4: Network Activity in EGEE/EGI Grid infrastructures are distributed by nature, involving many sites, normally in different administrative domains. Individual sites are connected together by a network, which is therefore a critical part of the whole Grid infrastructure; without the network there is no Grid. Monitoring is a key component for the successful operation of any infrastructure, helping in the discovery and diagnosis of any problem which may arise. Network monitoring is able to contribute to the day-to-day operations of the Grid by helping to provide answers to specific questions from users and site administrators. This chapter will discuss all the effort lavished by EGEE and EGI in the Grid Network domain. Chapter 5: Grid Network Monitoring based on Grid Jobs Net Jobs is a prototype of a light weight solution for the Grid network monitoring. A job-based approach has been used in order to prove the feasibility of this non intrusive solution. It is currently configured to monitor eight production sites spread from Italy to France but this method could be applied to the vast majority of Grid sites. The prototype provides coherent RTT, MTU, number of hops and TCP achievable bandwidth tests

Tracing Resource Usage over Heterogeneous Grid Platforms: A Prototype RUS Interface for DGAS

Tracing resource usage by Grid users is of utmost importance especially in the context of large-scale scientific collaborations such as within the High Energy Physics (HEP) community to guarantee fairness of resource sharing, but many difficulties can arise when tracing the resource usage of distributed applications over heterogeneous Grid platforms. These difficulties are often related to a lack of interoperability of the accounting components across middlewares. This paper brie y describes the architecture and workflow of the Distributed Grid Accounting System (DGAS) [1] and evaluates the possibility to extend it with a Resource Usage Service (RUS) [2, 3] interface according to the Open Grid Forum (OGF) sped cation that allows to store and retrieve OGF Usage Records (URs) [4, 5] via Web Services. In this context the OGF RUS and UR sped cations are critically analyzed. Furthermore, a prototype of a RUS interface for DGAS (DGAS-RUS) is presented and the most recent test results towards a full interoperability between heterogeneous Grid platforms are outlined