APEL (Accounting Processor for Event Logs) gathers CPU accounting information by parsing batch system and blah accounting log files and inserting the data into a local MySQL database. APEL then publishes the data into a centralized repositor

Gordon, John

English

ZENODO

 APEL CLIENT FUNCTIONAL DESCRIPTION    Date: 26.04.2013  EUROPEAN M IDDLEWARE INITIATIVE  APEL  CLIENT FUNCTIONAL DESCRIPTION  Document Version: 2.1 EMI Component Version:  4.1.0 Date: 25.04.2013             This work is co-funded by the European Commission as part of the EMI project under Grant Agreement INFSO-RI-261611.  APEL CLIENT FUNCTIONAL DESCRIPTION Date: 26.04.2013     DOCUMENT CHANGE LOG  Version Date Comment Author 2.1 25/04/2013 Add SLURM parser. Will Rogers 2.0 13/02/2013 Rewritten for new APEL implementation in EMI3. Will Rogers 1.0.1 16/05/2012 Added EMI funding notice, reviewed Will Rogers 1.0.0 07/04/2011 First version Cristina del Cano Novales                      Copyright notice:  Copyright (c) Members of the EMI Collaboration. 2013. See http://www.eu-emi.eu/about/Partners/ for details on the copyright holders. EMI (“European Middleware Initiative”) is a project partially funded by the European Commission.  For more information on the project, its partners and contributors please see http://www.eu-emi.eu.  This document is released under the Open Access license. You are permitted to copy and distribute verbatim copies of this document containing this copyright notice, but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: "Copyright (C) 2013. Members of the EMI Collaboration. http://www.eu-emi.eu ". The information contained in this document represents the views of EMI as of the date they are published. EMI does not guarantee that any information contained herein is error-free, or up to date.  EMI MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, BY PUBLISHING THIS DOCUMENT.  APEL CLIENT FUNCTIONAL DESCRIPTION Date: 26.04.2013    1. INTRODUCTION 1.1. PURPOSE APEL (Accounting Processor for Event Logs) is a tool that gathers CPU accounting information by parsing batch system and blah accounting log files and inserting the data into a local MySQL database. APEL then publishes the data into a centralised repository. 1.2. DEFINITIONS AND ACRONYMS APEL: Accounting Processor for Event Logs CE: Computing Element – the APEL log parser is installed as part of this service. LRMS: Local Resource Management System – batch system used to manage computing jobs. STOMP [1]: Text-based messaging protocol supported by various message brokers. SSM: Secure STOMP Messenger – a simple program used to send files using STOMP.  2. OVERALL DESCRIPTION The APEL client has two components: the APEL parsers and APEL publisher. The APEL parsers interpret log files to extract job information. Specifically, processing the log files produced by a batch system (PBS, LSF, SGE or SLURM) and the blahd/grid-jobmap accounting log files, which are produced by a CE. Extracted data is stored within a local MySQL database on the APEL publisher machine.  ‘APEL Publisher’ is used to describe both the APEL client and APEL SSM. The client makes LDAP queries of the BDII or CE to obtain the CPU performance figures for the worker node clusters.  It then generates accounting records derived from the parsed logging data and stores the records in the local database.  It writes the accounting data to the local filesystem so that it can be sent to the central server by SSM. SSM reads files containing accounting data from the filesystem, and then sends it, using the configured message brokers, to the central APEL repository, where the information is aggregated. 3. ARCHITECTURE            CE/LRMS APEL SSM APEL Parsers APEL Client Local DB  APEL CLIENT FUNCTIONAL DESCRIPTION Date: 26.04.2013    4. REFERENCES [1] http://stomp.codehaus.org/ 

APEL CLIENT FUNCTIONAL DESCRIPTION

Abstract

Similar works

Full text

Available Versions

ZENODO

DataCite MDS

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY