APEL (Accounting Processor for Event Logs) gathers CPU accounting information by parsing batch system and blah accounting log files and inserting the data into a local MySQL database. APEL then publishes the data into a centralized repositor

Gordon, John

English

ZENODO

 APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE    Date: 26.04.2013  EUROPEAN  M IDDLEWARE INITIATIVE   APEL  PUBLISHER SYSTEM ADMINISTRATOR GUIDE  Document Version: 2.1 EMI Component Version:  4.1.0 Date: 15.04.2013                   This work is co-funded by the EC EMI project under the FP7 Collaborative Projects Grant Agreement Nr. INFSO-RI-261611.     APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013   DOCUMENT CHANGE LOG  Version Date Comment Author 2.1 15/04/2013 Correct database permissions. Will Rogers 2.0 13/02/2013 Rewritten for new APEL implementation in EMI3. Will Rogers 1.0.2 16/05/2012 Added hardware requirements and funding notice. Will Rogers 1.0.1 12/05/2011 Updated Added EMI 1 repository URL Updated YAIM configuration instructions Cristina del Cano Novales 1.0.0 07/04/2011 First version Cristina del Cano Novales                  Copyright notice:  Copyright (c) Members of the EMI Collaboration. 2013. See http://www.eu-emi.eu/about/Partners/ for details on the copyright holders. EMI (“European Middleware Initiative”) is a project partially funded by the European Commission.  For more information on the project, its partners and contributors please see http://www.eu-emi.eu.  This document is released under the Open Access license. You are permitted to copy and distribute verbatim copies of this document containing this copyright notice, but modifying this document is not allowed. You are permitted to copy this document in whole or in part into other documents if you attach the following reference to the copied elements: "Copyright (C) 2013. Members of the EMI Collaboration. http://www.eu-emi.eu ". The information contained in this document represents the views of EMI as of the date they are published. EMI does not guarantee that any information contained herein is error-free, or up to date.  EMI MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, BY PUBLISHING THIS DOCUMENT.    APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013 Contents 1. INTRODUCTION ......................................................................................................................................... 4 2. REQUIREMENTS ........................................................................................................................................ 4 2.1. HARDWARE REQUIREMENTS .................................................................................................................... 4 2.2. SOFTWARE REQUIREMENTS ...................................................................................................................... 4 3. INSTALLATION INSTRUCTIONS ........................................................................................................... 4 3.1. MYSQL-SERVER ........................................................................................................................................ 4 3.2. CA-POLICY-EGI-CORE ............................................................................................................................... 5 3.3. APEL PUBLISHER PACKAGES................................................................................................................... 5 3.4. INSTALL THE APEL CLIENT DATABASE .................................................................................................. 5 4. CONFIGURATION INSTRUCTIONS ....................................................................................................... 5 4.1. CLIENT CONFIGURATION FILE ................................................................................................................. 5 4.1.1 db section ..................................................................................................................................... 5 4.1.2 spec_updater section .................................................................................................................... 5 4.1.3 joiner section................................................................................................................................ 5 4.1.4 unloader section ........................................................................................................................... 5 4.1.5 ssm section ................................................................................................................................... 6 4.1.6 logging section ............................................................................................................................. 6 4.2. SSM CONFIGURATION FILE ..................................................................................................................... 6 4.3. CRON JOB ................................................................................................................................................ 6 5. RUNNING THE PUBLISHER ..................................................................................................................... 7     APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013  1. INTRODUCTION If you are upgrading APEL from a previous installation, please read the separate document ‘APEL Client Upgrade Plan’ first.  The APEL Publisher includes the APEL client package, which is used for collecting a site’s accounting information before the APEL SSM sends it to the central APEL server.  It works in conjunction with the APEL parsers. These instructions describe how to install and configure the APEL Publisher packages. 2. REQUIREMENTS 2.1. HARDWARE REQUIREMENTS The following minimum specifications are recommended.    x86_64 CPU  1GB RAM  20GB disk space 2.2. SOFTWARE REQUIREMENTS The APEL Publisher is currently supported for 64-bit SL5 and SL6.  The EPEL repository must be enabled. The following packages must be installed before the APEL packages:  mysql-server  ca-policy-egi-core  A host certificate and key in PEM format should be installed in the following locations:  /etc/grid-security/hostkey.pem   /etc/grid-security/hostcert.pem  3. INSTALLATION INSTRUCTIONS Before starting, run yum update  3.1. MYSQL-SERVER yum install mysql-server service mysqld start chkconfig --level 345 mysqld on mysqladmin –u root password '<new-password>'     APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013 3.2. CA-POLICY-EGI-CORE Follow the instructions on this page to install the EGI-approved certificates: https://wiki.egi.eu/wiki/EGI_IGTF_Release  3.3. APEL PUBLISHER PACKAGES yum install apel-ssm yum install apel-lib yum install apel-client 3.4. INSTALL THE APEL CLIENT DATABASE The APEL client database schema is located at /usr/share/apel/client.sql. mysql -p –e "create database apelclient" mysql -p apelclient < /usr/share/apel/client.sql 3.4.1 Grant Permissions to the Database  mysql> GRANT ALL PRIVILEGES ON apelclient.* TO 'apel'@'localhost' IDENTIFIED BY '<apel-password>';  For each machine on which APEL Parsers are installed: mysql> GRANT ALL PRIVILEGES ON apelclient.* TO 'apel'@'<FQDN>' IDENTIFIED BY '<apel-password>';   4. CONFIGURATION INSTRUCTIONS The following instructions are for a standard configuration. 4.1. CLIENT CONFIGURATION FILE The APEL configuration file is located at /etc/apel/client.cfg. 4.1.1 db section Provide the hostname, port, database name, username and password for the APEL client database. 4.1.2 spec_updater section Ensure that you put the same site name as is found in GOCDB.  This must be the same as the site name in the BDII.  It is essential that all APEL parsers are configured with the same site name.  You may specify any top-bdii configuration. 4.1.3 joiner section Leave enabled as true.  If you want to send information about local jobs, set local_jobs to true. 4.1.4 unloader section The default configuration sends individual job records.  Sending summaries of the data only will reduce network load, and will not lose any information.  To do this: send_summaries = true     APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013 Leave send_ur as false. If you want to send only certain VOs, list them after include_vos. If you want to exclude certain VOs, list them after exclude_vos. 4.1.5 ssm section Leave this section as enabled.  To configure SSM, see section 4.2. 4.1.6 logging section You may change the logging configuration here.  4.2. SSM CONFIGURATION FILE The APEL publisher uses SSM to send the data to the central APEL server.  The SSM configuration file is located at /etc/apel/sender.cfg.  To send data to the test server:  ... [broker] ... network: TEST-NWOB ... [messaging] ... destination: /queue/global.accounting.cputest.CENTRAL ...  To send data to the production server: ... [broker] ... network: PROD ... use_ssl: true ... [messaging] ... destination: /queue/global.accounting.cpu.central ...  The file should otherwise stay as it is.  4.3. CRON JOB The APEL publisher should run once per day, from a cron.  Here is an example: # Run APEL client once daily mm hh * * * root /usr/bin/apelclient  Do not schedule the parser cron jobs and the client cron job at the same time.    APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE   Date: 26.04.2013  5. RUNNING THE PUBLISHER To run the publisher using the default configuration, run: # /usr/bin/apelclient  

APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE

https://zenodo.org/record/6803/files/APEL_Publisher_System_Administrator_Guide.pdf

APEL PUBLISHER SYSTEM ADMINISTRATOR GUIDE

Abstract

Similar works

Full text

Available Versions

ZENODO