Dvorák,  F.

Filipovic,  J.

Kouril,  D.

Krenek,  A.

Matyska,  L.

Salvet,  Z.

Sitera,  J.

Sustr, Z.

Vocu,  M.

English

ZENODO

Full History of Finished Jobs over MessagingŠustr, Z.; Sitera, J.; Krˇenek, A.; Dvorˇák, F.; Matyska, L.; Vocu˚, M.; Kourˇil, D. Salvet, Z.; Filipovicˇ, J.;CESNETIn its primary role the L&B server receives events from grid com-ponents and computes job states, presenting the processed dataon its output to save others the need to interpret the informationthemselves. There are, however, user groups that actually re-quire the raw information. Chief among them are dashboardsand computer scientists focusing on grid dynamics. The JobHistory capability is intended for them.ImplementationFrom a technical standpoint, job history is an extension of L&B no-tifications. Contrary to standard messages, which contain only theprocessed job status, job history also packs all raw events using theJSON format. It is typically sent out when a job reaches terminal stateor gets purged after prolonged inactivity. Notification messages canbe dispatched over the Messaging infrastructure, or through L&B’slegacy delivery chain. Detailed job history records allow grid trafficanalysts to reconstruct the goings-on in the grid more precisely, andprovide the community with a better picture of life within the grid.Highlights• Very detailed information on job history withoutaccessing/parsing the L&B database• Following contemporary trends, leveraging the commonmessaging infrastructure [3]• Extending the current functionality of L&B notifications• Working jointly with specific user groups to come up with asolution matching their needs.ExampleRegister for notifications containing a full job history (-H) on yourown jobs (-O) to be sent out when a job reaches terminal state.glite-lb-notify new -O -H -a x-msg://<topic>Then use your favourite messaging client to listen to <topic>.L&B NotificationsL&B notifications are a well-established tool used by user groups tomonitor the progress of their jobs. In the original scenario, notifica-tions are sent out any time the L&B server receives an event match-ing pre-set criteria, saving the necessity to poll the server periodicallyfor job status updates. The filters have also been extended so thatonly events resulting in a job state change can be reported to usersthrough notification messages. As of L&B version 3.0 (introduced withthe EMI-1 release), OpenWire/STOMP-based Messaging has beenadopted as an alternative path to deliver L&B’s notification messagesalongside the legacy delivery path [2].References[1] The Logging and Bookkeeping Subsystem,http://egee.cesnet.cz/cs/JRA1/LB/[2] L&B with Messaging, http://egee.cesnet.cz/cvsweb/LB/UF11-messaging.pdf[3] EMI Messaging, https://twiki.cern.ch/twiki/bin/view/EMI/EMIMessagingFuture and Open IssuesThe Job History feature will be made generally available with the in-troduction of L&B 3.1 (EMI 2 Matterhorn).Anonymization L&B already has a solution to white-out personal orotherwise sensitive information from its output feed. That feature willbe tuned to preserve important information while satisfying privacyconcerns.Authorization Since Messaging services in their essential form allowunauthorized access, it may be difficult to achieve authorization onthe message passing layer. The possibility of employing encryptionto protect sensitive information may be explored in the future. L&B’slegacy delivery mechanism can also be used as an alternative solu-tion since it provides full PKI-based authorization.On top of that, ActiveMQ (EMI’s selected messaging solution [3]) al-lows fine-grain (i.e. per destination) authorization. Although it is notbeing used at the moment, it is a feature that may be capitalized uponin the future.Performance testing A real-world comparison of Messaging vs.L&B’s legacy delivery mechanism will be done once the feature isdeployed.Contact: L&B Product TeamE-mail: emi-lb@metacentrum.cz

Full History of Finished Jobs over Messaging

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY

https://zenodo.org/record/6652/files/job_history.pdf

Full History of Finished Jobs over Messaging

Abstract

Similar works

Full text

Available Versions

ZENODO

NEUROSURGERY ENTHUSIASTIC WOMEN SOCIETY