GRID-launcher-1.0 was built within the VO-Tech framework, as a software interface between the UK-ASTROGRID and a generic GRID infrastructures in order to allow any ASTROGRID user to launch on the GRID computing intensive tasks from the ASTROGRID Workbench or Desktop. Even though of general application, so far the Grid-Launcher has been tested on a few selected softwares (VONeural-MLP, VONeural-SVM, Sextractor and SWARP) and on the SCOPE-GRID

Brescia M

Cavuoti S

d'Angelo G

Deniskina N

Laurino O

Longo G

English

Archivio della ricerca - Università degli studi di Napoli Federico II

Mem. S.A.It. Vol. 80, 571c© SAIt 2009 Memorie dellaGRID-Launcher v.1.0N. Deniskina1, M. Brescia2, S. Cavuoti1, G. d’Angelo1, O. Laurino1,G. Longo1,21 Department of Physical Sciences University Federico II, Naples, via Cinthia 6 80131Naples, Italy, e-mail: natalia.deniskina@unina.it2 INAF - Osservatorio Astronomico di Capodimonte, via Moiariello 16, 80131, Napoli,ItalyAbstract. GRID-launcher-1.0 was built within the VO-Tech framework, as a software in-terface between the UK-ASTROGRID and a generic GRID infrastructures in order to al-low any ASTROGRID user to launch on the GRID computing intensive tasks from theASTROGRID Workbench or Desktop. Even though of general application, so far the Grid-Launcher has been tested on a few selected softwares (VONeural-MLP, VONeural-SVM,Sextractor and SWARP) and on the SCOPE-GRID.Key words. distributed computing; virtual observatory1. IntroductionThe main goal of the International VirtualObservatory URL.2 (2000) infrastructure isto provide the community at large with aneasy and user friendly access to astronomi-cal data, software tools, and computing fa-cilities. While the first two parts of the task,namely the federation and fusion of heteroge-neous data archives and the implementation offlexible data reduction and data analysis toolshave been widely addressed and, at least intheir fundamental aspects, solved, the possibil-ity to access large distributed computing facil-ities to perform computing intensive tasks hasnot yet been satisfactorily answered. The mainreasons for this delay being mainly two: theneed to match the GRID security requirementsand the lack of homogeneity in the definitionsof the storage space. The first issue can be eas-Send offprint requests to: N. Deniskinaily explained: most users of a specific VirtualOrganization (VO) do not possess the personalcertificates which are requested to access theGRID or, even when they do have a personalcertificate, the computing GRID which theyneed does not recognize their own certificationauthority. The second issue arises instead fromthe fact that the data usually reside locally orin a remote storage space which is not seen bythe selected GRID as a storage element (SE).The lack of an user friendly access to theGRID is a main obstacles against the use ofcomputing intensive data mining methods andtools such as, for instance, Support VectorMachines Cavuoti (2008) or ProbabilisticPrincipal Surfaces D’Abrusco et al. (2008)on massive data sets (MDS). Therefore, in theframework of the VONeural project, whichaims at implementing a package of data min-ing routines capable to work in a distributedcomputing environment and on large data setsof high dimensionality, we have implemented572 N. Deniskina et al.: GRID-Launcher v.1.0and tested a general purpose interface betweenthe UK-ASTROGRID URL.1 (2000) and theGRID which solves part of the above quotedproblems. We wish to stress that the followedapproach is quite general and that GRID-Launcher can be easily adapted to other VirtualOrganizations and to any GRID. For testing weused the GRID-SCOPE URL.3 (2000) whichis part of the recently funded GRID infrastruc-ture for SOuthern Italy.2. GRID-launcherAs already mentioned, an ASTROGRID useris recognized by the UAS (User AuthenticationService) of ASTROGRID but, often does notpossess a personal GRID certificate. As it isthe case for most virtual organizations, theASTROGRID and GRID authentication pro-cedures are incompatible due to a number offactors: a) different CAs (certification author-ity); b) the lack of compatibility between theASTROGRID and GRID certificates and PKIpolicy. The EGEE-II boards are currently dis-cussing how to allow the use of specific ser-vices by means of application certificates (or”robot” certificates) but, at the time GRID-Launcher 1.0 was developed this option hadnot yet be introduced and the first version dis-cussed here needs therefore to be regarded asa test version of a more flexible tool (GRID-Launcher 2.0) which will be made available assoon as the new standards will be introduced.In other words, the version 1.0 relies ona personal GRID certificate (signed by theINFN-GRID CA) which is recognized by theSCOPE-GRID used for testing. In the figures1-3, we give a schematic representation of theway GRID-Launcher works: (Fig.1)1. Grid-launcher:– takes the user input from theUser Interface of ASTROGRID(Workbench);– collects all files, tabs and programsneeded;– generates automatically threescripts:task.sh, task.jdl andwn runner.sh to be executed onthe GRID;Fig. 1. Workflow. UI: user interface; RB: resourcebroker; SE: storage element; CE: computing ele-ment; WN: working node.Fig. 2. The workflow from ASTROGRID to theWN of the GRID. The task.sh, wn runner.sh andtask.jdl - are automatically generated by the GRIDlauncher.– wraps them in an archive and sends itto the Scope-GRID UI. (The authenti-cation on the GRID is done by publickeys exchange).2. The GRID User Interface (UI) receivesthe data and the JDL program, unpacksthem, copies the data to the SE, copieswn runner.sh to the WN’s, starts task.shand task.jdl. (Fig. 2)3. wn runner.sh starts on the WNs, takes thedata from SE, starts the executable of theapplication, and puts the results on theSE. The GRID generates automatically twooutput files task.err and task.out and sendsthem to the UI using the Output SandBox.N. Deniskina et al.: GRID-Launcher v.1.0 573Fig. 3. The workflow from the WN toASTROGRID. The task.err and task.out filesare automatically generated by the GRID.4. GRID-launcher periodically checks thestatus of job and when it ends, it movesthe results from the UI to the ASTROGRIDmachine. (Fig. 3)5. GRID-launcher receives the data archive,unpacks it and puts the results into the stor-age of ASTROGRID (VO-Space).SO far, GRID-Launcher has been imple-mented and tested on:– VO-Neural-MLP: a Multi LayerPerceptron Neural Network used forsupervised clustering and classificationSkordovski (2008); Brescia et al. (2008);– VO-Neural-SVM: Support VectorMachines for supervised clusteringCavuoti (2008); Brescia et al. (2008);– Sextractor: for the extraction of object-catalogues from astronomical imagesBertin & Arnouts, (1996),– SWarp: an application that resamples andcoadds FITS images using any arbitrary as-trometric projection defined in the WCSstandardURL.4 (2000).3. Future developmentsIn the version 2.0, we plan:– to insert a ”switch” which will checkwhether the user has a GRID certificate(and in this case the task will start on theGRID of the user), or has not (in this casethe program will be launched as a serviceon the SCOPE-GRID);– to render the program capable to launchany program (and not only applicationsregistered in the ASTROGRID) matchingsome trivial specifications;– to offer the possibility to use GRID-Launcher as a subtask inside theASTROGRID workflows and, finally,– to unify the VO Space with the SE of theGRID.Additional information about ”GRID-launcher” can be found at URL.5 (2000).ReferencesBertin E. & Arnouts S., 1996, A& AS, 117,393Brescia M., et al., 2008, these proceedings.S. Cavuoti, Laurea Thesis, 2008, UniversityFederico II in NapoliD’Abrusco R., Longo G., Walton N., 2008,astro-ph/0805.0156v1B. Skordovski, Laurea Thesis, 2008,University Federico II in NapoliURL.1: http://www.astrogrid.uk/URL.2: http://www.ivoa.org/URL.3: http://scope.unina.it/URL.4: SWarp User manual , E. Bertinat: http://terapix.iap.fr/rubrique.php?id rubrique=49URL.5: http://voneural.na.infn.it/

GRID-Launcher v.1.0

https://www.iris.unina.it/bitstream/11588/900815/1/42-Deniskina-571.pdf

GRID-Launcher v.1.0

Abstract

Similar works

Full text

Available Versions

Archivio della ricerca - Università degli studi di Napoli Federico II