Also in the case of astrophysics, the capability of performing “Big Science” requires the availability of large HPC facilities. But computational resources alone are far from being enough for the community: as a matter of fact,

the whole set of e-infrastructures (network, computing nodes, data repositories, applications) need to work in an interoperable way. This implies the development of common (or at least compatible) user interfaces to computing resources, transparent access to observations and numerical simulations through the Virtual Observatory, integrated data processing pipelines, data mining and semantic web applications. Achieving this interoperability goal is a must to build a real “Knowledge Infrastructure” in the astrophysical domain

Pasian, F.

English

Scientific Open-access Literature Archive and Repository

DOI 10.1393/ncc/i2009-10372-1Colloquia: CSFI 2008IL NUOVO CIMENTO Vol. 32 C, N. 2 Marzo-Aprile 2009HPC, grid and data infrastructures for astrophysics:An integrated viewF. PasianINAF - Information Systems Unit and Osservatorio Astronomico di TriesteVia G. B. Tiepolo 11, 34131 Trieste, Italy(ricevuto il 15 Maggio 2009; pubblicato online il 6 Agosto 2009)Summary. — Also in the case of astrophysics, the capability of performing “BigScience” requires the availability of large HPC facilities. But computational re-sources alone are far from being enough for the community: as a matter of fact,the whole set of e-infrastructures (network, computing nodes, data repositories, ap-plications) need to work in an interoperable way. This implies the development ofcommon (or at least compatible) user interfaces to computing resources, transparentaccess to observations and numerical simulations through the Virtual Observatory,integrated data processing pipelines, data mining and semantic web applications.Achieving this interoperability goal is a must to build a real “Knowledge Infrastruc-ture” in the astrophysical domain.PACS 95.75.Mn – Image processing (including source extraction).PACS 95.75.Pq – Mathematical procedures and computer techniques.PACS 95.80.+p – Astronomical catalogs, atlases, sky surveys, databases, retrievalsystems, archives, etc.1. – Recognition of the importance of e-infrastructures at European levelIt is to be noted that e-infrastructures are absolutely necessary for the developmentof science, but they do not come for free, and cannot be given for granted. This is wellunderstood at the European level, and e-infrastructures are an integral part of EU’spolicies.ESFRI, which is a multi-disciplinary initiative for the implementation of researchinfrastructures, has focussed on the need for networking, capability and throughputcomputing, grid architectures, software, data management and curation as the mainpriorities.ASTRONET is another initiative, dedicated to Astronomy and Astrophysics, aimedat defining the scientific priorities in our discipline for the next two decades, andthe corresponding priorities in the allocation of resources. The ASTRONET workingc© Societa` Italiana di Fisica 2930 F. PASIANgroups have recognized e-infrastructures as must-haves to tackle the challenges of the fu-ture: computing (both capacity and capability), theory and simulations and the VirtualObservatory [1], together with laboratories, must have priority over the rest, becausewithout them science cannot be made.2. – Requirements for e-infrastructures in AstrophysicsAstrophysics, considering it to be the sum of a set of branches (astronomy, radio-astronomy, cosmology, space physics, planetology, solar-terrestrial physics, etc.) is mainlyan observational science. The progress in sensor technology has led to an exponentialgrowth in the data volume. The first reason to need computing power is therefore toprocess (i.e. reduce) observational data.All observations are kept; as a matter of fact, each observation is unique since many ofthe astrophysical phenomena are time-dependent. As a consequence, there is a continuousincrease of the data available. This implies the need for publicly accessible archives.Modern astrophysics is increasingly interested in multi-wavelength studies, and in an-alyzing time variability, studying supernovae progenitors, etc. There is therefore the needto harmonize the data archives world wide, creating what is known as the internationalVirtual Observatory.An additional reason calling for an increasingly higher computing power is the desireto perform data mining and statistical analysis on archived data.Astrophysical phenomena are usually modelled to be better understood. Computingpower is therefore needed to perform numerical simulations.It is then necessary to compare the data obtained from an observation, those retrievedfrom archives and the simulations. Matching observed data with numerical simulationsis an additional reason why computing power is necessary.3. – e-infrastructures for Astrophysics, today and tomorrowCurrently, astrophysicists use a wide range of facilities: PCs, local clusters, advancedcomputing (Grid, HPC centers), exploiting their computing and storage capacities. Plus,of course, there is a heavy usage of the research network. Astronomical institutions(INAF and Universities) participate actively in the national and international initiativesin the field.With reference to HPC, it is to be noted that a MoU with CINECA has been activesince 2002, and has been renewed in December 2007 until the end of 2010. INAF is oneof the contributors to the Italian Supercomputing and Grid Initiative roadmaps, ISI andIGI, both proposed by the Italian research community to the Ministry of University andResearch (MiUR). It is furthermore to be noticed that the community has successfullyparticipated in the selection of projects for the DEISA Extreme Computing Initiative(DECI).The community has also participated in the national PON initiatives for (su-per)computing resources, and is extensively using campus and geographically distributedGrids for a number of tasks (numerical simulations, stellar evolution models, simulationsof the Planck mission, etc.). Italian astronomers also lead the Astronomy & Astrophysicscluster of EGEE-III, and lead the Database WG within the project itself. They are alsoactive in developing mechanisms to integrate databases within the Grid middleware.In the field of the “data Grid”, Italy participates in the international initiatives for theVirtual Observatory. Italian astronomers hold leadership positions in the InternationalHPC, GRID AND DATA INFRASTRUCTURES FOR ASTROPHYSICS: AN INTEGRATED VIEW 31Virtual Observatory Alliance (IVOA Vice-Chair and Vice-Chair of the Theory InterestGroup). They furthermore participate in three projects funded by the EU FrameworkProgrammes: VO-TECH, EuroVO-DCA and EuroVO-AIDA. In particular, the commu-nity holds the leadership in the integration of the Virtual Observatory with numericalsimulations and the Grid.The “big science” challenges in astrophysics call for an expansion of the comput-ing infrastructures—and of network and data management infrastructures as well. Thecommunity, and INAF in particular, is interested in using, and participating in defining,competitive computing infrastructures so to let its know-how in the field grow. But thereis furthermore the desire to integrate the network, data and computing infrastructures,or at least to let them interoperate.4. – Integrated/interoperable e-infrastructuresTo offer scientists a useful service, all of the components of the informatics infrastruc-ture need to be thought as integrated, or at least fully interoperable. In other words,the various infrastructure components (applications, computing, data) should interactseamlessly exchanging information, and be based on a strong underlying network com-ponent [2].Considering first the computing infrastructure, the facilities available to the scientistscan be very different: from the laptop or desktop PC to the HPC center, passing throughlocal clusters and “the Grid”. From the user perspective, ideally, all facilities should beseen homogeneously; in reality, they all tend to have different access modes. As a result,users find obstacles to their ideal exploitation.As for the data infrastructure, again from the user perspective, there is the needto transparently and homogeneously access a wide variety of data (multi-frequency andmulti-instrument observations and numerical simulations). The World-wide Astronomi-cal Virtual Observatory is progressively fulfilling this requirement by providing seamlessaccess to data centers and facilities through its standards, but this is only the beginningof the story. User queries could be expressed in natural language, or a user query mayimply, besides a data infrastructure, the implicit use of applications and computing re-sources. This, again, requires tight integration of the various infrastructure components.The first thing to do is to enhance the basic infrastructure: network and computingresources. The GARR Network needs to be improved by increasing the bandwidth andeliminating the bottlenecks; in particular, last mile connections to the individual localsites need to be upgraded. At the same time, resources must be found to allow Italyto harmonise with the European plan for HPC by allowing to build at very minimum anational Tier-1 facility, aiming at a PetaFlop Tier-0 system.Not in contrast with this approach, the Grid concept should become far morewidespread than currently. The old perception of the Grid re-using the idle cycles ofa set of PCs (as was in the SETI@home experiment that originated the Grid concept) isnowadays completely misleading—from the computing and storage viewpoints, the Gridhas now the possibility of linking powerful clusters and even HPC hardware, provided ituses the proper middleware.The long-term goal is to allow complete interoperation of HPC centres: initially,Tier-2 (regional) facilities shall be integrated with the Grid, followed by full integrationof local clusters (“Tier 3”) and, as final goal, achieving inclusion of Tier-0/1 within acommon scheme. In any case, a necessary step is to achieve as soon as possible the fullcompatibility between HPC and Grid at least at the User Interface level.32 F. PASIANFig. 1. – The different layers of the integrated ICT infrastructure for astrophysics. Scientificapplications may interact with the Data Infrastructure, or with the Computing Infrastructure, ordirectly with the Network Infrastructure (each interacting with each other). The interoperationof specific applications creates a Knowledge Infrastructure. The dashed box defines the areawhere the Virtual Observatory is expected to act.There is a further point. Up to now, the Grid has mainly delivered computing power,the main issue that its implementers, mostly involved in large High-Energy Physicsexperiments (e.g. at CERN), needed to solve. Accessing data, and databases using theGrid paradigm is a step still to be improved. Some activities are being carried out, alsowith the participation of INAF, within EGEE. This is of course a further step in thedirection of interoperability.To fulfill this requirement applications, computing power, data repositories anddatabases holding metadata or catalogues should be accessed as a single utility. Wemay call it Grid, or Cloud (as is now fashionable), or Virtual Observatory (which maybe more familiar to our community), . . . . It does not really matter, provided the goal isthe same.The far-end goal, in any case, is achieving the full interoperation of the relevantapplications (data processing, analysis, mining, bibliography, semantic web) to startbuilding a Knowledge Infrastructure for Astronomy, as shown in fig. 1.∗ ∗ ∗The INAF-CINECA agreement is funded by INAF and managed by the InformationSystems Unit of the INAF Projects Department. The support of its Director, G. Vet-tolani, to the participation of INAF in the IGI and HPC-ISI proposals to the MIURroadmap is acknowledged. The author wishes to thank M. Mazzucato of INFN for in-formation on Grid initiatives, G. Erbacci and C. Gheller of CINECA for informationon the status of HPC.REFERENCES[1] Solano E., Lect. Notes Essays Astrophys., 2 (2006) 71.[2] Pasian F., (Super)computing within integrated e-Infrastructures, in ComputationalAstrophysics in Italy: Results and Perspectives, edited by Becciani U. (Mem. Soc. Astron.It., Suppl., 13 (2009) 117).

HPC, grid and data infrastructures for astrophysics: An integrated view

http://eprints.bice.rm.cnr.it/16480/1/ncc9375.pdf

HPC, grid and data infrastructures for astrophysics: An integrated view

Abstract

Similar works

Full text

Available Versions

Scientific Open-access Literature Archive and Repository