Cosmological Simulations using Grid Middleware

One way to access the aggregated power of a collection of heterogeneous machines is to use a grid middleware, such as DIET, GridSolve or NINF. It addresses the problem of monitoring the resources, of handling the submissions of jobs and as an example the inherent transfer of input and output data, in place of the user. In this paper we present how to run cosmological simulations using the RAMSES application along with the DIET middleware. We will describe how to write the corresponding DIET client and server. The remainder of the paper is organized as follows: Section 2 presents the DIET middleware. Section 3 describes the RAMSES cosmological software and simulations, and how to interface it with DIET. We show how to write a client and a server in Section 4. Finally, Section 5 presents the experiments realized on Grid'5000, the French Research Grid, and we conclude in Section 6.Comment: submitted Nov 200

AstroGrid-D: Enhancing Astronomic Science with Grid Technology

We present AstroGrid-D, a project bringing together astronomers and experts in Grid technology to enhance astronomic science in many aspects. First, by sharing currently dispersed resources, scientists can calculate their models in more detail. Second, by developing new mechanisms to efficiently access and process existing datasets, scientific problems can be investigated that were until now impossible to solve. Third, by adopting Grid technology large instruments such as robotic telescopes and complex scientific workflows from data aquisition to analysis can be managed in an integrated manner. In this paper, we present prominent astronomic use cases, discuss requirements on a Grid middleware and present our approach to extend/augment existing middleware to facilitate the improvements mentioned above

Comparison on OpenStack and OpenNebula performance to improve multi-Cloud architecture on cosmological simulation use case

With the increasing numbers of Cloud Service Providers and the migration of the Grids to the Cloud paradigm, it is necessary to be able to leverage these new resources. Moreover, a large class of High Performance Computing (HPC) applications can run these resources without (or with minor) modifications. But using these resources come with the cost of being able to interact with these new resource providers. In this paper we introduce the design of a HPC middleware that is able to use resources coming from an environment that compose of multiple Clouds as well as classical \hpc resources. Using the \diet middleware, we are able to deploy a large-scale, distributed HPC platform that spans across a large pool of resources aggregated from different providers. Furthermore, we hide to the end users the difficulty and complexity of selecting and using these new resources even when new Cloud Service Providers are added to the pool. Finally, we validate the architecture concept through cosmological simulation RAMSES. Thus we give a comparison of 2 well-known Cloud Computing Software: OpenStack and OpenNebula.Avec l'augmentation du nombre de fournisseurs de service Cloud et la migration des applications depuis les grilles de calcul vers le Cloud, il est nécessaire de pouvoir tirer parti de ces nouvelles ressources. De plus, une large classe des applications de calcul haute performance peuvent s'exécuter sur ces ressources sans modifications (ou avec des modifications mineures). Mais utiliser ces ressources vient avec le coût d'être capable d'intéragir avec des nouveaux fournisseurs de ressources. Dans ce papier, nous introduisons la conception d'un nouveau intergiciel HPC qui permet d'utiliser les ressources qui proviennent d'un environement composé de plusieurs Clouds comme des ressources classiques. En utilisant l'intergiciel \diet, nous sommes capable de déployer une plateforme HPC distribuée et large échelle qui s'étend sur un large ensemble de ressources aggrégées entre plusieurs fournisseurs Cloud. De plus, nous cachons à l'utilisateur final la difficulté et la complexité de sélectionner et d'utiliser ces nouvelles ressources quand un nouveau fournisseur de service Cloud est ajouté dans l'ensemble. Finalement, nous validons notre concept d'architecture via une application de simulation cosmologique RAMSES. Et nous fournissons une comparaison entre 2 intergiciels de Cloud: OpenStack et OpenNebula

Statistical Computations with AstroGrid and the Grid

We outline our first steps towards marrying two new and emerging technologies; the Virtual Observatory (e.g, AstroGrid) and the computational grid. We discuss the construction of VOTechBroker, which is a modular software tool designed to abstract the tasks of submission and management of a large number of computational jobs to a distributed computer system. The broker will also interact with the AstroGrid workflow and MySpace environments. We present our planned usage of the VOTechBroker in computing a huge number of n-point correlation functions from the SDSS, as well as fitting over a million CMBfast models to the WMAP data.Comment: Invited talk to appear in "Proceedings of PHYSTAT05: Statistical Problems in Particle Physics, Astrophysics and Cosmology

The GalMer database: Galaxy Mergers in the Virtual Observatory

We present the GalMer database, a library of galaxy merger simulations, made available to users through tools compatible with the Virtual Observatory (VO) standards adapted specially for this theoretical database. To investigate the physics of galaxy formation through hierarchical merging, it is necessary to simulate galaxy interactions varying a large number of parameters: morphological types, mass ratios, orbital configurations, etc. On one side, these simulations have to be run in a cosmological context, able to provide a large number of galaxy pairs, with boundary conditions given by the large-scale simulations, on the other side the resolution has to be high enough at galaxy scales, to provide realistic physics. The GalMer database is a library of thousands simulations of galaxy mergers at moderate spatial resolution and it is a compromise between the diversity of initial conditions and the details of underlying physics. We provide all coordinates and data of simulated particles in FITS binary tables. The main advantages of the database are VO access interfaces and value-added services which allow users to compare the results of the simulations directly to observations: stellar population modelling, dust extinction, spectra, images, visualisation using dedicated VO tools. The GalMer value-added services can be used as virtual telescope producing broadband images, 1D spectra, 3D spectral datacubes, thus making our database oriented towards the usage by observers. We present several examples of the GalMer database scientific usage obtained from the analysis of simulations and modelling their stellar population properties, including: (1) studies of the star formation efficiency in interactions; (2) creation of old counter-rotating components; (3) reshaping metallicity profiles in elliptical galaxies; (4) orbital to internal angular momentum transfer; (5) reproducing observed colour bimodality of galaxies.Comment: 15 pages, 11 figures, 10 tables accepted to A&A. Visualisation of GalMer simulations, access to snapshot files and value-added tools described in the paper are available at http://galmer.obspm.fr