PASSION: Parallel And Scalable Software for Input-Output

We are developing a software system called PASSION: Parallel And Scalable Software for Input-Output which provides software support for high performance parallel I/O. PASSION provides support at the language, compiler, runtime as well as file system level. PASSION provides runtime procedures for parallel access to files (read/write), as well as for out-of-core computations. These routines can either be used together with a compiler to translate out-of-core data parallel programs written in a language like HPF, or used directly by application programmers. A number of optimizations such as Two-Phase Access, Data Sieving, Data Prefetching and Data Reuse have been incorporated in the PASSION Runtime Library for improved performance. PASSION also provides an initial framework for runtime support for out-of-core irregular problems. The goal of the PASSION compiler is to automatically translate out- of-core data parallel programs to node programs for distributed memory machines, with calls to the PASSION Runtime Library. At the language level, PASSION suggests extensions to HPF for out-of-core programs. At the file system level, PASSION provides support for buffering and prefetching data from disks. A portable parallel file system is also being developed as part of this project, which can be used across homogeneous or heterogeneous networks of workstations. PASSION also provides support for integrating data and task parallelism using parallel I/O techniques. We have used PASSION to implement a number of out-of-core applications such as a Laplace\u27s equation solver, 2D FFT, Matrix Multiplication, LU Decomposition, image processing applications as well as unstructured mesh kernels in molecular dynamics and computational fluid dynamics. We are currently in the process of using PASSION in applications in CFD (3D turbulent flows), molecular structure calculations, seismic computations, and earth and space science applications such as Four-Dimensional Data Assimilation. PASSION is currently available on the Intel Paragon, Touchstone Delta and iPSC/860. Efforts are underway to port it to the IBM SP-1 and SP-2 using the Vesta Parallel File System

Performance enhancement of a GIS-based facility location problem using desktop grid infrastructure

This paper presents the integration of desktop grid infrastructure with GIS technologies, by proposing a parallel resolution method in a generic distributed environment. A case study focused on a discrete facility location problem, in the biomass area, exemplifies the high amount of computing resources (CPU, memory, HDD) required to solve the spatial problem. A comprehensive analysis is undertaken in order to analyse the behaviour of the grid-enabled GIS system. This analysis, consisting of a set of the experiments on the case study, concludes that the desktop grid infrastructure is able to use a commercial GIS system to solve the spatial problem achieving high speedup and computational resource utilization. Particularly, the results of the experiments showed an increase in speedup of fourteen times using sixteen computers and a computational efficiency greater than 87 % compared with the sequential procedure.This work has been developed under the support of the program Formacion de Personal Investigador, grants number BFPI/2009/103 and BES-2007-17019, from the Conselleria d'Educacio of the Generalitat Valenciana and the Spanish Ministry of Science and Technology.García García, A.; Perpiñá Castillo, C.; Alfonso Laguna, CD.; Hernández García, V. (2013). Performance enhancement of a GIS-based facility location problem using desktop grid infrastructure. Earth Science Informatics. 6(4):199-207. https://doi.org/10.1007/s12145-013-0119-1S19920764Anderson D (2004) Boinc: a system for public-resource computing and storage. Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing. IEEE Computer Society, Washington DC, pp 4–10Available scripts webpage: http://personales.upv.es/angarg12/Campos I et al (2012) Modelling of a watershed: a distributed parallel application in a grid framework. Comput Informat 27(2):285–296Church RL (2002) Geographical information systems and location science. Comput Oper Res 29:541–562Clarke KC (1986) Advances in geographic information systems, computers. Environ Urban Syst 10:175–184Dowers S, Gittings BM, Mineter MJ (2000) Towards a framework for high-performance geocomputation: handling vector-topology within a distributed service environment. Comput Environ Urban Syst 24:471–486Geograma SL (2009). Teleatlas. http://www.geograma.com . Accessed September 2009GRASS Development Team (2012) GRASS GIS. http://grass.osgeo.org/Hoekstra AG, Sloot PMA (2005) Introducing grid speedup: a scalability metric for parallel applications on the grid, EGC 2005, LNCS 3470, pp. 245–254Hu Y et al. (2004) Feasibility study of geo-spatial analysis using grid computing. Computational Science-ICCS. Springer Berlin Heidelberg, 956–963Huang Z et al (2009) Geobarn: a practical grid geospatial database system. Adv Electr Comput Eng 9:7–11Huang F et al (2011) Explorations of the implementation of a parallel IDW interpolation algorithm in a Linux cluster-based parallel GIS. Comput Geosci 37:426–434Laure E et al (2006) Programming the grid with gLite. CMST 12(1):33–45Li WJ et al (2005) The Design and Implementation of GIS Grid Services. In: Zhuge H, Fox G (eds) Grid and Cooperative Computing. Vol. 3795 of Lecture Notes in Computer Science 10. Springer, Berlin, pp 220–225National Geographic Institute (2010) BCN25: numerical cartographic database. http://www.ign.es/ign/main/index.do . Accessed April 2010Open Geospatial Consortium, Inc (2012) Open GIS Specification Model, http://www.opengeospatial.org/Openshaw S, Turton I (1996) A parallel Kohonen algorithm for the classification of large spatial datasets. Comput Geosci 22:1019–1026Perpiñá C, Alfonso D, Pérez-Navarro A (2007) BIODER project: biomass distributed energy resources assessment and logistic strategies for sitting biomass plants in the Valencia province (Spain), 17th European Biomass Conference and Exhibition Proceedings, Hamburg, Germany, pp. 387–393Perpiñá C et al (2008) Methodology based on Geographic Information Systems for biomass logistics and transport optimization. Renew Energ 34:555–565Shen Z et al (2007) Distributed computing model for processing remotely sensed images based on grid computing. Inf Sci 177:504–518Spanish Ministry of Agriculture, fisheries and food (2009). http://www.magrama.gob.es/es/ . Accessed March 2009Spanish Ministry of Environment (2008). http://www.magrama.gob.es/es/ . Accessed May 2008University of California. List of BOINC projects. http://boinc.berkeley.edu/projects.phpXiao N, Fu W (2003) SDPG: Spatial data processing grid. J Comput Sci Technol 18:523–53

Many-Task Computing and Blue Waters

This report discusses many-task computing (MTC) generically and in the context of the proposed Blue Waters systems, which is planned to be the largest NSF-funded supercomputer when it begins production use in 2012. The aim of this report is to inform the BW project about MTC, including understanding aspects of MTC applications that can be used to characterize the domain and understanding the implications of these aspects to middleware and policies. Many MTC applications do not neatly fit the stereotypes of high-performance computing (HPC) or high-throughput computing (HTC) applications. Like HTC applications, by definition MTC applications are structured as graphs of discrete tasks, with explicit input and output dependencies forming the graph edges. However, MTC applications have significant features that distinguish them from typical HTC applications. In particular, different engineering constraints for hardware and software must be met in order to support these applications. HTC applications have traditionally run on platforms such as grids and clusters, through either workflow systems or parallel programming systems. MTC applications, in contrast, will often demand a short time to solution, may be communication intensive or data intensive, and may comprise very short tasks. Therefore, hardware and software for MTC must be engineered to support the additional communication and I/O and must minimize task dispatch overheads. The hardware of large-scale HPC systems, with its high degree of parallelism and support for intensive communication, is well suited for MTC applications. However, HPC systems often lack a dynamic resource-provisioning feature, are not ideal for task communication via the file system, and have an I/O system that is not optimized for MTC-style applications. Hence, additional software support is likely to be required to gain full benefit from the HPC hardware

Development of Grid e-Infrastructure in South-Eastern Europe

Over the period of 6 years and three phases, the SEE-GRID programme has established a strong regional human network in the area of distributed scientific computing and has set up a powerful regional Grid infrastructure. It attracted a number of user communities and applications from diverse fields from countries throughout the South-Eastern Europe. From the infrastructure point view, the first project phase has established a pilot Grid infrastructure with more than 20 resource centers in 11 countries. During the subsequent two phases of the project, the infrastructure has grown to currently 55 resource centers with more than 6600 CPUs and 750 TBs of disk storage, distributed in 16 participating countries. Inclusion of new resource centers to the existing infrastructure, as well as a support to new user communities, has demanded setup of regionally distributed core services, development of new monitoring and operational tools, and close collaboration of all partner institution in managing such a complex infrastructure. In this paper we give an overview of the development and current status of SEE-GRID regional infrastructure and describe its transition to the NGI-based Grid model in EGI, with the strong SEE regional collaboration.Comment: 22 pages, 12 figures, 4 table

A grid-based infrastructure for distributed retrieval

In large-scale distributed retrieval, challenges of latency, heterogeneity, and dynamicity emphasise the importance of infrastructural support in reducing the development costs of state-of-the-art solutions. We present a service-based infrastructure for distributed retrieval which blends middleware facilities and a design framework to ‘lift’ the resource sharing approach and the computational services of a European Grid platform into the domain of e-Science applications. In this paper, we give an overview of the DILIGENT Search Framework and illustrate its exploitation in the field of Earth Science