Efficient Implicit Parallel Patterns for Geographic Information System

International audienceWith the data growth, the need to parallelize treatments become crucial in numerous domains. But for non-specialists it is still difficult to tackle parallelism technicalities as data distribution, communications or load balancing. For the geoscience domain we propose a solution based on implicit parallel patterns. These patterns are abstract models for a class of algorithms which can be customized and automatically transformed in a parallel execution. In this paper, we describe a pattern for stencil computation and a novel pattern dealing with computation following a pre-defined order. They are particularly used in geosciences and we illustrate them with the flow direction and the flow accumulation computations

Performance enhancement of a GIS-based facility location problem using desktop grid infrastructure

This paper presents the integration of desktop grid infrastructure with GIS technologies, by proposing a parallel resolution method in a generic distributed environment. A case study focused on a discrete facility location problem, in the biomass area, exemplifies the high amount of computing resources (CPU, memory, HDD) required to solve the spatial problem. A comprehensive analysis is undertaken in order to analyse the behaviour of the grid-enabled GIS system. This analysis, consisting of a set of the experiments on the case study, concludes that the desktop grid infrastructure is able to use a commercial GIS system to solve the spatial problem achieving high speedup and computational resource utilization. Particularly, the results of the experiments showed an increase in speedup of fourteen times using sixteen computers and a computational efficiency greater than 87 % compared with the sequential procedure.This work has been developed under the support of the program Formacion de Personal Investigador, grants number BFPI/2009/103 and BES-2007-17019, from the Conselleria d'Educacio of the Generalitat Valenciana and the Spanish Ministry of Science and Technology.García García, A.; Perpiñá Castillo, C.; Alfonso Laguna, CD.; Hernández García, V. (2013). Performance enhancement of a GIS-based facility location problem using desktop grid infrastructure. Earth Science Informatics. 6(4):199-207. https://doi.org/10.1007/s12145-013-0119-1S19920764Anderson D (2004) Boinc: a system for public-resource computing and storage. Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing. IEEE Computer Society, Washington DC, pp 4–10Available scripts webpage: http://personales.upv.es/angarg12/Campos I et al (2012) Modelling of a watershed: a distributed parallel application in a grid framework. Comput Informat 27(2):285–296Church RL (2002) Geographical information systems and location science. Comput Oper Res 29:541–562Clarke KC (1986) Advances in geographic information systems, computers. Environ Urban Syst 10:175–184Dowers S, Gittings BM, Mineter MJ (2000) Towards a framework for high-performance geocomputation: handling vector-topology within a distributed service environment. Comput Environ Urban Syst 24:471–486Geograma SL (2009). Teleatlas. http://www.geograma.com . Accessed September 2009GRASS Development Team (2012) GRASS GIS. http://grass.osgeo.org/Hoekstra AG, Sloot PMA (2005) Introducing grid speedup: a scalability metric for parallel applications on the grid, EGC 2005, LNCS 3470, pp. 245–254Hu Y et al. (2004) Feasibility study of geo-spatial analysis using grid computing. Computational Science-ICCS. Springer Berlin Heidelberg, 956–963Huang Z et al (2009) Geobarn: a practical grid geospatial database system. Adv Electr Comput Eng 9:7–11Huang F et al (2011) Explorations of the implementation of a parallel IDW interpolation algorithm in a Linux cluster-based parallel GIS. Comput Geosci 37:426–434Laure E et al (2006) Programming the grid with gLite. CMST 12(1):33–45Li WJ et al (2005) The Design and Implementation of GIS Grid Services. In: Zhuge H, Fox G (eds) Grid and Cooperative Computing. Vol. 3795 of Lecture Notes in Computer Science 10. Springer, Berlin, pp 220–225National Geographic Institute (2010) BCN25: numerical cartographic database. http://www.ign.es/ign/main/index.do . Accessed April 2010Open Geospatial Consortium, Inc (2012) Open GIS Specification Model, http://www.opengeospatial.org/Openshaw S, Turton I (1996) A parallel Kohonen algorithm for the classification of large spatial datasets. Comput Geosci 22:1019–1026Perpiñá C, Alfonso D, Pérez-Navarro A (2007) BIODER project: biomass distributed energy resources assessment and logistic strategies for sitting biomass plants in the Valencia province (Spain), 17th European Biomass Conference and Exhibition Proceedings, Hamburg, Germany, pp. 387–393Perpiñá C et al (2008) Methodology based on Geographic Information Systems for biomass logistics and transport optimization. Renew Energ 34:555–565Shen Z et al (2007) Distributed computing model for processing remotely sensed images based on grid computing. Inf Sci 177:504–518Spanish Ministry of Agriculture, fisheries and food (2009). http://www.magrama.gob.es/es/ . Accessed March 2009Spanish Ministry of Environment (2008). http://www.magrama.gob.es/es/ . Accessed May 2008University of California. List of BOINC projects. http://boinc.berkeley.edu/projects.phpXiao N, Fu W (2003) SDPG: Spatial data processing grid. J Comput Sci Technol 18:523–53

Parallel ODETLAP for terrain compression and reconstruction

We introduce a parallel approximation of an Over-determined Laplacian Partial Differential Equation solver (ODETLAP) applied to the compression and restoration of terrain data used for Geographical Information Systems (GIS). ODET-LAP can be used to reconstruct a compressed elevation map, or to generate a dense regular grid from airborne Light Detection and Ranging (LIDAR) point cloud data. With previous methods, the time to execute ODETLAP does not scale well with the size of the input elevation map, resulting in running times that are prohibitively long for large data sets. Our algorithm divides the data set into patches, runs ODET-LAP on each patch, and then merges the patches together. This method gives two distinct speed improvements. First, we provide scalability by reducing the complexity such that the execution time grows almost linearly with the size of the input, even when run on a single processor. Second, we are able to calculate ODETLAP on the patches concurrently in a parallel or distributed environment. Our new patchbased implementation takes 2 seconds to run ODETLAP on an 800 × 800 elevation map using 128 processors, while the original version of ODETLAP takes nearly 10 minutes on a single processor (271 times longer). We demonstrate the effectiveness of the new algorithm by running it on data sets as large as 16000 × 16000 on a cluster of computers. We also discuss our preliminary results from running on an IBM Blue Gene/L system with 32,768 processors

Building applications with FOSS4G bricks: two examples of the use of GRASS GIS modules as a high-level “language” for the analyses of continuous space data in economic geography

In a world where researchers are more and more confronted to large sets of micro- data, new algorithms are constantly developed that have to be translated into usable programs. Modular GIS toolkits such as GRASS GIS offer a middle way between low-level programming approaches and GUI-based desktop GIS. The modules can be seen as elements of a programming language which makes the implementation of algorithms for spatial analysis very easy for researchers. Using two examples of algorithms in economic geography, for estimating regional exports and for determining raster-object neighbourhood matrices, this paper shows how just a few module calls can replace more complicated low-level programs, as long as the re- searcher can change perspective from a pixel-by-pixel view to a map view of the problem at hand. Combining GRASS GIS with Python as general glue between modules also offers options for easy multi-processing, as well as supporting the increasingly loud call for open research, including open source computing tools in research

DWSI: AN APPROACH TO SOLVING THE POLYGON INTERSECTION-SPREADING PROBLEM WITH A PARALLEL UNION ALGORITHM AT THE FEATURE LAYER LEVEL

A dual-way seeds indexing (DWSI) method based on R-tree and the OpenGeospatial Consortium (OGC) simple feature model was proposed to solve the polygon intersection-spreading problem. The parallel polygon union algorithm based on the improved DWSI and the OpenMP parallel programming model was developed to validate the usability of the data partition method. The experimental results reveal that the improved DWSI method can implement a robust parallel task partition by overcoming the polygon intersection-spreading problem. The parallel union algorithm applied DWSI not only scaled up the data processing but alsospeeded up the computation compared with the serial proposal, and it showed ahigher computational efficiency with higher speedup benchmarks in the treatment of larger-scale dataset. Therefore, the improved DWSI can be a potential approach to parallelizing the vector data overlay algorithms based on the OGC simple data model at the feature layer level

DWSI: AN APPROACH TO SOLVING THE POLYGON INTERSECTION-SPREADING PROBLEM WITH A PARALLEL UNION ALGORITHM AT THE FEATURE LAYER LEVEL

A dual-way seeds indexing (DWSI) method based on R-tree and the OpenGeospatial Consortium (OGC) simple feature model was proposed to solve the polygon intersection-spreading problem. The parallel polygon union algorithm based on the improved DWSI and the OpenMP parallel programming model was developed to validate the usability of the data partition method. The experimental results reveal that the improved DWSI method can implement a robust parallel task partition by overcoming the polygon intersection-spreading problem. The parallel union algorithm applied DWSI not only scaled up the data processing but alsospeeded up the computation compared with the serial proposal, and it showed ahigher computational efficiency with higher speedup benchmarks in the treatment of larger-scale dataset. Therefore, the improved DWSI can be a potential approach to parallelizing the vector data overlay algorithms based on the OGC simple data model at the feature layer level

From SpaceStat to CyberGIS: Twenty Years of Spatial Data Analysis Software

This essay assesses the evolution of the way in which spatial data analytical methods have been incorporated into software tools over the past two decades. It is part retrospective and prospective, going beyond a historical review to outline some ideas about important factors that drove the software development, such as methodological advances, the open source movement and the advent of the internet and cyberinfrastructure. The review highlights activities carried out by the author and his collaborators and uses SpaceStat, GeoDa, PySAL and recent spatial analytical web services developed at the ASU GeoDa Center as illustrative examples. It outlines a vision for a spatial econometrics workbench as an example of the incorporation of spatial analytical functionality in a cyberGIS.

Parallelization of web processing services on cloud computing: A case study of Geostatistical Methods

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.In the last decade the publication of geographic information has increased in Internet, especially with the emergence of new technologies to share information. This information requires the use of technologies of geoprocessing online that use new platforms such as Cloud Computing. This thesis work evaluates the parallelization of geoprocesses on the Cloud platform Amazon Web Service (AWS), through OGC Web Processing Services (WPS) using the 52North WPS framework. This evaluation is performed using a new implementation of a Geostatistical library in Java with parallelization capabilities. The geoprocessing is tested by incrementing the number of micro instances on the Cloud through GridGain technology. The Geostatistical library obtains similar interpolated values compared with the software ArcGIS. In the Inverse Distance Weight (IDW) and Radial Basis Functions (RBF) methods were not found differences. In the Ordinary and Universal Kriging methods differences have been found of 0.01% regarding the Root Mean Square (RMS) error.The parallelization process demonstrates that the duration of the interpolation decreases when the number of nodes increases. The duration behavior depends on the size of input dataset and the number of pixels to be interpolated. The maximum reduction in time was found with the largest configuration used in the research (1.000.000 of pixels and a dataset of 10.000 points). The execution time decreased in 83% working with 10 nodes in the Ordinary Kriging and IDW methods. However, the differences in duration working with 5 nodes and 10 nodes were not statistically significant. The reductions with 5 nodes were 72% and 71% in the Ordinary Kriging and IDW methods respectively. Finally, the experiments show that the geoprocessing on Cloud Computing is feasible using the WPS interface. The performance of the geostatistical methods deployed through the WPS services can improve by the parallelization technique. This thesis proves that the parallelization on the Cloud is viable using a Grid configuration. The evaluation also showed that parallelization of geoprocesses on the Cloud for academic purposes is inexpensive using Amazon AWS platform

Development of a New Framework for Distributed Processing of Geospatial Big Data

Geospatial technology is still facing a lack of “out of the box” distributed processing solutions which are suitable for the amount and heterogeneity of geodata, and particularly for use cases requiring a rapid response. Moreover, most of the current distributed computing frameworks have important limitations hindering the transparent and flexible control of processing (and/or storage) nodes and control of distribution of data chunks. We investigated the design of distributed processing systems and existing solutions related to Geospatial Big Data. This research area is highly dynamic in terms of new developments and the re-use of existing solutions (that is, the re-use of certain modules to implement further specific developments), with new implementations continuously emerging in areas such as disaster management, environmental monitoring and earth observation. The distributed processing of raster data sets is the focus of this paper, as we believe that the problem of raster data partitioning is far from trivial: a number of tiling and stitching requirements need to be addressed to be able to fulfil the needs of efficient image processing beyond pixel level. We attempt to compare the terms Big Data, Geospatial Big Data and the traditional Geospatial Data in order to clarify the typical differences, to compare them in terms of storage and processing backgrounds for different data representations and to categorize the common processing systems from the aspect of distributed raster processing. This clarification is necessary due to the fact that they behave differently on the processing side, and particular processing solutions need to be developed according to their characteristics. Furthermore, we compare parallel and distributed computing, taking into account the fact that these are used improperly in several cases. We also briefly assess the widely-known MapReduce paradigm in the context of geospatial applications. The second half of the article reports on a new processing framework initiative, currently at the concept and early development stages, which aims to be capable of processing raster, vector and point cloud data in a distributed IT ecosystem. The developed system is modular, has no limitations on programming language environment, and can execute scripts written in any development language (e.g. Python, R or C#)