Array Requirements for Scientific Applications and an Implementation for Microsoft SQL Server

This paper outlines certain scenarios from the fields of astrophysics and fluid dynamics simulations which require high performance data warehouses that support array data type. A common feature of all these use cases is that subsetting and preprocessing the data on the server side (as far as possible inside the database server process) is necessary to avoid the client-server overhead and to minimize IO utilization. Analyzing and summarizing the requirements of the various fields help software engineers to come up with a comprehensive design of an array extension to relational database systems that covers a wide range of scientific applications. We also present a working implementation of an array data type for Microsoft SQL Server 2008 to support large-scale scientific applications. We introduce the design of the array type, results from a performance evaluation, and discuss the lessons learned from this implementation. The library can be downloaded from our website at http://voservices.net/sqlarray

EcoGIS – GIS tools for ecosystem approaches to fisheries management

Executive Summary: The EcoGIS project was launched in September 2004 to investigate how Geographic Information Systems (GIS), marine data, and custom analysis tools can better enable fisheries scientists and managers to adopt Ecosystem Approaches to Fisheries Management (EAFM). EcoGIS is a collaborative effort between NOAA’s National Ocean Service (NOS) and National Marine Fisheries Service (NMFS), and four regional Fishery Management Councils. The project has focused on four priority areas: Fishing Catch and Effort Analysis, Area Characterization, Bycatch Analysis, and Habitat Interactions. Of these four functional areas, the project team first focused on developing a working prototype for catch and effort analysis: the Fishery Mapper Tool. This ArcGIS extension creates time-and-area summarized maps of fishing catch and effort from logbook, observer, or fishery-independent survey data sets. Source data may come from Oracle, Microsoft Access, or other file formats. Feedback from beta-testers of the Fishery Mapper was used to debug the prototype, enhance performance, and add features. This report describes the four priority functional areas, the development of the Fishery Mapper tool, and several themes that emerged through the parallel evolution of the EcoGIS project, the concept and implementation of the broader field of Ecosystem Approaches to Management (EAM), data management practices, and other EAM toolsets. In addition, a set of six succinct recommendations are proposed on page 29. One major conclusion from this work is that there is no single “super-tool” to enable Ecosystem Approaches to Management; as such, tools should be developed for specific purposes with attention given to interoperability and automation. Future work should be coordinated with other GIS development projects in order to provide “value added” and minimize duplication of efforts. In addition to custom tools, the development of cross-cutting Regional Ecosystem Spatial Databases will enable access to quality data to support the analyses required by EAM. GIS tools will be useful in developing Integrated Ecosystem Assessments (IEAs) and providing pre- and post-processing capabilities for spatially-explicit ecosystem models. Continued funding will enable the EcoGIS project to develop GIS tools that are immediately applicable to today’s needs. These tools will enable simplified and efficient data query, the ability to visualize data over time, and ways to synthesize multidimensional data from diverse sources. These capabilities will provide new information for analyzing issues from an ecosystem perspective, which will ultimately result in better understanding of fisheries and better support for decision-making. (PDF file contains 45 pages.

GDMS-R: A mixed SQL to manage raster and vector data

11pInternational audienceTo evaluate urbanization impact on territories, an accurate knowledge of the urban and peri-urban fabrics is unavoidable. To provide advanced characterization of the terrain, modern GIS applications target even wider geographic areas at finer resolutions but they also have to mix data of different types such as Digital Elevation Model (raster layer), buildings (polygonal layer) and roads (polylines layer). Processing both raster and vector data with the same semantic and in an efficient way presents significant challenges to GIS insofar as underlying granularities but also data layout and processing patterns might be absolutely different. We have already focused on the definition and the implementation of an abstraction layer called GDMS (Generic Datasource Management System) to handle and process vector data. Main objectives with GDMS, were to provide the user not only a simple and powerful API but also a spatial SQL derived language. Moreover, as an intermediate layer between the user and the information source, GDMS intends to reduce the coupling between the processes and the specificities of each underlying format. As a consequence, former work may easily be reused in a much larger set of scenarii. The learning curve is consequently even simpler. In this paper, we propose a raster extension to the GDMS layer called GDMS-R. Even if, there is currently no OGC standard concerning raster processing (using well-known SQL language), there already exists a de facto standard called Map Algebra defined by C. D. Tomlin in 1990 and commonly implemented in a wide set of GIS. Our objective is a bit different insofar as we propose to extend SQL language. We present the integration of Map Algebra concepts in GDMS through the GRAP (GeoRAster Processing) language. As for GDMS, reuse is enhanced by the possibility of being vendor-independent (middle-ware approach) and the extension capabilities of the underlying SQL language. To demonstrate the capabilities of GDMS-R, we present a use case relative to the deep impact of increased urbanization on the vulnerability of peri-urban hydro-systems: impact of the linear constraints on the runoff water pathways and accumulation that uses both vector and raster data in an unified way