A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

BARD: Better Automated Redistricting

BARD is the first (and at time of writing, only) open source software package for general redistricting and redistricting analysis. BARD provides methods to create, display, compare, edit, automatically refine, evaluate, and profile political districting plans. BARD aims to provide a framework for scientific analysis of redistricting plans and to facilitate wider public participation in the creation of new plans. BARD facilitates map creation and refinement through command-line, graphical user interface, and automatic methods. Since redistricting is a computationally complex partitioning problem not amenable to an exact optimization solution, BARD implements a variety of selectable metaheuristics that can be used to refine existing or randomly-generated redistricting plans based on user-determined criteria. Furthermore, BARD supports automated generation of redistricting plans and profiling of plans by assigning different weights to various criteria, such as district compactness or equality of population. This functionality permits exploration of trade-offs among criteria. The intent of a redistricting authority may be explored by examining these trade-offs and inferring which reasonably observable plans were not adopted. Redistricting is a computationally-intensive problem for even modest-sized states. Performance is thus an important consideration in BARD's design and implementation. The program implements performance enhancements such as evaluation caching, explicit memory management, and distributed computing across snow clusters.

Mobile Map Browsers: Anticipated User Interaction for Data Pre-fetching

When browsing a graphical display of geospatial data on mobile devices, users typically change the displayed maps by panning, zooming in and out, or rotating the device. Limited storage space on mobile devices and slow wireless communications, however, impede the performance of these operations. To overcome the bottleneck that all map data to be displayed on the mobile device need to be downloaded on demand, this thesis investigates how anticipated user interactions affect intelligent pre-fetching so that an on-demand download session is extended incrementally. User interaction is defined as a set of map operations that each have corresponding effects on the spatial dataset required to generate the display. By anticipating user interaction based on past behavior and intuition on when waiting for data is acceptable, it is possible to device a set of strategies to better prepare the device with data for future use. Users that engage with interactive map displays for a variety of tasks, whether it be navigation, information browsing, or data collection, experience a dynamic display to accomplish their goal. With vehicular navigation, the display might update itself as a result of a GPS data stream reflecting movement through space. This movement is not random, especially as is the case of moving vehicles and, therefore, this thesis suggests that mobile map data could be pre-fetched in order to improve usability. Pre-fetching memory-demanding spatial data can benefit usability in several ways, but in particular it can (1) reduce latency when downloading data over wireless connections and (2) better prepare a device for situations where wireless internet connectivity is weak or intermittent. This thesis investigates mobile map caching and devises an algorithm for pre-fetching data on behalf of the application user. Two primary models are compared: isotropic (direction-independent) and anisotropic (direction-dependent) pre-fetching. A prefetching simulation is parameterized with many trajectories that vary in complexity (a metric of direction change within the trajectory) and it is shown that, although anisotropic pre-fetching typically results in a better pre-fetching accuracy, it is not ideal for all scenarios. This thesis suggests a combination of models to accommodate the significant variation in moving object trajectories. In addition, other methods for pre-fetching spatial data are proposed for future research

Exploiting Information-centric Networking to Federate Spatial Databases

This paper explores the methodologies, challenges, and expected advantages related to the use of the information-centric network (ICN) technology for federating spatial databases. ICN services allow simplifying the design of federation procedures, improving their performance, and providing so-called data-centric security. In this work, we present an architecture that is able to federate spatial databases and evaluate its performance using a real data set coming from OpenStreetMap within a heterogeneous federation formed by MongoDB and CouchBase spatial database systems

Evaluación del tiempo de respuesta de un geoservicio utilizando una base de datos híbrida y distribuida

Web mapping services provide information directly to users and other software programs that can consume and produce information. One of the main challenges this type of service presents is improving its performance. Therefore, in this research, a new geoservice integrated into GeoServer was developed, called GeoToroTur, with an OWS implementation of vector layers that consumes the information from a hybrid and distributed database that was implemented with PostgreSQL and MongoDB, making use of ToroDB for document replication. This geoservice was evaluated by executing geographic and descriptive attribute filter queries. Based on the results, we can conclude that the response time for GeoToroTur is shorter than that for Geoserver.Los servicios de cartografía Web proporcionan información directamente, no sólo a los usuarios, sino también a otros programas de software que pueden consumir y producir información. Uno de los principales retos que presentan este tipo de servicios es mejorar su rendimiento. Por ello, en esta investigación se desarrolló un nuevo geoservicio integrado a GeoServer, denominado GeoToroTur con una implementación OWS de capas vectoriales que consume la información de una base de datos híbrida y distribuida que fue implementada con PostgreSQL y MongoDB haciendo uso de ToroDB para la replicación de documentos. Este geoservicio fue evaluado mediante la ejecución de consultas geográficas y de filtro de atributos descriptivos. Los resultados obtenidos permiten concluir que el geoservicio GeoToroTur tiene un menor tiempo de respuesta que Geoserver

Mobile map client API : design and implementation for Android

The fast development of computational power of the mobile phone makes it a suitable platform for running map applications. Both public and field working professionals can benefit from easy access to a mobile map client application with features such as route planning, location based services and simple GIS operations. This master's thesis describes the mobile operating system (OS) Android from a geographic information aspect and relates it to other major mobile OS. Available map client application programming interfaces (API) are investigated. It is concluded that Android is a good platform choice for implementing mobile map applications. But there is need of a generic open source API for Android. Such an API is implemented, resulting in a good performing map client. Though it needs additional development to perform all features aimed for in the suggested API design.Populärvetenskaplig sammanfattning: Dagens datorlika mobiltelefoner lämpar sig utmärkt för att visa olika typer av kartor. Det är inte svårt att hitta tillämpningar för både privatpersoner och professionella användare med behov som att till exempel hitta lämpliga färdvägar eller visa information om platsen man befinner sig på. Det här examensarbetet går ut på att undersöka hur geografisk data kan behandlas och visas som kartor på Android-mobiler. Målet är att skapa ett generellt programmeringsverktyg som ska kunna användas för att snabbt och enkelt kunna utveckla specialiserade kartapplikationer för Android. Ett sådant verktyg kallas inom databranschen för API, vilket står för Application Programming Interface. Ett API är ett gränssnitt som ger programmerare smidig tillgång till grundläggande funktioner i ett system eller en typ av applikation. I det här fallet får programmeraren tillgång till grundläggande kartfunktioner, som t.ex. kartdataladdning eller panorering och zoom, utan att behöva programmera hela logiken bakom dessa funktioner. Istället kan programmeraren fokusera sitt arbete på de delarna som gör just den kartapplikationen unik. Arbetet är uppdelat i en undersökande del för att klarlägga var utvecklingen av tekniken befinner sig idag och vilka liknande försök som gjorts tidigare och en mer praktisk utvecklingsdel med målet att implementera ett API för kartapplikationer på Android. I undersökningen jämförs Android med andra mobila plattformar, t.ex. Apples iPhone och Nokias Symbian. Det konstateras att Android står sig bra i konkurens med de andra och lämpar sig väl för kartapplikationer men att det skulle behövas ett generellt kartverktyg som inte är så hårt styrt som Google maps, som är det vanligaste kartverktyget för Android. För att tydliggöra vad API:et borde innehålla och klara av att utföra gjordes först en teoretisk modell av dess olika delar. De viktigaste delarna, t.ex. dataladdning och användargränssnitt med kartbildsvisning, panorering och zoom, implementerades sedan för att kunna testköras och vidareutvecklas på en Android-mobil. En demoapplikation utvecklades för att demonstrera användningen av API:et. Resultatet är ett verktyg för ett energibolags servicepersonal att ha med sig i fält för att se deras fjärrvärmenäts geografiska utsträckning och info om nätets olika delar, kopplingar och kunder.Scientific summary: The fast development of computational power of the mobile phone have made it a suitable platform for running map applications. Both public and field working professionals can benefit from easy access to a mobile map client application with features such as route planning, location based services and simple GIS operations. This master's thesis describes the mobile operating system (OS) Android from a geographic information aspect and relates it to other major mobile OS. A map client application programming interfaces (API) is implemented, resulting in a good performing map client. Though it needs additional development to perform all features aimed for in the suggested API design. The study is divided into a state-of-the-art investigation of available technology, and an effort to design and implement a map client API for Android. The investigation was made by referencing available scientific papers, technical articles and websites. Available OS's and API's are investigated. It is concluded that Android is a good platform choice for implementing mobile map applications. But there is need of a generic open source API for Android. Another objective of the study was to compare a map client running as a web application to the native client developed regarding performance. Unfortunately no suiting API for setting up the web application was found. Since then the OpenLayers community have made a great effort for running OpenLayers clients on mobile devices, which would have been an ideal choice for doing the comparison. The work on the API contains a theoretical model which is partly implemented. The aim of the model is to define the overall functionality, some specialized features and cohesive data structures. Then the base functionality, such as data loading, map drawing, panning and zooming, was implemented. The implementation was carried out in an iterative manner. Starting with complementing a basic, incomplete open source map API to gain understanding of mobile map API design and avoid re-inventing the wheel. Added was support for coordinate systems and projections other than WGS 84 Web Mercator (EPSG 4326), better Web Map Service (WMS) support, vector data loading and improved responsiveness when loading data. A demo application was put together in order to test performance on the hardware emulator integrated in the Android SDK and on a real device, Motorola Milestone running Android version 2.1. The scenario for the demo application is a district heating pipe system, an useful field application for a city energy company. The application shows a raster background map loaded from a tiled WMS with the pipe system as vector overlays loaded from a custom Keyhole Markup Language (KML) service. The vector data is click-able in order to show information about different parts of the pipe system

Slicer

Explorative data visualization is a widespread tool for gaining insights from datasets. Investigating data in linked visualizations lets users explore potential relationships in their data at will. Furthermore, this type of analysis does not require any technical knowledge, widening the userbase from developers to anyone. Implementing explorative data visualizations in web browsers makes data analysis accessible to anyone with a PC. In addition to accessibility, the available types of visualizations and their interactive latency are essential for the utility of data exploration. Available visualizations limit the number of datasets eligible for use in the application, and latency limits how much exploring the users are willing to do. Existing solutions often do all the computation involved in either the client application or on a backend server. However, using the client limits performance and data size since hardware resources in web browsers are scarce, and sending large datasets over a network is not feasible. Whereas server-based computation often comes with high requirements for server hardware and is limited by network latency and bandwidth on each interaction. This thesis presents Slicer, a framework for creating explorative data visualizations in web browsers. Applications can be created with minimal developer effort, requiring only a description of the visualizations. Slicer implements bar charts and choropleth maps. The visualizations are linked and can be filtered either by brushing or clicking on single targets. To overcome the hurdles of pure client- and server-reliant solutions, Slicer uses a hybrid approach, where prioritized interactions are handled client-side. Recognizing that different types of interactions have different latency thresholds, we trade the cost of switching views for low latency on filtering. To achieve real-time filtering performance, we follow the principle that the chosen resolution of the visualizations, not data size, should limit interactive scalability. We describe use of data tiles accommodating more interactions than shown in earlier work, using an approach based on delta differencing, which ensures constant time complexity when filtering. For computing data tiles, we present techniques for efficient computation on consumer hardware. Our results show that Slicer can offer real-time interactivity on latency-sensitive interactions regardless of data size, averaging above 150Hz on a consumer laptop. For less sensitive interactions, acceptable latency is shown for datasets with tens of millions of records, depending on the resolution of the visualizations