A grid-based infrastructure for distributed retrieval

In large-scale distributed retrieval, challenges of latency, heterogeneity, and dynamicity emphasise the importance of infrastructural support in reducing the development costs of state-of-the-art solutions. We present a service-based infrastructure for distributed retrieval which blends middleware facilities and a design framework to ‘lift’ the resource sharing approach and the computational services of a European Grid platform into the domain of e-Science applications. In this paper, we give an overview of the DILIGENT Search Framework and illustrate its exploitation in the field of Earth Science

Web mapping for promoting interaction and collaboration in community land planning

There is an inherent advantage of geographic information Systems (GIS) and mapping in facilitating dialogue between experts and non-experts during land use plan development. Combining visual mapping information and effective user interaction can result in considerable benefits for developing countries like Botswana. Although the adoption of information and communication technologies has lagged behind that for developed countries, initiatives by the Botswana government in providing suitable information infrastructures, including internet and web based communications, are enabling multiple users to interact and collaborate in community land planning. A web mapping application was developed for the Maun Development Plan (MDP) in the Okavango Delta region in Botswana. It was designed according to requirements of land planners and managers and implemented using ArcGIS Viewer for Flex. Land planners and managers from two organisations in Maun involved in the development of the MDP were asked to evaluate the web mapping tools. This paper describes the results of implementation and some preliminary results of the web mapping evaluation

Low-latency, query-driven analytics over voluminous multidimensional, spatiotemporal datasets

2017 Summer.Includes bibliographical references.Ubiquitous data collection from sources such as remote sensing equipment, networked observational devices, location-based services, and sales tracking has led to the accumulation of voluminous datasets; IDC projects that by 2020 we will generate 40 zettabytes of data per year, while Gartner and ABI estimate 20-35 billion new devices will be connected to the Internet in the same time frame. The storage and processing requirements of these datasets far exceed the capabilities of modern computing hardware, which has led to the development of distributed storage frameworks that can scale out by assimilating more computing resources as necessary. While challenging in its own right, storing and managing voluminous datasets is only the precursor to a broader field of study: extracting knowledge, insights, and relationships from the underlying datasets. The basic building block of this knowledge discovery process is analytic queries, encompassing both query instrumentation and evaluation. This dissertation is centered around query-driven exploratory and predictive analytics over voluminous, multidimensional datasets. Both of these types of analysis represent a higher-level abstraction over classical query models; rather than indexing every discrete value for subsequent retrieval, our framework autonomously learns the relationships and interactions between dimensions in the dataset (including time series and geospatial aspects), and makes the information readily available to users. This functionality includes statistical synopses, correlation analysis, hypothesis testing, probabilistic structures, and predictive models that not only enable the discovery of nuanced relationships between dimensions, but also allow future events and trends to be predicted. This requires specialized data structures and partitioning algorithms, along with adaptive reductions in the search space and management of the inherent trade-off between timeliness and accuracy. The algorithms presented in this dissertation were evaluated empirically on real-world geospatial time-series datasets in a production environment, and are broadly applicable across other storage frameworks

SYSTEM ARCHITECTURE FOR GEOSPATIAL VIRTUAL DATA INTEGRATION IN WEB-BASED APPLICATIONS

The wide availability of geospatial data from different sources makes it necessary to create systems that are able to use and integrate the data to generate added value. We propose a system architecture following FAIR principles (Findable, Accessible, Interoperable, Reusable) and state-of-the-art methodologies for a server-side web-based application that performs virtual data integration over data sources that implement geospatial information standards. The architecture extends the mediator-wrapper design pattern with additional components that provide the system with additional flexibility and modularity, much needed for modern web applications. The architecture is composed of the mask, which acts as the interface of the system towards external users; a mediator that handles processing and data integration logic; a set of wrappers that communicate with the external data sources; persistent storage to provide flexible configuration and metadata capabilities to the system; and messaging queue for enabling asynchronous processing. At the same time, the architecture’s components are divided into four layers, each one with a specific role: presentation, configuration, processing, and communication

BUILDING A DIGITAL TWIN OF THE ITALIAN COASTS

Abstract. The "Destination Earth" initiative of the European Union encompasses the creation of Digital Twin Earths (DTEs), high-precision digital models of the Earth integrating various aspects of the Earth's system to monitor and simulate natural phenomena and related human activities, being able to explore the past, understand the present, and build predictive models of the future. To achieve this goal, huge amounts of good-quality data are necessary, but also, means to combine and add further utility to them.To tackle this problem, we created a novel web application that implements the mediator-wrapper architecture as a data integration strategy and uses only open-source software to put together more than 60 geospatial layers from 3 different data sources. This application is a proof of concept of how data integration can be used to implement Digital Twins and is focused solely on the Italian coasts. It integrates data from Copernicus and WorldPop to provide tools for analysing and describing the interaction of marine, land, and demographic variables on coastal areas. It offers both visualization and analysis capabilities, which is a unique feature amongst similar applications, thanks to the implementation of virtual data integration and geospatial data standards

An enhanced dynamic replica creation and eviction mechanism in data grid federation environment

Data Grid Federation system is an infrastructure that connects several grid systems, which facilitates sharing of large amount of data, as well as storage and computing resources. The existing mechanisms on data replication focus on finding file values based on the number of files access in deciding which file to replicate, and place new replicas on locations that provide minimum read cost. DRCEM finds file values based on logical dependencies in deciding which file to replicate, and allocates new replicas on locations that provide minimum replica placement cost. This thesis presents an enhanced data replication strategy known as Dynamic Replica Creation and Eviction Mechanism (DRCEM) that utilizes the usage of data grid resources, by allocating appropriate replica sites around the federation. The proposed mechanism uses three schemes: 1) Dynamic Replica Evaluation and Creation Scheme, 2) Replica Placement Scheme, and 3) Dynamic Replica Eviction Scheme. DRCEM was evaluated using OptorSim network simulator based on four performance metrics: 1) Jobs Completion Times, 2) Effective Network Usage, 3) Storage Element Usage, and 4) Computing Element Usage. DRCEM outperforms ELALW and DRCM mechanisms by 30% and 26%, in terms of Jobs Completion Times. In addition, DRCEM consumes less storage compared to ELALW and DRCM by 42% and 40%. However, DRCEM shows lower performance compared to existing mechanisms regarding Computing Element Usage, due to additional computations of files logical dependencies. Results revealed better jobs completion times with lower resource consumption than existing approaches. This research produces three replication schemes embodied in one mechanism that enhances the performance of Data Grid Federation environment. This has contributed to the enhancement of the existing mechanism, which is capable of deciding to either create or evict more than one file during a particular time. Furthermore, files logical dependencies were integrated into the replica creation scheme to evaluate data files more accurately

1st INCF Workshop on Sustainability of Neuroscience Databases

The goal of the workshop was to discuss issues related to the sustainability of neuroscience databases, identify problems and propose solutions, and formulate recommendations to the INCF. The report summarizes the discussions of invited participants from the neuroinformatics community as well as from other disciplines where sustainability issues have already been approached. The recommendations for the INCF involve rating, ranking, and supporting database sustainability

Using GHSL to Analyze Urbanization and Land-Use Efficiency in the Philippines from 1975-2020: Trends and Implications for Sustainable Development

This study analyzed the trends and patterns of urbanization and changes in land-use efficiency in the Philippines from 1975-2020 using the Global Human Settlement Layers (GHSL). Utilizing the GHS-BUILT-S, GHS POP, and GHS-SMOD raster datasets from the GHSL Data Package 2023, we examined the spatiotemporal expansion of built-up areas and the growth of population in urban and rural regions of the country. Using the same datasets, we also measured the country's achievement of Sustainable Development Goal (SDG)11.3, particularly on inclusive and sustainable urbanization through efficient land utilization, by computing the ratio of land consumption rate (LCR) to the population growth rate (PGR), also known as LCRPGR. The results of our analysis revealed an increasing trend in the overall built-up area and population of the Philippines within the examined period. Built-up areas and population in urban regions more than tripled in size from 1975 to 2020, demonstrating a notable shift towards more urbanized regions over time. In addition to presenting evidence of the Philippines' developmental progress and urbanization, our analysis of GHSL data shows a decline in land consumption, a deceleration in population growth, and an overall enhancement in land-use efficiency within the country. These findings suggest a shift towards more controlled and sustainable land development practices, supporting the country's goal of sustainable urbanization and land management. The implications of these findings are crucial for policymakers and urban planners in the Philippines, offering valuable insights to guide the formulation of effective and comprehensive land management strategies. Further work includes conducting localized analyses at the city or municipality level to provide valuable insights into the unique urbanization patterns and land-use dynamics across different islands and regions, enabling tailored policy interventions and spatial planning strategies to promote sustainable development