Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations

Earth Observation data are a vital resource for studying long term changes, but the large data volumes can be challenging to analyze. Time series analysis in particular is hampered by the typical thin-time-slice file organization. We examine several potential solutions inspired in large part by the data-parallel methods that have arisen with cloud computing. These solutions include various combinations of data re-organization, spatial indexing, distributed storage and pre-computation that we term "Analytics Optimized Data Stores" (AODS). We find that even simple solutions (such as a data cube) produce more than an order of magnitude improvement; the best provide two to three orders of magnitude improvement. The most performant solutions have tradeoffs in terms of generality or storage footprint, but may nonetheless be useful components in data analytics frameworks where performance is critical

Distributed Geospatial Information Service (Distributed GIService)

Distributed Geospatial Information Service (Distributed GIService) in this context refers to an emerging paradigm for offering geospatial data and processing services by using distributed computing technologies. Through on-line access and integration of distributed data and geoprocessing services, Distributed GIService provides reduced technology risk, better ability to leverage the value of legacy data and systems, and more efficient information sharing. Development of Distributed GIService is largely driven by 1) the increasing use of serviceoriented architecture (SOA), 2) the adoption of interoperable standards for sharing geospatial information resources, 3) the fast development of enabling distributed computing technologies, (e.g., agent, grid, and P2P) for transparent and reliable access to computing infrastructure. This paper provides practitioners with an introduction to Distributed GIService and discusses some fundamental issues in Distributed GIService from a geospatial computing perspective

Yang et al, 2006, Journal of Geographic Information Sciences, 12(1):38-43 Spatial Web Portal for Building Spatial Data Infrastructure

Abstract: The past decades have witnessed the rapid growth of heterogeneous geospatial information systems. An important way to share these valuable assets is a spatial data infrastructure (SDI). Recent developments in Web Services and distributed geospatial information services 1 provide a practical approach, Web Portals, to building a SDI. This article describes research, development, and challenges related to Web Portals for SDI

COVID-Scraper: An Open-Source Toolset for Automatically Scraping and Processing Global Multi-Scale Spatiotemporal COVID-19 Records

In 2019, COVID-19 quickly spread across the world, infecting billions of people and disrupting the normal lives of citizens in every country. Governments, organizations, and research institutions all over the world are dedicating vast resources to research effective strategies to fight this rapidly propagating virus. With virus testing, most countries publish the number of confirmed cases, dead cases, recovered cases, and locations routinely through various channels and forms. This important data source has enabled researchers worldwide to perform different COVID-19 scientific studies, such as modeling this virus’s spreading patterns, developing prevention strategies, and studying the impact of COVID-19 on other aspects of society. However, one major challenge is that there is no standardized, updated, and high-quality data product that covers COVID-19 cases data internationally. This is because different countries may publish their data in unique channels, formats, and time intervals, which hinders researchers from fetching necessary COVID-19 datasets effectively, especially for fine-scale studies. Although existing solutions such as John’s Hopkins COVID-19 Dashboard and 1point3acres COVID-19 tracker are widely used, it is difficult for users to access their original dataset and customize those data to meet specific requirements in categories, data structure, and data source selection. To address this challenge, we developed a toolset using cloud-based web scraping to extract, refine, unify, and store COVID-19 cases data at multiple scales for all available countries around the world automatically. The toolset then publishes the data for public access in an effective manner, which could offer users a real time COVID-19 dynamic dataset with a global view. Two case studies are presented about how to utilize the datasets. This toolset can also be easily extended to fulfill other purposes with its open-source nature