Peters, Debra

Savoy, Heathe

Digitale Bibliothek Thüringen

Automating the integration and accessibility of diverse geospatial data cubesHeather M. Savoy1 and Debra P.C. Peters21 New Mexico State University, Jornada Experimental Range, and Jornada Basin Long Term Ecological Research Program, Las Cruces, NM, USA2 USDA ARS, Jornada Experimental Range, and Jornada Basin Long Term Ecological Research Program , Las Cruces, NM, USADASH: Data Access and Spatiotemporal Harmonization Continental-scale geospatial data, such as dailyprecipitation or soil properties across North America,have the capability to serve the data needs of a widevariety of ecological projects, but their cost in terms oftime and memory can hinder efficient integration. Toreduce the time of downloading big data sets fromonline repositories, cropping the data to desiredextents, harmonizing the resolutions and coordinatereference systems, and performing statistical analysesfor multiple data sets, we aim to create a portal thatautomates these steps.ObjectiveTools• Overview statistics about the data hosted are generated dynamically from the current metadata database and present files  • Number of datasets• Number of built-in boundaries/shapefiles• Total amount of data hosted in GB• The sidebar on the left provides navigation and is populated with data selections as the user performs tasks in the following screensMotivation: Ecological StudiesEcology of Complex Disease Systems• A USDA Grand Challenge project was created to use VesicularStomatitis Virus (VSV) as a model for predictive disease ecology.• The disease ecology system for VSV is a complex network ofinteractions among the virus, multiple insect vectors, multiple livestockhosts, and the environment. A big data-model integration approach isbeing used for this transdisciplinary project (Peters et al., 2018).• Predictive models are being built to serve as an early warning system(Peters et al., submitted), and these models incorporate manygeospatial data layers, such as:• Climate (precipitation, temperature) [PRISM, NOAA]• Vegetation [LANDSAT]• Horse density [USDA]• Distance to streams with water [USGS]• The DASH portal will serve as a data repository for this project to helpscientists to access both the original and processed geospatial datalayers used in this project.ComparisonsHome Page Dataset Explorer Data SelectionExport ToolData Cube ViewMap ViewEcology of the Dust Bowl• This study is using historical agricultural data to predict ecologicalconsequences of changes in multi-scale connectivity, climate, andland use.• Data records from the 1933-1939 large-scale drought in the CentralUS, as well as before (1926-1932) and after (1940-1946), are beingused to relate changes in agricultural production to climate,landscape, and broad-scale features of the land surface.• Geospatial data involved include:• County-level corn yield and land use records• Climate (precipitation, temperature)• Soil texture properties• Simulated soil water content• Broad-scale dust and sand movement, and landform distribution• The DASH portal will host the organized historical data and thederived variables from SOILWAT simulations for use by otherscientists.• Search for any dataset, variable, or property available in the portal• The search is applied to the entire metadata database• A table is generated listing all datasets that match the search• The rows in the table can be selected to see formatted metadata about the dataset printed to the right• A button to use data from the selected dataset is available to partially pre-populate the Data Selection screen Web Interface• The portal is built as a Shiny app with the shinypackage in the R language (Chang et al., 2018; R CoreTeam, 2018). R provides data wrangling and statisticalcapabilities and Shiny allows for building web-basedinteractive tools to extend those capabilities.• The map visualization feature relies on the leafletpackage (Cheng et al., 2018). Leaflet is an opensourceJavaScript library for making interactive maps.• The data cube visualization feature relies on theplotly package (Sievert, 2018). Plotly is anopensource JavaScript library for making interactiveplots in general.Geospatial Processing• For smaller datasets, R functions are used fortransforming, cropping, etc.• For larger datasets, external calls to the Geospatial DataAbstraction Library (GDAL, 2018) are used for geospatialmanipulations for increasing computational efficiency.• Select data variables by class and/or type. Live search is supported, i.e. “temp” can be typed to reduce options to those related to temperature. • If more than one dataset contains that variable, potential datasets are listed for selection• If the variable has a temporal component, relevant temporal aspects will appear for selection• Add selection to the list, which appears on the left sidebar• Repeat as necessary until all desired data is selected• Display selected data as either original extent, or clipped to a boundary.• Boundaries can be chosen from the built-in library or uploaded• The background map changes its level of detail depending on the zoom level to provide spatial context. There are different map options to choose from. • The color scheme can be modified.• This view is best for visualizing one data layer at a time. • Display selected data layers in a stacked view to visualize the data cube.• The stack can be created with a single variable over time, multiple variables, or a combination. • Hovering provides data from every layer at that location. • This view supports interactive zooming and rotation to spin the data cube around for exploration. • This view is best for visualizing multiple data layers together.• Choose how the selected data should be exported.• Spatial options exist to choose file formats, spatial resolution, and coordinate reference system. • Temporal options exist to choose temporal aggregations, e.g. summing monthly precipitation into seasonal values.• There is an option to export a log of operations performed throughout the session and metadata related to the original and exported data. This feature is designed to help in writing the methods section for publications.ReferencesChang, W., Cheng, J., Allaire, J. J., Xie, Y., & McPherson, J. (2018). shiny:Web Application Framework for R. Retrieved from https://cran.r-project.org/package=shinyCheng, J., Karambelkar, B., & Xie, Y. (2018). leaflet: Create InteractiveWeb Maps with the JavaScript “Leaflet” Library. Retrieved fromhttps://cran.r-project.org/package=leafletGDAL. 2018. GDAL - Geospatial Data Abstraction Library: Version 2.2.4,Open Source Geospatial Foundation, http://gdal.osgeo.orgR Core Team. (2018). R: A language and environment for statisticalcomputing. Vienna, Austria: R Foundation for Statistical Computing.Retrieved from https://www.r-project.org/Sievert, C. (2018). plotly for R. Retrieved from https://plotly-book.cpsievert.mePeters, D.P.C., Burruss, N.D., Rodriguez, L.L., McVey, D.S., Elias, E.H.,Pelzel-McCluskey, A.M., Derner, J.D., Schrader, T.S., Yao, J., Pauszek,S.J., Lombard, J., Archer, S.R., Bestelmeyer, B.T., Browning, D.M.,Brungard, C.W., Hatfield, J.L., Hanan, N.P., Herrick, J.E., Okin, G.S.,Sala, O.E., Savoy, H.M. & Vivoni, E.R. (2018). An integrated view ofcomplex landscapes: a big data-model integration approach to trans-disciplinary science. BioScience, 68(9), 653–669.Peters, D.P.C, McVey, D.S., Elias, E.H., Pelzel-McCluskey, A.M., Derner,J.D., Burruss, N.D., Schrader, T.S., Yao, J., Pauszek, S.J., Lombard, J.& Rodriguez, L.L. (submitted). Developing an early warning strategy fora vector-borne disease using big data-model integration and machinelearning. Nature.• Geographic Information Systems (GIS) software programs exist to performmany of the operations in this tool, but the over-abundance of features can slowdown the programs. Processing large datasets in GIS can cause unreasonableprocessing times or software crashes. This portal strives to support scientistsneeding large datasets or who potentially do not have GIS experience.• Google Earth Engine (GEE) is a powerful tool for server-side geospatialprocessing and large dataset access. However, performing those processesrequires knowledge of a GEE variant of JavaScript or python. Download optionsare limited to Google Drive or Google Cloud, which could incur fees for largedatasets. This portal does not require any programming skill or fees.• Online Data Portals exist to host differentdatasets. For example, The National Map(https://nationalmap.gov/) provides multiple datalayers for the US, but they are topographyrelated and downloading multiple layers mayresult in different spatial extents. This portal willcontain data from many fields to supportcomplex, multi-faceted agro-ecosystem researchand provide fully harmonized data cubes fordownloading.Tools

Automating the integration and accessibility of diverse geospatial data cubes

https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00044040/Savoy_Poster_ICEI2018.pdf

Automating the integration and accessibility of diverse geospatial data cubes

Abstract

Similar works

Full text

Available Versions

Digitale Bibliothek Thüringen