Design of a Metadata Framework for the Environmental Models with an Example Hydrologic Application in HydroShare

Environmental modelers rely on a variety of computational models to make predictions, test hypotheses, and address specific problems related to environmental science and natural resource management. Scientists and engineers must devote significant effort to preparing these computational models. While significant attention has been devoted to sharing and reusing environmental data, less attention has been devoted to sharing and reusing environmental models. A first step toward increasing environmental model sharing and reuse is to define a general metadata framework for models that is flexible and, therefore, applicable across the wide variety of models used by environmental modelers. This paper proposes a general approach for representing environmental model metadata that extends the Dublin Core metadata framework. The framework is implemented within the HydroShare system and applied for a hydrologic model sharing use case. This example application demonstrates how the metadata framework implemented within HydroShare can assist in model sharing, publication, reuse, and reproducibility

Integrating Hydrologic Modeling Web Services With Online Data Sharing to Prepare, Store, and Execute Hydrologic Models

Web based applications, web services, and online data and model sharing technology are becoming increasingly available to support hydrologic research. This promises benefits in terms of collaboration, computer platform independence, and reproducibility of modeling workflows and results. In this research, we designed an approach that integrates hydrologic modeling web services with an online data sharing system to support web-based simulation for hydrologic models. We used this approach to integrate example systems as a case study to support reproducible snowmelt modeling for a test watershed in the Colorado River Basin, USA. We demonstrated that this approach enabled users to work within an online environment to create, describe, share, discover, repeat, modify, and analyze the modeling work. This approach encourages collaboration and improves research reproducibility. It can also be adopted or adapted to integrate other hydrologic modeling web services with data sharing systems for different hydrologic models

Toward Open and Reproducible Environmental Modeling by Integrating Online Data Repositories, Computational Environments, and Model Application Programming Interfaces

Cyberinfrastructure needs to be advanced to enable open and reproducible environmental modeling research. Recent efforts toward this goal have focused on advancing online repositories for data and model sharing, online computational environments along with containerization technology and notebooks for capturing reproducible computational studies, and Application Programming Interfaces (APIs) for simulation models to foster intuitive programmatic control. The objective of this research is to show how these efforts can be integrated to support reproducible environmental modeling. We present first the high-level concept and general approach for integrating these three components. We then present one possible implementation that integrates HydroShare (an online repository), CUAHSI JupyterHub and CyberGIS-Jupyter for Water (computational environments), and pySUMMA (a model API) to support open and reproducible hydrologic modeling. We apply the example implementation for a hydrologic modeling use case to demonstrate how the approach can advance reproducible environmental modeling through the seamless integration of cyberinfrastructure services

Advancing Cyberinfrastructure for Collaborative Data Sharing and Modeling in Hydrology

Hydrologic research is increasingly data and computationally intensive, and often involves hydrologic model simulation and collaboration among researchers. With the development of cyberinfrastructure, researchers are able to improve the efficiency, impact, and effectiveness of their research by utilizing online data sharing and hydrologic modeling functionality. However, further efforts are still in need to improve the capability of cyberinfrastructure to serve the hydrologic science community. This dissertation first presents the evaluation of a physically based snowmelt model as an alternative to a temperature index model to improve operational water supply forecasts in the Colorado River Basin. Then it presents the design of the functionality to share multidimensional space-time data in the HydroShare hydrologic information system. It then describes a web application developed to facilitate input preparation and model execution of a snowmelt model and the storage of these results in HydroShare. The snowmelt model evaluation provided use cases to evaluate the cyberinfrastructure elements developed. This research explored a new approach to advance operational water supply forecasts and provided potential solutions for the challenges associated with the design and implementation of cyberinfrastructure for hydrologic data sharing and modeling

A Resource Centric Approach For Advancing Collaboration Through Hydrologic Data And Model Sharing

HydroShare is an online, collaborative system being developed for open sharing of hydrologic data and models. The goal of HydroShare is to enable scientists to easily discover and access hydrologic data and models, retrieve them to their desktop or perform analyses in a distributed computing environment that may include grid, cloud or high performance computing model instances as necessary. Scientists may also publish outcomes (data, results or models) into HydroShare, using the system as a collaboration platform for sharing data, models and analyses. HydroShare is expanding the data sharing capability of the CUAHSI Hydrologic Information System by broadening the classes of data accommodated, creating new capability to share models and model components, and taking advantage of emerging social media functionality to enhance information about and collaboration around hydrologic data and models. One of the fundamental concepts in HydroShare is that of a Resource. All content is represented using a Resource Data Model that separates system and science metadata and has elements common to all resources as well as elements specific to the types of resources HydroShare will support. These will include different data types used in the hydrology community and models and workflows that require metadata on execution functionality. The HydroShare web interface and social media functions are being developed using the Drupal content management system. A geospatial visualization and analysis component enables searching, visualizing, and analyzing geographic datasets. The integrated Rule-Oriented Data System (iRODS) is being used to manage federated data content and perform rule-based background actions on data and model resources, including parsing to generate metadata catalog information and the execution of models and workflows. This presentation will introduce the HydroShare functionality developed to date, describe key elements of the Resource Data Model and outline the roadmap for future development

Advancing Water Resources Systems Modeling Cyberinfrastructure to Enable Systematic Data Analysis, Modeling, and Comparisons

Water resources systems models aid in managing water resources holistically considering water, economic, energy, and environmental needs, among others. Developing such models require data that represent a water system’s physical and operational characteristics such as inflows, demands, reservoir storage, and release rules. However, such data is stored and described in different formats, metadata, and terminology. Therefore, Existing tools to store, query, and visualize modeling data are model, location, and dataset-specific, and developing such tools is time-consuming and requires programming experience. This dissertation presents an architecture and three software tools to enable researchers to more readily and consistently prepare and reuse data to develop, compare, and synthesize results from multiple models in a study area: (1) a generalized database design for consistent organization and storage of water resources datasets independent of study area or model, (2) software to extract data out of and populate data for any study area into the Water Evaluation and Planning system, and (3) software tools to visualize online, compare, and publish water management networks and their data for many models and study areas. The software tools are demonstrated using dozens of example and diverse local, regional, and national datasets from three watersheds for four models; the Bear and Weber Rivers in the USA and the Monterrey River in Mexico

Advancing the Implementation of Hydrologic Models as Web-Based Applications

Deeper understanding of relationships between flow in river sand various hydrologic elements such as rainfall, land use, and soil type is imperative to solve water related problems like droughts and floods. Advanced computer models are becoming essential in helping us understand such relationships. However, preparing such models requires huge investment of time and resources, much of which are concentrated on acquisition and curation of data. This work introduces agree and open source web Application (web App) that provides researchers with simplified access to hydrological data and modeling functionality. The web App helps in the creation of both hydrologic models, and climatic and geographic data. Free and open source platforms such as Tethys and Hydro Share were used in the development of the web Apia physics based model called TOPographic Kinematic APproximation and Integration (TOPKAPI) was used as the driving use case for which a complete hydrologic modeling service was developed to demonstrate the approach. The final product is a complete modeling system accessible through the web to create hydrologic data and run a hydrologic model for a watershed of interest. An additional model, TOPNET, was incorporated to demonstrate the generality of the approach and capability for adding other models into the framework

Advancing Distributed Data Management for the HydroShare Hydrologic Information System

HydroShare (https://www.hydroshare.org) is an online collaborative system to support the open sharing of hydrologic data, analytical tools, and computer models. Hydrologic data and models are often large, extending to multi-gigabyte or terabyte scale, and as a result, the scalability of centralized data management poses challenges for a system such as HydroShare. A distributed data management framework that enables distributed physical data storage and management in multiple locations thus becomes a necessity. We use the iRODS (Integrated Rule-Oriented Data System) data grid middleware as the distributed data storage and management back end in HydroShare. iRODS provides a unified virtual file system for distributed physical storages in multiple locations and enables data federation across geographically dispersed institutions around the world. In this paper, we describe the iRODS-based distributed data management approaches implemented in HydroShare to provide a practical demonstration of a production system for supporting big data in the environmental sciences

HydroShare GIS: Visualizing Spatial Data in the Cloud

Cloud-based data management systems are more conducive to collaborative efforts when they are integrated with cloud-computing tools that interact with their stored data. HydroShare, a web based data management system for climate and water data, has implemented an Application Programming Interface and a web application platform deployed using Tethys Platform to encourage the development of apps that interact with its data. HydroShare GIS is the result of one such development effort to provide cloud-based visualization of spatial data stored in HydroShare. It functions by accessing the spatial metadata contained within the HydroShare resource data model and overlaying datasets as layers within the OpenLayers JavaScript library. Data are passed from the app’s server to a GeoServer data server and shared as web mapping service layers. Thus, users can easily build map projects from data sources registered in HydroShare and save them back to HydroShare as map project resources, which can both be shared with others and re-opened in HydroShare GIS. This paper will describe the design of the HydroShare GIS app and the cyber-infrastructure that supports it, and evaluate its efficacy as a web based mapping tool

Enabling collaborative numerical modeling in earth sciences using knowledge infrastructure

Knowledge Infrastructure is an intellectual framework for creating, sharing, and distributing knowledge. In this paper, we use Knowledge Infrastructure to address common barriers to entry to numerical modeling in Earth sciences: computational modeling education, replicating published model results, and reusing published models to extend research. We outline six critical functional requirements: 1) workflows designed for new users; 2) a community-supported collaborative web platform; 3) distributed data storage; 4) a software environment; 5) a personalized cloud-based high-performance computing platform; and 6) a standardized open source modeling framework. Our methods meet these functional requirements by providing three interactive computational narratives for hands-on, problem-based research demonstrating how to use Landlab on HydroShare. Landlab is an open-source toolkit for building, coupling, and exploring two-dimensional numerical models. HydroShare is an online collaborative environment for the sharing of data and models. We describe the methods we are using to accelerate knowledge development by providing a suite of modular and interoperable process components that allows students, domain experts, collaborators, researchers, and sponsors to learn by exploring shared data and modeling resources. The system is designed to support uses on the continuum from fully-developed modeling applications to prototyping research software tools