Integrating Hydrologic Modeling Web Services With Online Data Sharing to Prepare, Store, and Execute Hydrologic Models

Web based applications, web services, and online data and model sharing technology are becoming increasingly available to support hydrologic research. This promises benefits in terms of collaboration, computer platform independence, and reproducibility of modeling workflows and results. In this research, we designed an approach that integrates hydrologic modeling web services with an online data sharing system to support web-based simulation for hydrologic models. We used this approach to integrate example systems as a case study to support reproducible snowmelt modeling for a test watershed in the Colorado River Basin, USA. We demonstrated that this approach enabled users to work within an online environment to create, describe, share, discover, repeat, modify, and analyze the modeling work. This approach encourages collaboration and improves research reproducibility. It can also be adopted or adapted to integrate other hydrologic modeling web services with data sharing systems for different hydrologic models

Advancing Cyberinfrastructure for Collaborative Data Sharing and Modeling in Hydrology

Hydrologic research is increasingly data and computationally intensive, and often involves hydrologic model simulation and collaboration among researchers. With the development of cyberinfrastructure, researchers are able to improve the efficiency, impact, and effectiveness of their research by utilizing online data sharing and hydrologic modeling functionality. However, further efforts are still in need to improve the capability of cyberinfrastructure to serve the hydrologic science community. This dissertation first presents the evaluation of a physically based snowmelt model as an alternative to a temperature index model to improve operational water supply forecasts in the Colorado River Basin. Then it presents the design of the functionality to share multidimensional space-time data in the HydroShare hydrologic information system. It then describes a web application developed to facilitate input preparation and model execution of a snowmelt model and the storage of these results in HydroShare. The snowmelt model evaluation provided use cases to evaluate the cyberinfrastructure elements developed. This research explored a new approach to advance operational water supply forecasts and provided potential solutions for the challenges associated with the design and implementation of cyberinfrastructure for hydrologic data sharing and modeling

Advancing the Implementation of Hydrologic Models as Web-Based Applications

Deeper understanding of relationships between flow in river sand various hydrologic elements such as rainfall, land use, and soil type is imperative to solve water related problems like droughts and floods. Advanced computer models are becoming essential in helping us understand such relationships. However, preparing such models requires huge investment of time and resources, much of which are concentrated on acquisition and curation of data. This work introduces agree and open source web Application (web App) that provides researchers with simplified access to hydrological data and modeling functionality. The web App helps in the creation of both hydrologic models, and climatic and geographic data. Free and open source platforms such as Tethys and Hydro Share were used in the development of the web Apia physics based model called TOPographic Kinematic APproximation and Integration (TOPKAPI) was used as the driving use case for which a complete hydrologic modeling service was developed to demonstrate the approach. The final product is a complete modeling system accessible through the web to create hydrologic data and run a hydrologic model for a watershed of interest. An additional model, TOPNET, was incorporated to demonstrate the generality of the approach and capability for adding other models into the framework

Enabling collaborative numerical modeling in earth sciences using knowledge infrastructure

Knowledge Infrastructure is an intellectual framework for creating, sharing, and distributing knowledge. In this paper, we use Knowledge Infrastructure to address common barriers to entry to numerical modeling in Earth sciences: computational modeling education, replicating published model results, and reusing published models to extend research. We outline six critical functional requirements: 1) workflows designed for new users; 2) a community-supported collaborative web platform; 3) distributed data storage; 4) a software environment; 5) a personalized cloud-based high-performance computing platform; and 6) a standardized open source modeling framework. Our methods meet these functional requirements by providing three interactive computational narratives for hands-on, problem-based research demonstrating how to use Landlab on HydroShare. Landlab is an open-source toolkit for building, coupling, and exploring two-dimensional numerical models. HydroShare is an online collaborative environment for the sharing of data and models. We describe the methods we are using to accelerate knowledge development by providing a suite of modular and interoperable process components that allows students, domain experts, collaborators, researchers, and sponsors to learn by exploring shared data and modeling resources. The system is designed to support uses on the continuum from fully-developed modeling applications to prototyping research software tools

Design of a Metadata Framework for the Environmental Models with an Example Hydrologic Application in HydroShare

Environmental modelers rely on a variety of computational models to make predictions, test hypotheses, and address specific problems related to environmental science and natural resource management. Scientists and engineers must devote significant effort to preparing these computational models. While significant attention has been devoted to sharing and reusing environmental data, less attention has been devoted to sharing and reusing environmental models. A first step toward increasing environmental model sharing and reuse is to define a general metadata framework for models that is flexible and, therefore, applicable across the wide variety of models used by environmental modelers. This paper proposes a general approach for representing environmental model metadata that extends the Dublin Core metadata framework. The framework is implemented within the HydroShare system and applied for a hydrologic model sharing use case. This example application demonstrates how the metadata framework implemented within HydroShare can assist in model sharing, publication, reuse, and reproducibility

A CyberGIS Integration and Computation Framework for High‐Resolution Continental‐Scale Flood Inundation Mapping

We present a Digital Elevation Model (DEM)-based hydrologic analysis methodology for continental flood inundation mapping (CFIM), implemented as a cyberGIS scientific workflow in which a 1/3rd arc-second (10m) Height Above Nearest Drainage (HAND) raster data for the conterminous U.S. (CONUS) was computed and employed for subsequent inundation mapping. A cyberGIS framework was developed to enable spatiotemporal integration and scalable computing of the entire inundation mapping process on a hybrid supercomputing architecture. The first 1/3rd arc-second CONUS HAND raster dataset was computed in 1.5 days on the CyberGIS ROGER supercomputer. The inundation mapping process developed in our exploratory study couples HAND with National Water Model (NWM) forecast data to enable near real-time inundation forecasts for CONUS. The computational performance of HAND and the inundation mapping process was profiled to gain insights into the computational characteristics in high-performance parallel computing scenarios. The establishment of the CFIM computational framework has broad and significant research implications that may lead to further development and improvement of flood inundation mapping methodologies

HydroDS: Data Services in Support of Physically Based, Distributed Hydrological Models

Physically based distributed hydrologic models require geospatial and time-series data that take considerable time and effort in processing them into model inputs. Tools that automate and speed up input processing facilitate the application of these models. In this study, we developed a set of web-based data services called HydroDS to provide hydrologic data processing ‘software as a service.’ HydroDS provides functions for processing watershed, terrain, canopy, climate, and soil data. The services are accessed through a Python client library that facilitates developing simple but effective data processing workflows with Python. Evaluations of HydroDS by setting up the Utah Energy Balance and TOPNET models for multiple headwater watersheds in the Colorado River basin show that HydroDS reduces the input preparation time compared to manual processing. It also removes the requirements for software installation and maintenance by the user, and the Python workflows enhance reproducibility of hydrologic data processing and tracking of provenance

An Architectural Overview Of HydroShare, A Next-Generation Hydrologic Information System

HydroShare is an online, open-source, collaborative system being developed for sharing hydrologic data and models as part of the NSF’s Software Infrastructure for Sustained Innovation (SI2) program. The goal of HydroShare is to enable scientists to easily discover and access hydrologic data and models, retrieve them to their desktop, or perform analyses in a distributed computing environment that may include grid, cloud, or high performance computing. Scientists may also publish outcomes (data, results or models) into HydroShare, using the system as a collaboration platform for sharing data, models, and analyses. HydroShare involves a large distributed software development effort requiring collaboration between domain scientists, software engineers, and software developers across eight U.S. universities, RENCI, and CUAHSI. HydroShare expands the data sharing capabilities of the Hydrologic Information System of the Consortium of Universities for the Advancement of Hydrologic Sciences, Inc. (CUAHSI): It broadens the classes of data accommodated, enables sharing of models and model components, and leverages social media functionality to enhance collaboration around hydrologic data and models. The HydroShare architecture is a stack of storage and computation, web services, and user applications. A content management system, Django+Mezzanine, provides user interface, search, social media functions, and services. A geospatial visualization and analysis component enables searching, visualizing, and analyzing geographic datasets. The integrated Rule-Oriented Data System (iRODS) is used to manage federated data content and perform rule-based background actions on data and model resources, including parsing to generate metadata catalog information and the distributed execution of models and workflows. A web browser is the main interface to HydroShare, however a web services applications programming interface (API) supports access through HydroDesktop and other hydrologic modeling systems, and the architecture separates the interface layer and services layer exposing all functionality through these web services. This presentation will describe key components of HydroShare and discuss how HydroShare is designedto enable better hydrologic science concomitant with sustainable open-source software practices

HydroShare: Sharing Diverse Environmental Data Types and Models as Social Objects with Application to the Hydrology Domain

The types of data and models used within the hydrologic science community are diverse. New repositories have succeeded in making data and models more accessible, but are, in most cases, limited to particular types or classes of data or models and also lack the type of collaborative and iterative functionality needed to enable shared data collection and modeling workflows. File sharing systems currently used within many scientific communities for private sharing of preliminary and intermediate data and modeling products do not support collaborative data capture, description, visualization, and annotation. In this article, we cast hydrologic datasets and models as “social objects” that can be published, collaborated around, annotated, discovered, and accessed. This article describes the generic data model and content packaging scheme for diverse hydrologic datasets and models used by a new hydrologic collaborative environment called HydroShare to enable storage, management, sharing, publication, and annotation of the diverse types of data and models used by hydrologic scientists. The flexibility of HydroShare\u27s data model and packaging scheme is demonstrated using multiple hydrologic data and model use cases that highlight its features

Toward Open and Reproducible Environmental Modeling by Integrating Online Data Repositories, Computational Environments, and Model Application Programming Interfaces

Cyberinfrastructure needs to be advanced to enable open and reproducible environmental modeling research. Recent efforts toward this goal have focused on advancing online repositories for data and model sharing, online computational environments along with containerization technology and notebooks for capturing reproducible computational studies, and Application Programming Interfaces (APIs) for simulation models to foster intuitive programmatic control. The objective of this research is to show how these efforts can be integrated to support reproducible environmental modeling. We present first the high-level concept and general approach for integrating these three components. We then present one possible implementation that integrates HydroShare (an online repository), CUAHSI JupyterHub and CyberGIS-Jupyter for Water (computational environments), and pySUMMA (a model API) to support open and reproducible hydrologic modeling. We apply the example implementation for a hydrologic modeling use case to demonstrate how the approach can advance reproducible environmental modeling through the seamless integration of cyberinfrastructure services