Report on the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3)

This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics including developing pathways to funding scientific software; constructing useful common metrics for crediting software stakeholders; identifying principles for sustainable software engineering design; reaching out to research software organizations around the world; and building communities for software sustainability. For each group, we include a point of contact and a landing page that can be used by those who want to join that group's future activities. The main challenge left by the workshop is to see if the groups will execute these activities that they have scheduled, and how the WSSSPE community can encourage this to happen

Fourth Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE4)

This report records and discusses the Fourth Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE4). The report includes a description of the keynote presentation of the workshop, the mission and vision statements that were drafted at the workshop and finalized shortly after it, a set of idea papers, position papers, experience papers, demos, and lightning talks, and a panel discussion. The main part of the report covers the set of working groups that formed during the meeting, and for each, discusses the participants, the objective and goal, and how the objective can be reached, along with contact information for readers who may want to join the group. Finally, we present results from a survey of the workshop attendees

Report on the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3)

This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics including developing pathways to funding scientific software; constructing useful common metrics for crediting software stakeholders; identifying principles for sustainable software engineering design; reaching out to research software organizations around the world; and building communities for software sustainability. For each group, we include a point of contact and a landing page that can be used by those who want to join that group’s future activities. The main challenge left by the workshop is to see if the groups will execute these activities that they have scheduled, and how the WSSSPE community can encourage this to happe

Report on the 3rd Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3)

This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics including developing pathways to funding scientific software; constructing useful common metrics for crediting software stakeholders; identifying principles for sustainable software engineering design; reaching out to research software organizations around the world; and building communities for software sustainability. For each group, we include a point of contact and a landing page that can be used by those who want to join that group's future activities. The main challenge left by the workshop is to see if the groups will execute these activities that they have scheduled, and how the WSSSPE community can encourage this to happen

HydroShare: Advancing Hydrology through Collaborative Data and Model Sharing

Advancing Hydrologic Understanding - requires integration of information from multiple sources - is data and computationally intensive - requires collaboration and working as a team/communit

A Flexible File Sharing Mechanism for iRODS

The traditional iRODS mechanisms for file sharing, including user groups, often require some form of iRODS administrative privilege. In the HydroShare project for enabling hydrology research, we perceived a need for more flexible file sharing, including unprivileged creation and management of user groups according to policies quite distinct from the Linux/Unix policies that initially motivated iRODS protections. This is enabled by a policy database in PostgreSQL and management API written in Python that are deployed in parallel to iCAT. Innovations in iRODS 4.1 allow access control based upon this PostgreSQL database rather than the default iCAT server, by interposing access control code before the access event using iRODS Policy Enforcement Points. The result is an access control mechanism that closely matches scientific needs for file sharing, and brings “dropbox-like” file sharing semantics to the network filesystem level

The Essential Terrestrial Variables (ETV’s) in Support of a National Framework for Numerical Watershed Prediction (Invited)

There is a clear national need to provide geoscience researchers with seamless and fast access to essential geo-spatial/geo-temporal data to support physics-based numerical models necessary to understand, predict and manage the nations surface and groundwater resources. Fundamental advances in science such as the evaluation of ecosystem and watershed services, the detection and attribution of the impact of climatic change, represent examples that will require high resolution spatially explicit assessments. In this paper we propose the concept of Essential Terrestrial Variables (ETV\u27s), which we define as those variables that are nominally required to support watershed/catchment numerical prediction anywhere in the continental US and ultimately at the global scale. ETV\u27s would represent a fundamental community resource necessary to build the products/parameters/forcings commonly used in distributed, fully-coupled watershed and river basin models. We argue that there are at last 3 fundamental issues that must be resolved before implementation of ETV\u27s in support of a national water model: 1) data access and accessibility, 2) data scale and scalability, 3) community provenance and data sustainability. At the present time, there is no unified data infrastructure for supporting watershed models, and the data resource itself (weather/climate reanalysis products, stream flow, groundwater, soils, land cover, satellite data products, etc.) resides on many federal servers with limited or poorly organized access, with many data formats and without common geo-referencing. Beyond the problem of access to national data, the scale and scalability of computation for both data processing and model computational represents a major hurdle. This predicament is especially true since a full-scale national strategy for numerical watershed prediction will require data resources to reside very close to numerical model computation. Finally model/data provenance should be sufficient to allow reproducible results and scientific workflows that support continuous data and model tracking, geo-referencing, and general support for model/data reproducibility, data analytics and visualization. Furthermore, these workflows should be readily publishable and discoverable by the broader community with provisions for community comment on workflow utility, effectiveness, and improvement. US policy on access to basic data is generally enlightened in that most national geospatial data can be acquired. However, it is clear that fast and efficient access to all the data and models is not yet available to the scientific community for systems that support integrated watershed modeling. Adoption of ETV\u27s and the concomitant supporting cyber infrastructure would be an important step towards a national framework for numerical watershed prediction

Collaborative Sharing of Multidimensional Space-time Data Using HydroShare

HydroShare is a collaborative environment being developed for sharing hydrological data and models. It includes capability to upload data in many formats as resources that can be shared. The HydroShare data model for resources uses a specific format for the representation of each type of data and specifies metadata common to all resource types as well as metadata unique to specific resource types. The Network Common Data Form (NetCDF) was chosen as the format for multidimensional space-time data in HydroShare. NetCDF is widely used in hydrological and other geoscience modeling because it contains self-describing metadata and supports the creation of array-oriented datasets that may include three spatial dimensions, a time dimension and other user defined dimensions. For example, NetCDF may be used to represent precipitation or surface air temperature fields that have two dimensions in space and one dimension in time. This presentation will illustrate how NetCDF files are used in HydroShare. When a NetCDF file is loaded into HydroShare, header information is extracted using the ncdump utility. Python functions developed for the Django web framework on which HydroShare is based, extract science metadata present in the NetCDF file, saving the user from having to enter it. Where the file follows Climate Forecast (CF) convention and Attribute Convention for Dataset Discovery (ACDD) standards, metadata is thus automatically populated. Users also have the ability to add metadata to the resource that may not have been present in the original NetCDF file. HydroShare\u27s metadata editing functionality then writes this science metadata back into the NetCDF file to maintain consistency between the science metadata in HydroShare and the metadata in the NetCDF file. This further helps researchers easily add metadata information following the CF and ACDD conventions. Additional data inspection and subsetting functions were developed, taking advantage of Python and command line libraries for working with NetCDF files. We describe the design and implementation of these features and illustrate how NetCDF files from a modeling application may be curated in HydroShare and thus enhance reproducibility of the associated research. We also discuss future development planned for multidimensional space-time data in HydroShare