Big Data Challenges in Climate Science: Improving the Next-Generation Cyberinfrastructure

The knowledge we gain from research in climate science depends on the generation, dissemination, and analysis of high-quality data. This work comprises technical practice as well as social practice, both of which are distinguished by their massive scale and global reach. As a result, the amount of data involved in climate research is growing at an unprecedented rate. Climate model intercomparison (CMIP) experiments, the integration of observational data and climate reanalysis data with climate model outputs, as seen in the Obs4MIPs, Ana4MIPs, and CREATE-IP activities, and the collaborative work of the Intergovernmental Panel on Climate Change (IPCC) provide examples of the types of activities that increasingly require an improved cyberinfrastructure for dealing with large amounts of critical scientific data. This paper provides an overview of some of climate science's big data problems and the technical solutions being developed to advance data publication, climate analytics as a service, and interoperability within the Earth System Grid Federation (ESGF), the primary cyberinfrastructure currently supporting global climate research activities

Advancing coastal ocean modelling, analysis, and prediction for the US Integrated Ocean Observing System

Author Posting. © The Author(s), 2017. This is the author's version of the work. It is posted here by permission of Taylor & Francis for personal use, not for redistribution. The definitive version was published in Journal of Operational Oceanography 10 (2017): 115-126, doi:10.1080/1755876X.2017.1322026.This paper outlines strategies that would advance coastal ocean modeling, analysis and prediction as a complement to the observing and data management activities of the coastal components of the U.S. Integrated Ocean Observing System (IOOS®) and the Global Ocean Observing System (GOOS). The views presented are the consensus of a group of U.S. based researchers with a cross-section of coastal oceanography and ocean modeling expertise and community representation drawn from Regional and U.S. Federal partners in IOOS. Priorities for research and development are suggested that would enhance the value of IOOS observations through model-based synthesis, deliver better model-based information products, and assist the design, evaluation and operation of the observing system itself. The proposed priorities are: model coupling, data assimilation, nearshore processes, cyberinfrastructure and model skill assessment, modeling for observing system design, evaluation and operation, ensemble prediction, and fast predictors. Approaches are suggested to accomplish substantial progress in a 3-8 year timeframe. In addition, the group proposes steps to promote collaboration between research and operations groups in Regional Associations, U.S. Federal Agencies, and the international ocean research community in general that would foster coordination on scientific and technical issues, and strengthen federal-academic partnerships benefiting IOOS stakeholders and end users.2018-05-2

Climate–ecosystem modelling made easy: The Land Sites Platform

Dynamic Global Vegetation Models (DGVMs) provide a state-of-the-art process-based approach to study the complex interplay between vegetation and its physical environment. For example, they help to predict how terrestrial plants interact with climate, soils, disturbance and competition for resources. We argue that there is untapped potential for the use of DGVMs in ecological and ecophysiological research. One fundamental barrier to realize this potential is that many researchers with relevant expertize (ecology, plant physiology, soil science, etc.) lack access to the technical resources or awareness of the research potential of DGVMs. Here we present the Land Sites Platform (LSP): new software that facilitates single-site simulations with the Functionally Assembled Terrestrial Ecosystem Simulator, an advanced DGVM coupled with the Community Land Model. The LSP includes a Graphical User Interface and an Application Programming Interface, which improve the user experience and lower the technical thresholds for installing these model architectures and setting up model experiments. The software is distributed via version-controlled containers; researchers and students can run simulations directly on their personal computers or servers, with relatively low hardware requirements, and on different operating systems. Version 1.0 of the LSP supports site-level simulations. We provide input data for 20 established geo-ecological observation sites in Norway and workflows to add generic sites from public global datasets. The LSP makes standard model experiments with default data easily achievable (e.g., for educational or introductory purposes) while retaining flexibility for more advanced scientific uses. We further provide tools to visualize the model input and output, including simple examples to relate predictions to local observations. The LSP improves access to land surface and DGVM modelling as a building block of community cyberinfrastructure that may inspire new avenues for mechanistic ecosystem research across disciplines.publishedVersio

Comparing Storm Resolving Models and Climates via Unsupervised Machine Learning

Storm-resolving models (SRMs) have gained widespread interest because of the unprecedented detail with which they resolve the global climate. However, it remains difficult to quantify objective differences in how SRMs resolve complex atmospheric formations. This lack of appropriate tools for comparing model similarities is a problem in many disparate fields that involve simulation tools for complex data. To address this challenge we develop methods to estimate distributional distances based on both nonlinear dimensionality reduction and vector quantization. Our approach automatically learns appropriate notions of similarity from low-dimensional latent data representations that the different models produce. This enables an intercomparison of nine SRMs based on their high-dimensional simulation data and reveals that only six are similar in their representation of atmospheric dynamics. Furthermore, we uncover signatures of the convective response to global warming in a fully unsupervised way. Our study provides a path toward evaluating future high-resolution simulation data more objectively.Comment: 22 pages, 19 figures. Submitted to journal for consideratio

Deciphering ocean carbon in a changing world

Author Posting. © The Author(s), 2016. This is the author's version of the work. It is posted here for personal use, not for redistribution. The definitive version was published in Proceedings of the National Academy of Sciences of the United States of America 113 (2016): 3143-3151, doi:10.1073/pnas.1514645113.Dissolved organic matter (DOM) in the oceans is one of the largest pools of reduced carbon on Earth, comparable in size to the atmospheric CO2 reservoir. A vast number of compounds are present in DOM and they play important roles in all major element cycles, contribute to the storage of atmospheric CO2 in the ocean, support marine ecosystems, and facilitate interactions between organisms. At the heart of the DOM cycle lie molecular-level relationships between the individual compounds in DOM and the members of the ocean microbiome that produce and consume them. In the past, these connections have eluded clear definition because of the sheer numerical complexity of both DOM molecules and microorganisms. Emerging tools in analytical chemistry, microbiology and informatics are breaking down the barriers to a fuller appreciation of these connections. Here we highlight questions being addressed using recent methodological and technological developments in those fields and consider how these advances are transforming our understanding of some of the most important reactions of the marine carbon cycle.Support was provided by National Science Foundation grants OCE1356010, OCE1154320, and OCE1356890, and Gordon and Betty Moore Foundation Grant #3304

INTELLIGENT CYBERINFRASTRUCTURE FOR BIG DATA ENABLED HYDROLOGICAL MODELING, PREDICTION, AND EVALUATION

Most hydrologic data are associated with spatiotemporal information, which is capable of presenting patterns and changes in both spatial and temporal aspects. The demands of retrieving, managing, analyzing, visualizing, and sharing these data have been continuously increasing. However, spatiotemporal hydrologic data are generally complex, which can be difficult to work with knowledge from hydrology alone. With the assistance of geographic information systems (GIS) and web-based technologies, a solution of establishing a cyberinfrastructure as the backbone to support such demands has emerged. This interdisciplinary dissertation described the advancement of traditional approaches for organizing and managing spatiotemporal hydrologic data, integrating and executing hydrologic models, analyzing and evaluating the results, and sharing the entire process. A pilot study was conducted in Chapter 2, in which a globally shared flood cyberinfrastructure was created to collect, organize, and manage flood databases that visually provide useful information to authorities and the public in real-time. The cyberinfrastructure used public cloud services provided by Google Fusion Table and crowdsourcing data collection methods to provide location-based visualization as well as statistical analysis and graphing capabilities. This study intended to engage citizen-scientists and presented an opportunity to modernize the existing paradigm used to collect, manage, analyze, and visualize water-related disasters eventually. An observationally based monthly evapotranspiration (ET) product was produced in Chapter 3, using the simple water balance equation across the conterminous United States (CONUS). The best quality ground- and satellite-based observations of the water budget components, i.e., precipitation, runoff, and water storage change were adopted, while ET is computed as the residual. A land surface model-based downscaling approach to disaggregate the monthly GRACE equivalent water thickness (EWT) data to daily, 0.125º values was developed. The derived ET was evaluated against three sets of existing ET products and showed reliable results. The new ET product and the disaggregated GRACE data could be used as a benchmark dataset for researches in hydrological and climatological changes and terrestrial water and energy cycle dynamics over the CONUS. The study in Chapter 4 developed an automated hydrological modeling framework for any non-hydrologists with internet access, who can organize hydrologic data, execute hydrologic models, and visualize results graphically and statistically for further analysis in real-time. By adopting Hadoop distributed file system (HDFS) and Apache Hive, the efficiency of data processing and query were significantly increased. Two lumped hydrologic models, lumped Coupled Routing and Excess STorage (CREST) model and HyMOD model, were integrated as a proof of concept in this web framework. Evaluation of selected basins over the CONUS were performed as a demonstration. Our vision is to simplify the processes of using hydrologic models for researchers and modelers, as well as to unlock the potential and educate the less experienced public on hydrologic models

Advancing Cyberinfrastructure for Collaborative Data Sharing and Modeling in Hydrology

Hydrologic research is increasingly data and computationally intensive, and often involves hydrologic model simulation and collaboration among researchers. With the development of cyberinfrastructure, researchers are able to improve the efficiency, impact, and effectiveness of their research by utilizing online data sharing and hydrologic modeling functionality. However, further efforts are still in need to improve the capability of cyberinfrastructure to serve the hydrologic science community. This dissertation first presents the evaluation of a physically based snowmelt model as an alternative to a temperature index model to improve operational water supply forecasts in the Colorado River Basin. Then it presents the design of the functionality to share multidimensional space-time data in the HydroShare hydrologic information system. It then describes a web application developed to facilitate input preparation and model execution of a snowmelt model and the storage of these results in HydroShare. The snowmelt model evaluation provided use cases to evaluate the cyberinfrastructure elements developed. This research explored a new approach to advance operational water supply forecasts and provided potential solutions for the challenges associated with the design and implementation of cyberinfrastructure for hydrologic data sharing and modeling