The Climate-G testbed: towards large scale distributed data management for climate change

Climate-G is a large scale distributed testbed devoted to climate change research. It is an unfunded effort started in 2008 and involving a wide community both in Europe and US. The testbed is an interdisciplinary effort involving partners from several institutions and joining expertise in the field of climate change and computational science. Its main goal is to allow scientists carrying out geographical and cross-institutional data discovery, access, analysis, visualization and sharing of climate data. It represents an attempt to address, in a real environment, challenging data and metadata management issues. This paper presents a complete overview about the Climate-G testbed highlighting the most important results that have been achieved since the beginning of this project

A Global Repository for Planet-Sized Experiments and Observations

Working across U.S. federal agencies, international agencies, and multiple worldwide data centers, and spanning seven international network organizations, the Earth System Grid Federation (ESGF) allows users to access, analyze, and visualize data using a globally federated collection of networks, computers, and software. Its architecture employs a system of geographically distributed peer nodes that are independently administered yet united by common federation protocols and application programming interfaces (APIs). The full ESGF infrastructure has now been adopted by multiple Earth science projects and allows access to petabytes of geophysical data, including the Coupled Model Intercomparison Project (CMIP)—output used by the Intergovernmental Panel on Climate Change assessment reports. Data served by ESGF not only include model output (i.e., CMIP simulation runs) but also include observational data from satellites and instruments, reanalyses, and generated images. Metadata summarize basic information about the data for fast and easy data discovery.This work was supported by the U.S. Department of Energy Office of Science/Office of Biological and Environmental Research under Contract DE-AC52-07NA27344 at Lawrence Livermore National Laboratory. VB is supported by the Cooperative Institute for Climate Science, Princeton University, under Award NA08OAR4320752 from the National Oceanic and Atmospheric Administration, U.S. Department of Commerce. Part of this work was undertaken with the assistance of resources from the National Computational Infrastructure (NCI), which is supported by the Australian Government. Part of this activity was performed on behalf of the Jet Propulsion Laboratory, California Institute of Technology, under a contract with NASA. Part of this activity was performed on behalf of the Goddard Space Flight Center, under a contract with NASA. This work was supported by ANR Convergence project (Grant Agreement ANR-13-MONU-0008). This work was supported by FP7 IS-ENES2 project (Grant Agreement 312979)

Assessing the impacts of 1.5 °C global warming – simulation protocol of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2b)

In Paris, France, December 2015, the Conference of the Parties (COP) to the United Nations Framework Convention on Climate Change (UNFCCC) invited the Intergovernmental Panel on Climate Change (IPCC) to provide a "special report in 2018 on the impacts of global warming of 1.5°C above pre-industrial levels and related global greenhouse gas emission pathways". In Nairobi, Kenya, April 2016, the IPCC panel accepted the invitation. Here we describe the response devised within the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) to provide tailored, cross-sectorally consistent impact projections to broaden the scientific basis for the report. The simulation protocol is designed to allow for (1) separation of the impacts of historical warming starting from pre-industrial conditions from impacts of other drivers such as historical land-use changes (based on pre-industrial and historical impact model simulations); (2) quantification of the impacts of additional warming up to 1.5°C, including a potential overshoot and long-term impacts up to 2299, and comparison to higher levels of global mean temperature change (based on the low-emissions Representative Concentration Pathway RCP2.6 and a no-mitigation pathway RCP6.0) with socio-economic conditions fixed at 2005 levels; and (3) assessment of the climate effects based on the same climate scenarios while accounting for simultaneous changes in socio-economic conditions following the middle-of-the-road Shared Socioeconomic Pathway (SSP2, Fricko et al., 2016) and in particular differential bioenergy requirements associated with the transformation of the energy system to comply with RCP2.6 compared to RCP6.0. With the aim of providing the scientific basis for an aggregation of impacts across sectors and analysis of cross-sectoral interactions that may dampen or amplify sectoral impacts, the protocol is designed to facilitate consistent impact projections from a range of impact models across different sectors (global and regional hydrology, lakes, global crops, global vegetation, regional forests, global and regional marine ecosystems and fisheries, global and regional coastal infrastructure, energy supply and demand, temperature-related mortality, and global terrestrial biodiversity)

Assessing the impacts of 1.5°C global warming -- simulation protocol of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP2b)

In Paris, France, December 2015, the Conference of the Parties (COP) to the United Nations Framework Convention on Climate Change (UNFCCC) invited the Intergovernmental Panel on Climate Change (IPCC) to provide a special report in 2018 on the impacts of global warming of 1.5 °C above pre-industrial levels and related global greenhouse gas emission pathways. In Nairobi, Kenya, April 2016, the IPCC panel accepted the invitation. Here we describe the response devised within the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) to provide tailored, cross-sectorally consistent impact projections to broaden the scientific basis for the report. The simulation protocol is designed to allow for (1) separation of the impacts of historical warming starting from pre-industrial conditions from impacts of other drivers such as historical land-use changes (based on pre-industrial and historical impact model simulations); (2) quantification of the impacts of additional warming up to 1.5 °C, including a potential overshoot and long-term impacts up to 2299, and comparison to higher levels of global mean temperature change (based on the low-emissions Representative Concentration Pathway RCP2.6 and a no-mitigation pathway RCP6.0) with socio-economic conditions fixed at 2005 levels; and (3) assessment of the climate effects based on the same climate scenarios while accounting for simultaneous changes in socio-economic conditions following the middle-of-the-road Shared Socioeconomic Pathway (SSP2, Fricko et al., 2016) and in particular differential bioenergy requirements associated with the transformation of the energy system to comply with RCP2.6 compared to RCP6.0. With the aim of providing the scientific basis for an aggregation of impacts across sectors and analysis of cross-sectoral interactions that may dampen or amplify sectoral impacts, the protocol is designed to facilitate consistent impact projections from a range of impact models across different sectors (global and regional hydrology, lakes, global crops, global vegetation, regional forests, global and regional marine ecosystems and fisheries, global and regional coastal infrastructure, energy supply and demand, temperature-related mortality, and global terrestrial biodiversity)

Strategie Roadmap for the Earth System Grid Federation

This article describes the Earth System Grid Federation (ESGF) mission and an international integration strategy for data, database and computational architecture, and stable infrastructure highlighted by the authors (the ESGF Executive Committee). These highlights are key developments needed over the next five to seven years in response to largescale national and international climate community projects that depend on ESGF for success. Quality assurance and baseline performance from laptop to high performance computing characterizes available and potential data streams and strategies. These are required for interactive data collections to remedy gaps in handling enormous international federated climate data archives. Appropriate cyber security ensures protection of data according to projects but still allows access and portability to different ESGF and individual groups and users. A timeline and plan for forecasting interoperable tools takes ESGF from a federated database archive to a robust virtual laboratory and concludes the article

Coordinating an operational data distribution network for CMIP6 data

The distribution of data contributed to the Coupled Model Intercomparison Project Phase 6 (CMIP6) is via the Earth System Grid Federation (ESGF). The ESGF is a network of internationally distributed sites that together work as a federated data archive. Data records from climate modelling institutes are published to the ESGF and then shared around the world. It is anticipated that CMIP6 will produce approximately 20 PB of data to be published and distributed via the ESGF. In addition to this large volume of data a number of value-added CMIP6 services are required to interact with the ESGF; for example the citation and errata services both interact with the ESGF but are not a core part of its infrastructure. With a number of interacting services and a large volume of data anticipated for CMIP6, the CMIP Data Node Operations Team (CDNOT) was formed. The CDNOT coordinated and implemented a series of CMIP6 preparation data challenges to test all the interacting components in the ESGF CMIP6 software ecosystem. This ensured that when CMIP6 data were released they could be reliably distributed. No. DE-ACO2-05CH11231 and authors at Lawrence Livermore National Laboratory (LLNL) under contract DE-AC52-07NA27344 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).Funding Agencies|US Department of EnergyUnited States Department of Energy (DOE) [DE-AC02-05CH11231, DE-AC52-07NA27344]; European UnionEuropean Commission [824084]; French National Research Agency project CONVERGENCEFrench National Research Agency (ANR) [ANR-13-MONU-0008-02]; National Collaborative Research Infrastructure Strategy (NCRIS)-funded National Computational Infrastructure (NCI) Australia; Australian Research Data Commons (ARDC)</p

Supporting the climate community by providing common metadata for climate modelling digital repositories: The metafor project

A poster to highlight common metadata for climate modelling repositories to support the climate community. There is more interest than ever in the results of climate models; users are no longer limited to the scientific and academic communities, and can now be found in as diverse areas as local government, policy and the general public. Climate modeling is a complex process, which requires accurate and complete metadata (data describing data) in order to identify, assess and use the climate data stored in digital repositories and made available to these users