Integrating e-infrastructures for remote climate data processing

Presented at EGU 2020 - Accessing and processing large climate data has nowadays become a particularly challenging task for end users, due to the rapidly increasing volumes being produced and made available. Access to climate data is crucial for sustaining research and performing climate change impact assessments. These activities have strong societal impact as climate change affects and requires that almost all economic and social sectors need adapting. The whole climate data archive is expected to reach a volume of 30 PB in 2020 and up to 2000 PB in 2024 (estimated), evolving from 0.03 PB (30 TB) in 2007 and 2 PB in 2014. Data processing and analysis must now take place remotely for the users: users typically have to rely on heterogeneous infrastructures and services between the data and their physical location. Developers of Research Infrastructures have to provide services to those users, hence having to define standards and generic services to fulfil those requirements. It will be shown how the DARE eScience Platform (http://project-dare.eu) will help developers to develop needed services more quickly and transparently for a large range of scientific researchers. The platform is designed for efficient and traceable development of complex experiments and domain-specific services. Most importantly, the DARE Platform integrates the following e-infrastructure services: the climate IS-ENES (https://is.enes.org) Research Infrastructure front-end climate4impact (C4I: https://climate4impact.eu), the EUDAT CDI (https://www.eudat.eu/eudat-collaborative-data-infrastructure-cdi) B2DROP Service, as well as the ESGF (https://esgf.llnl.gov). The DARE Platform itself can be deployed by research communities on local, public or commercial clouds, thanks to its containerized architecture. More specifically, two distinct Use Cases for the climate science domain will be presented. The first will show how an open source software to compute climate indices and indicators (icclim: https://github.com/cerfacs-globc/icclim) is leveraged using the DARE Platform to enable users to build their own workflows. The second Use Case will demonstrate how more complex tools, such as an extra-tropical and tropical cyclone tracking software (https://github.com/cerfacs-globc/cyclone_tracking), can be easily made available to end users by infrastructure and front-end software developers

Easy Access to Complex Analysis Tools for Climate Researchers and Climate Data End Users

Researchers and end users using climate data face a challenge when they analyze the data they need. Data volumes are increasing very rapidly, and the ability to download all needed data is often no longer possible. Also, it can be complex to install, configure and use some advanced analysis tools on such large datasets. This is especially true when they are stored a federated architecture like the ESGF. An example of a complex analysis tool used in climate research and adaptation studies is a tool to follow storm tracks. In the context of climate change, it is important to know if storm tracks will change in the future, in both their frequency and intensity. Storms can cause significant societal impacts, hence it is important to assess future patterns. Having access to this type of complex analysis tool is very useful, and integrating them with front-ends like the IS-ENES climate4impact (C4I) would enable the use of those tools by a larger number of researchers and end users. Integrating this type of complex tool is not an easy task. It requires significant development e ort, especially if one of the objectives is also to adhere to FAIR principles. The DARE Platform enables research developers to faster develop the implementations of scientific workflows more rapidly. This work presents how such a complex analysis tool has been implemented to be easily integrated with the C4I platform. The DARE Platform also provides easy access to e-infrastructure services like EUDAT B2DROP, to store intermediate original results and powerful provenance-powered tools to help researchers manage their work and data

Towards a dedicated impact portal to bridge the gap between the impact and climate communities : Lessons from use cases

International audienceFuture climate evolution is of primary importance for the societal, economical, political orientations and decision-making. It explains the increasing use of climate projections as input for quantitative impact studies, assessing vulnerability and defining adaptation strategies in different sectors. Here we analyse 17 national and representative use cases so as to identify the diversity of the demand for climate information depending on user profiles as well as the best practices, methods and tools that are needed to answer the different requests. A particular emphasis is put on the workflow that allows to translate climate data into suitable impact data, the way to deal with the different sources of uncertainty and to provide a suited product to users. We identified three complementary tools to close the gap between climate scientists and user needs: an efficient interface between users and providers; an optimized methodology to handle user requests and a portal to facilitate access to data and elaborated products. We detail in the paper how these three tools can limit the intervention of experts, educate users, and lead to the production of useful information. This work provides the basis on which the ENES (European Network for Earth System Modelling) Portal Interface for the Climate Impact Communities is built

Atmospheric data access for the geospatial user community (ADAGUC)

Historically the atmospheric and meteorological communities are separate worlds with their own data formats and tools for data handling making sharing of data difficult and cumbersome. On the other hand, these information sources are becoming increasingly of interest outside these communities because of the continuously improving spatial and temporal resolution of e.g. model and satellite data and the interest in historical datasets. New user communities that use geographically based datasets in a cross-domain manner are emerging. This development is supported by the progress made in Geographical Information System (GIS) software. The current GIS software is not yet ready for the wealth of atmospheric data, although the faint outlines of new generation software are already visible: support of HDF, NetCDF and an increasing understanding of temporal issues are only a few of the hints

Atmospheric data access to the geospatial user community

Historically the atmospheric and meteorological communities are separate worlds with their own data formats and tools for data handling making sharing of data difficult and cumbersome. On the other hand, these information sources are becoming increasingly of interest outside these communities because of the continuously improving spatial and temporal resolution of e.g. model and satellite data and the interest in historical datasets. New user communities that use geographically based datasets in a cross-domain manner are emerging. This development is supported by the progress made in Geographical Information System (GIS) software. The current GIS software is not yet ready for the wealth of atmospheric data, although the faint outlines of new generation software are already visible: support of HDF, NetCDF and an increasing understanding of temporal issues are only a few of the hints