Balancing EC-Earth3 improving the performance of EC-Earth CMIP6 configurations by minimizing the coupling cost

Earth System Models (ESMs) are complex systems used in weather and climate studies generally built from different independent components responsible for simulating a specific realm (ocean, atmosphere, biosphere, etc.). To replicate the interactions between these processes, ESMs typically use coupling libraries that manage the synchronization and field exchanges between the individual components, which run in parallel as a Multi-Program, Multiple-Data application. As ESMs get more complex (increase in resolution, number of components, configurations, etc.), achieving the best performance when running in High-performance Computing platforms has become increasingly challenging and of major concern. One of the critical bottlenecks is the load-imbalance, where the fastest components will have to wait for the slower ones. Finding the optimal number of processing elements to assign to each of the multiple independent constituents to minimize the performance loss due to synchronizations and maximize the overall parallel efficiency is impossible without the right performance metrics, methodology, and tools. This paper presents the results of balancing multiple Coupled Model Intercomparison Project phase 6 configurations for the EC-Earth3 ESM. We will show that intuitive approaches can lead to suboptimal resource allocations and propose new setups up to 25% fasters while reducing the computational cost by 72%. We prove that new methods are needed to deal with the load-balance of ESMs and hope that our study will serve as a guide to optimize any other coupled system.The research leading to these results has received co-funding from the National Research Agency through OEMES (PID2020-116324RA-I00).Peer ReviewedPostprint (published version

CAMP first GPU solver: a solution to accelerate chemistry in atmospheric models

Atmospheric models are a representation of dynamical, physical, chemical, dynamical, and radiative processes in the atmosphere [1]. The load of these models is often spread across multiple processes in HPC environments. Most of this load comes from the resolution of chemical processes, which can take up to 90% of the total time execution [2]. Recent studies reported a relevant speedup by translating a chemical module to GPUs [3] [4]. This study is based in some previous works of the authors. These works are tested in the Chemistry Accross Multiple Phases (CAMP) module [5] simulating the conditions of an atmospheric model experiment. In our first approach we present an strategy to efficiently integrate GPU routines without needing to translate the entire chemical module to GPU code [6]. In our second and last work, we integrated a GPU version of the linear solver used in CAMP and evaluated multiple kernel configurations, achieving up to 34x speedup from the base CPU linear solver in a singlethread execution, in addition to a 2.7x for an equivalent MPI implementation with the maximum number of physical cores available on a node (40) [7]. The main objective of this work is to develop a GPU version of the entire CAMP solving algorithm. Our second objective is to evaluate the performance of our work, comparing the results with other state of the art GPU chemical modules

Evaluation and optimisation of the I/O scalability for the next generation of Earth system models: IFS CY43R3 and XIOS 2.0 integration as a case study

Earth system models have considerably increased their spatial resolution to solve more complex problems and achieve more realistic solutions. However, this generates an enormous amount of model data which requires proper management. Some Earth system models use inefficient sequential input/output (I/O) schemes that do not scale well when many parallel resources are used. In order to address this issue, the most commonly adopted approach is to use scalable parallel I/O solutions that offer both computational performance and efficiency. In this paper we analyse the I/O process of the European Centre for Medium-Range Weather Forecasts (ECMWF) operational Integrated Forecasting System (IFS) CY43R3. IFS can use two different output schemes: a parallel I/O server developed by Météo-France used operationally and an obsolete sequential I/O scheme. The latter is the only scheme that is being exposed by the OpenIFS variant of IFS. “Downstream” Earth system models that have adopted older versions of an IFS derivative as a component – such as the EC-Earth 3 climate model – also face a bottleneck due to the limited I/O capabilities and performance of the sequential output scheme. Moreover, it is often desirable to produce grid-point-space Network Common Data Format (NetCDF) files instead of the IFS native spectral and grid-point output fields in General Regularly-distributed Information in Binary form (GRIB), which requires the development of model-specific post-processing tools. We present the integration of the XML Input/Output Server (XIOS) 2.0 into IFS CY43R3. XIOS is an asynchronous Message Passing Interface (MPI) I/O server that offers features especially targeted at climate models: NetCDF output files, inline diagnostics, regridding, and, when properly configured, the capability to produce CMOR-compliant data. We therefore expect our work to reduce the computational cost of data-intensive (high-resolution) climate runs, thereby shortening the critical path of EC-Earth 4 experiments. The performance evaluation suggests that the use of XIOS 2.0 in IFS CY43R3 to output data achieves an adequate performance as well, outperforming the sequential I/O scheme. Furthermore, when we also take into account the post-processing task, which is needed to convert GRIB files to NetCDF files and also transform IFS spectral output fields to grid-point space, our integration not only surpasses the sequential output scheme but also the operational IFS I/O server.This research has been supported by Horizon 2020 (ESiWACE2 (grant no. 823988) and PRIMAVERA (grant no. 641727)).Peer ReviewedPostprint (published version

Chemistry Across Multiple Phases (CAMP) version 1.0: an integrated multiphase chemistry model

A flexible treatment for gas- and aerosol-phase chemical processes has been developed for models of diverse scale, from box models up to global models. At the core of this novel framework is an “abstracted aerosol representation” that allows a given chemical mechanism to be solved in atmospheric models with different aerosol representations (e.g., sectional, modal, or particle-resolved). This is accomplished by treating aerosols as a collection of condensed phases that are implemented according to the aerosol representation of the host model. The framework also allows multiple chemical processes (e.g., gas- and aerosol-phase chemical reactions, emissions, deposition, photolysis, and mass transfer) to be solved simultaneously as a single system. The flexibility of the model is achieved by (1) using an object-oriented design that facilitates extensibility to new types of chemical processes and to new ways of representing aerosol systems, (2) runtime model configuration using JSON input files that permits making changes to any part of the chemical mechanism without recompiling the model (this widely used, human-readable format allows entire gas- and aerosol-phase chemical mechanisms to be described with as much complexity as necessary), and (3) automated comprehensive testing that ensures stability of the code as new functionality is introduced. Together, these design choices enable users to build a customized multiphase mechanism without having to handle preprocessors, solvers, or compilers. Removing these hurdles makes this type of modeling accessible to a much wider community, including modelers, experimentalists, and educators. This new treatment compiles as a stand-alone library and has been deployed in the particle-resolved PartMC model and in the Multiscale Online AtmospheRe CHemistry (MONARCH) chemical weather prediction system for use at regional and global scales. Results from the initial deployment to box models of different complexity and MONARCH will be discussed, along with future extension to more complex gas–aerosol systems and the integration of GPU-based solvers.Matthew L. Dawson has received funding from the European Union's Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement no. 747048. Matthew L. Dawson, Oriol Jorba, and Christian Guzman have been supported by the Ministerio de Ciencia, Innovación y Universidades (grant no. RTI2018-099894-BI00). Christian Guzman acknowledges funding from the AXA Research Fund. Nicole Riemer, Matthew West, and Jeffrey H. Curtis acknowledge funding from the National Science Foundation (grant no. AGS 19-41110). This material is based upon work supported by the National Center for Atmospheric Research, which is a major facility sponsored by the National Science Foundation under cooperative agreement no. 1852977.Peer ReviewedPostprint (published version

Chemistry Across Multiple Phases (CAMP) version 1.0: an integrated multiphase chemistry model

A flexible treatment for gas- and aerosol-phase chemical processes has been developed for models of diverse scale, from box models up to global models. At the core of this novel framework is an “abstracted aerosol representation” that allows a given chemical mechanism to be solved in atmospheric models with different aerosol representations (e.g., sectional, modal, or particle-resolved). This is accomplished by treating aerosols as a collection of condensed phases that are implemented according to the aerosol representation of the host model. The framework also allows multiple chemical processes (e.g., gas- and aerosol-phase chemical reactions, emissions, deposition, photolysis, and mass transfer) to be solved simultaneously as a single system. The flexibility of the model is achieved by (1) using an object-oriented design that facilitates extensibility to new types of chemical processes and to new ways of representing aerosol systems, (2) runtime model configuration using JSON input files that permits making changes to any part of the chemical mechanism without recompiling the model (this widely used, human-readable format allows entire gas- and aerosol-phase chemical mechanisms to be described with as much complexity as necessary), and (3) automated comprehensive testing that ensures stability of the code as new functionality is introduced. Together, these design choices enable users to build a customized multiphase mechanism without having to handle preprocessors, solvers, or compilers. Removing these hurdles makes this type of modeling accessible to a much wider community, including modelers, experimentalists, and educators. This new treatment compiles as a stand-alone library and has been deployed in the particle-resolved PartMC model and in the Multiscale Online AtmospheRe CHemistry (MONARCH) chemical weather prediction system for use at regional and global scales. Results from the initial deployment to box models of different complexity and MONARCH will be discussed, along with future extension to more complex gas–aerosol systems and the integration of GPU-based solvers.Matthew L. Dawson has received funding from the European Union's Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement no. 747048. Matthew L. Dawson, Oriol Jorba, and Christian Guzman have been supported by the Ministerio de Ciencia, Innovación y Universidades (grant no. RTI2018-099894-BI00). Christian Guzman acknowledges funding from the AXA Research Fund. Nicole Riemer, Matthew West, and Jeffrey H. Curtis acknowledge funding from the National Science Foundation (grant no. AGS 19-41110). This material is based upon work supported by the National Center for Atmospheric Research, which is a major facility sponsored by the National Science Foundation under cooperative agreement no. 1852977.Peer ReviewedPostprint (published version

Using EC-Earth for climate prediction research

Climate prediction at the subseasonal to interannual time range is now performed routinely and operationally by an increasing number of institutions. The feasibility of climate prediction largely depends on the existence of slow and predictable variations in the ocean surface temperature, sea ice, soil moisture and snow cover, and on our ability to model the atmosphere’s interactions with those variables. Climate prediction is typically performed with statistical-empirical or process-based models. The two methods are complementary. Although forecasting systems using global climate models (GCMs) have made substantial progress in the last few decades (Doblas-Reyes et al., 2013), systematic errors and misrepresentations of key processes still limit the value of dynamical prediction in certain areas of the globe. At the same time, model initialisation, ensemble generation, understanding the processes at the origin of predictability, forecasting extremes, bias adjustment and model evaluation are all challenging aspects of the climate prediction problem. Addressing them requires both a large base of researchers with expertise in physics, mathematics, statistics, high-performance computing and data analysis interested in climate prediction issues and a tool for them to work with. This article illustrates how one of these tools, the EC-Earth climate model (Box A), has been used to train scientists in climate prediction and to address scientific challenges in this field. The use of model components from ECMWF’s Integrated Forecasting System (IFS) in EC-Earth means that some of the results obtained with EC-Earth can feed back into ECMWF’s activities. EC-Earth has been run extensively on ECMWF’s high-performance computing facility (HPCF), among a range of HPCFs across Europe and North America. The availability of ECMWF’s HPCF to EC-Earth partners, including the use of the successful ECMWF Special Project programme, means that a substantial amount of EC-Earth’s collaborative work, both within the consortium and with ECMWF, takes place on this platform.Postprint (published version

The EC-Earth3 Earth system model for the Coupled Model Intercomparison Project 6

The Earth system model EC-Earth3 for contributions to CMIP6 is documented here, with its flexible coupling framework, major model configurations, a methodology for ensuring the simulations are comparable across different high-performance computing (HPC) systems, and with the physical performance of base configurations over the historical period. The variety of possible configurations and sub-models reflects the broad interests in the EC-Earth community. EC-Earth3 key performance metrics demonstrate physical behavior and biases well within the frame known from recent CMIP models. With improved physical and dynamic features, new Earth system model (ESM) components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond.The development of EC-Earth3 was supported by the European Union's Horizon 2020 research and innovation program under project IS-ENES3, the third phase of the distributed e-infrastructure of the European Network for Earth System Modelling (ENES) (grant agreement no. 824084, PRIMAVERA grant no. 641727, and CRESCENDO grant no. 641816). Etienne Tourigny and Raffaele Bernardello have received funding from the European Union’s Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement nos. 748750 (SPFireSD project) and 708063 (NeTNPPAO project). Ivana Cvijanovic was supported by Generalitat de Catalunya (Secretaria d'Universitats i Recerca del Departament d’Empresa i Coneixement) through the Beatriu de Pinós program. Yohan Ruprich-Robert was funded by the European Union's Horizon 2020 research and innovation program in the framework of Marie Skłodowska-Curie grant INADEC (grant agreement 800154). Paul A. Miller, Lars Nieradzik, David Wårlind, Roland Schrödner, and Benjamin Smith acknowledge financial support from the strategic research area “Modeling the Regional and Global Earth System” (MERGE) and the Lund University Centre for Studies of Carbon Cycle and Climate Interactions (LUCCI). Paul A. Miller, David Wårlind, and Benjamin Smith acknowledge financial support from the Swedish national strategic e-science research program eSSENCE. Paul A. Miller further acknowledges financial support from the Swedish Research Council (Vetenskapsrådet) under project no. 621-2013-5487. Shuting Yang acknowledges financial support from a Synergy Grant from the European Research Council under the European Community's Seventh Framework Programme (FP7/2007-2013)/ERC (grant agreement 610055) as part of the ice2ice project and the NordForsk-funded Nordic Centre of Excellence project (award 76654) ARCPATH. Marianne Sloth Madsen acknowledges financial support from the Danish National Center for Climate Research (NCKF). Andrea Alessandri and Peter Anthoni acknowledge funding from the Helmholtz Association in its ATMO program. Thomas Arsouze, Arthur Ramos, and Valentina Sicardi received funding from the Ministerio de Ciencia, Innovación y Universidades as part of the DeCUSO project (CGL2017-84493-R).​​​​​​​Peer Reviewed"Article signat per 61 autors/es: Ralf Döscher, Mario Acosta, Andrea Alessandri, Peter Anthoni, Thomas Arsouze, Tommi Bergman, Raffaele Bernardello, Souhail Boussetta, Louis-Philippe Caron, Glenn Carver, Miguel Castrillo, Franco Catalano, Ivana Cvijanovic, Paolo Davini, Evelien Dekker, Francisco J. Doblas-Reyes, David Docquier, Pablo Echevarria, Uwe Fladrich, Ramon Fuentes-Franco, Matthias Gröger, Jost v. Hardenberg, Jenny Hieronymus, M. Pasha Karami, Jukka-Pekka Keskinen, Torben Koenigk, Risto Makkonen, François Massonnet, Martin Ménégoz, Paul A. Miller, Eduardo Moreno-Chamarro, Lars Nieradzik, Twan van Noije, Paul Nolan, Declan O'Donnell, Pirkka Ollinaho11, Gijs van den Oord, Pablo Ortega, Oriol Tintó Prims, Arthur Ramos, Thomas Reerink, Clement Rousset, Yohan Ruprich-Robert, Philippe Le Sager, Torben Schmith, Roland Schrödner, Federico Serva, Valentina Sicardi, Marianne Sloth Madsen, Benjamin Smith, Tian Tian, Etienne Tourigny, Petteri Uotila, Martin Vancoppenolle, Shiyu Wang, David Wårlind, Ulrika Willén, Klaus Wyser, Shuting Yang, Xavier Yepes-Arbós, and Qiong Zhang"Postprint (author's final draft

Optimización de Modelos Hidrodinámicos 3D del Transporte y Mezcla Aplicados al Conocimiento y Predicción de Masas de Agua Continental

El objetivo general de esta tesis doctoral será mejorar la eficiencia computacional de los modelos hidrodinámicos existentes con el fin de abordar el estudio riguroso y detallado de los procesos de transporte y mezcla en sistemas de agua continental. Se mostrarán soluciones para reducir el coste computacional de estas simulaciones, de forma que se puedan obtener resultados útiles en un tiempo aceptable con grids de alta resolución, empleando tanto recursos fácilmente accesibles por cualquier científico como Arquitecturas de Altas Prestaciones (High Performance Computing, HPC). Las estrategias de optimización utilizadas han permitido mejorar la eficiencia de estos modelos, entendida esta como la relación entre prestaciones y coste y como la relación entre prestaciones y calidad de los resultados. Estas estrategias de optimización deberán facilitar y extender la utilización de los modelos 3D de transporte y mezcla como herramientas de trabajo en investigación aplicada al conocimiento de lagos, embalses y ríos, o como herramientas operacionales de predicción. Para ello se pretende realizar el procesamiento requerido en un tiempo aceptable en computadores personales, clúster de computadores personales y servidores de gama baja, evitando usar costosos servidores de gama alta. Para los casos más costosos, se presenta también una alternativa haciendo uso de HPC, implementada de forma eficiente. La segunda propuesta para reducir los costes de computación hace uso de un procedimiento conocido como anidamiento, ya usado en la literatura para reducir los costes de computación. En esta tesis se propone una implementación eficiente del procedimiento de anidamiento, combinándolo con clusters de computadores de gama media/baja, reduciendo aún más los costes computacionales y permitiendo resolver de forma eficiente toda la zona litoral (imposible en un tiempo aceptable mediante un procedimiento de anidamiento normal) de grandes lagos en alta resolución , del cual a pesar de su enorme diversidad e importancia como se ha explicado, se tiene un conocimiento bastante pobre (Kalff 2001). En línea a esto se muestran resultados útiles y de interés a la comunicad científica que indiquen el camino a seguir en el estudio de la zona litoral de grandes lagos y que demuestren la necesidad del uso de grids de alta resolución para poder realizar estos estudios con éxito.Tesis Univ. Granada. Programa Oficial de Doctorado en: Tecnologías de la Información y la Comunicació

ESTRUCTURA DE COMPUTADORS I (Examen 2n quadrimestre)

Resolve

ESTRUCTURA DE COMPUTADORS I (Examen 2n quadrimestre). Examen final (Curs 2019-2020)

Resolve