Optimizing an MPI weather forecasting model via processor virtualization

Abstract—Weather forecasting models are computationally intensive applications. These models are typically executed in parallel machines and a major obstacle for their scalability is load imbalance. The causes of such imbalance are either static (e.g. topography) or dynamic (e.g. shortwave radiation, moving thunderstorms). Various techniques, often embedded in the application’s source code, have been used to address both sources. However, these techniques are inflexible and hard to use in legacy codes. In this paper, we demonstrate the effectiveness of processor virtualization for dynamically balancing the load in BRAMS, a mesoscale weather forecasting model based on MPI paral-lelization. We use the Charm++ infrastructure, with its over-decomposition and object-migration capabilities, to move sub-domains across processors during execution of the model. Pro-cessor virtualization enables better overlap between computation and communication and improved cache efficiency. Furthermore, by employing an appropriate load balancer, we achieve better processor utilization while requiring minimal changes to the model’s code. I

A Comparative Analysis of Load Balancing Algorithms Applied to a Weather Forecast Model

Among the many reasons for load imbalance in weather forecasting models, the dynamic imbalance caused by lo-calized variations on the state of the atmosphere is the hard-est one to handle. As an example, active thunderstorms may substantially increase load at a certain timestep with re-spect to previous timesteps in an unpredictable manner – after all, tracking storms is one of the reasons for running a weather forecasting model. In this paper, we present a com-parative analysis of different load balancing algorithms to deal with this kind of load imbalance. We analyze the im-pact of these strategies on computation and communication and the effects caused by the frequency at which the load balancer is invoked on execution time. This is done with-out any code modification, employing the concept of proces-sor virtualization, which basically means that the domain is over-decomposed and the unit of rebalance is a sub-domain. With this approach, we were able to reduce the execution time of a full, real-world weather model. 1

On the scalability of CFD tool for supersonic jet flow configurations

New regulations are imposing noise emissions limitations for the aviation industry which are pushing researchers and engineers to invest efforts in studying the aeroacoustics phenomena. Following this trend, an in-house computational fluid dynamics tool is build to reproduce high fidelity results of supersonic jet flows for aeroacoustic analogy applications. The solver is written using the large eddy simulation formulation that is discretized using a finite difference approach and an explicit time integration. Numerical simulations of supersonic jet flows are very expensive and demand efficient high-performance computing. Therefore, non-blocking message passage interface protocols and parallel Input/Output features are implemented into the code in order to perform simulations which demand up to one billion grid points. The present work addresses the evaluation of code improvements along with the computational performance of the solver running on a computer with maximum theoretical peak of 2.727 PFlops. Different mesh configurations, whose size varies from a few hundred thousand to approximately one billion grid points, are evaluated in the present paper. Calculations are performed using different workloads in order to assess the strong and weak scalability of the parallel computational tool. Moreover, validation results of a realistic flow condition are also presented in the current work

The Brazilian Developments on the Regional Atmospheric Modeling System (BRAMS 5.2): An Integrated Environmental Model Tuned for Tropical Areas

We present a new version of the Brazilian developments on the Regional Atmospheric Modeling System where different previous versions for weather, chemistry and carbon cycle were unified in a single integrated software system. The new version also has a new set of state-of-the-art physical parameterizations and greater computational parallel and memory usage efficiency. Together with the description of the main features are examples of the quality of the transport scheme for scalars, radiative fluxes on surface and model simulation of rainfall systems over South America in different spatial resolutions using a scale-aware convective parameterization. Besides, the simulation of the diurnal cycle of the convection and carbon dioxide concentration over the Amazon Basin, as well as carbon dioxide fluxes from biogenic processes over a large portion of South America are shown. Atmospheric chemistry examples present model performance in simulating near-surface carbon monoxide and ozone in Amazon Basin and Rio de Janeiro megacity. For tracer transport and dispersion, it is demonstrated the model capabilities to simulate the volcanic ash 3-d redistribution associated with the eruption of a Chilean volcano. Then, the gain of computational efficiency is described with some details. BRAMS has been applied for research and operational forecasting mainly in South America. Model results from the operational weather forecast of BRAMS on 5 km grid spacing in the Center for Weather Forecasting and Climate Studies, INPE/Brazil, since 2013 are used to quantify the model skill of near surface variables and rainfall. The scores show the reliability of BRAMS for the tropical and subtropical areas of South America. Requirements for keeping this modeling system competitive regarding on its functionalities and skills are discussed. At last, we highlight the relevant contribution of this work on the building up of a South American community of model developers

The Brazilian developments on the Regional Atmospheric Modeling System (BRAMS 5.2): an integrated environmental model tuned for tropical areas

We present a new version of the Brazilian developments on the Regional Atmospheric Modeling System (BRAMS), in which different previous versions for weather, chemistry, and carbon cycle were unified in a single integrated modeling system software. This new version also has a new set of state-of-the-art physical parameterizations and greater computational parallel and memory usage efficiency. The description of the main model features includes several examples illustrating the quality of the transport scheme for scalars, radiative fluxes on surface, and model simulation of rainfall systems over South America at different spatial resolutions using a scale aware convective parameterization. Additionally, the simulation of the diurnal cycle of the convection and carbon dioxide concentration over the Amazon Basin, as well as carbon dioxide fluxes from biogenic processes over a large portion of South America, are shown. Atmospheric chemistry examples show the model performance in simulating near-surface carbon monoxide and ozone in the Amazon Basin and the megacity of Rio de Janeiro. For tracer transport and dispersion, the model capabilities to simulate the volcanic ash 3-D redistribution associated with the eruption of a Chilean volcano are demonstrated. The gain of computational efficiency is described in some detail. BRAMS has been applied for research and operational forecasting mainly in South America. Model results from the operational weather forecast of BRAMS on 5 km grid spacing in the Center for Weather Forecasting and Climate Studies, INPE/Brazil, since 2013 are used to quantify the model skill of near-surface variables and rainfall. The scores show the reliability of BRAMS for the tropical and subtropical areas of South America. Requirements for keeping this modeling system competitive regarding both its functionalities and skills are discussed. Finally, we highlight the relevant contribution of this work to building a South American community of model developers.CNPqFAPESPEarth System Research Laboratory at the National Oceanic and Atmospheric Administration (ESRL/NOAA), Boulder, USAInst Nacl Pesquisas Espaciais, Ctr Previsao Tempo & Estudos Climat, Cachoeira Paulista, SP, BrazilDiv Ciência da Computação, Instituto Tecnológico de Aeronáutica, São José dos Campos, SP, BrazilUniv Estadual Paulista Unesp, Fac Ciencias, Bauru, SP, BrazilCtr Meteorol Bauru IPMet, Bauru, SP, BrazilUniv Fed Sao Paulo, Dept Ciencias Ambientais, Diadema, SP, BrazilUniv Sao Paulo, Inst Astron Geofis & Ciencias Atmosfer, Sao Paulo, SP, BrazilUniv Fed Campina Grande, Dept Ciencias Atmosfer, Campina Grande, PB, BrazilEmbrapa Informat Agr, Campinas, SP, BrazilUniv Fed Sao Paulo, Inst Ciencia & Tecnol, Sao Jose Dos Campos, SP, BrazilUniv Fed Rio Grande do Norte, Dept Ciencias Atmosfer & Climat, Programa Pos Grad Ciencias Climat, Natal, RN, BrazilInst Nacl Pesquisas Espaciais, Ctr Ciencias Sistema, Sao Jose Dos Campos, SP, BrazilUniv Fed Sao Joao Del Rei, Dept Geociencias, Sao Joao Del Rei, MG, BrazilInst Nacl Pesquisas Espaciais, Lab Associado Computacao & Matemat Aplica, Sao Jose Dos Campos, BrazilUniv Evora, Inst Ciencias Agr & Ambientais Mediterr, Evora, PortugalUniv Lusofona Humanidades & Tecnol, Ctr Interdisciplinar Desenvolvimento Ambient Gest, Lisbon, PortugalUniv Fed Pelotas, Fac Meteorol, Pelotas, RS, BrazilUnive Tecnol Fed Parana, Londrina, PR, BrazilNASA, Goddard Space Flight Ctr, Univ Space Res Assoc, Goddard Earth Sci Technol & Res Global Modeling &, Greenbelt, MD USAUniv Fed Sao Paulo, Inst Ciencia & Tecnol, Sao Jose Dos Campos, SP, BrazilUniv Fed Sao Paulo, Inst Ciencia & Tecnol, Sao Jose Dos Campos, SP, BrazilCNPq: 306340/2011-9FAPESP: 2014/01563-1FAPESP: 2015/10206-0FAPESP: 2014/01564-8Web of Scienc

PRIMITIVE PARALLEL OPERATIONS FOR COMPUTATIONAL LINEAR ALGEBRA (PROCESSING, MATHEMATICAL SOFTWARE)

This work is a small step on the direction of code portability over parallel and vector machines. The proposal consists of a style of programming and a set of parallel operators built over abstract data types. Objects and operators are directed to the Computational Linear Algebra area, although the principles of the proposal can be applied to any other area. A subset of the operators was implemented on a 64 processor, distributed memory MIMD machine, and the results are that computationally intensive operators achieve asymptotically optimal speed-ups, but data movement operators are inefficient, some even intrinsically sequential

Parallelizing CPTEC's general circulation model

We describe the first parallel version of CPTECs General Circulation Model, targeting a 4 processor, shared memory NEC SX4. This paper emphasizes techniques to parallelize vintage production code, keeping results reproducible. Measured speed-ups compare favorably with Amdahls Law predicted values

Aplicação de grid computing em meteorologia

This paper shows case studies of Grid Computing applied to meteorology, developed at CPTEC/INPE. Initially, its presented computing areas that can benefit from this technology (on-demand computing, collaborative computing, high-throughput computing, data-intensive computing and distributed supercomputing) and possible uses in meteorology. CPTEC/INPE is investigating the use of this technology in data-intensive computing, using meteorological applications; collaborative computing, using "AccessGrid" for collaborative work and videoconference; and distributed supercomputing, in a project coordinated by SINAPAD (Sistema Nacional de Processamento de Alto Desempenho). Preliminary results show a promising future for the application of this new technology in meteorology. RESUMO: Este trabalho mostra casos de uso de "Grid Computing" na área de meteorologia desenvolvidos no CPTEC/INPE. Inicialmente apresenta-se as áreas da computação onde esta tecnologia pode ser utilizada (computação sob demanda, colaborativa, alta-vazão, intensiva de dados e supercomputação distribuída) e possíveis usos em meteorologia. O CPTEC/INPE vem investigando o uso desta tecnologia nas áreas de computação intensiva de dados, utilizando aplicações meteorológicas; colaborativa, através do uso do software "AccessGrid" para trabalho colaborativo e videoconferências; e supercomputação distribuída, em um projeto para construção de um "Grid" Computacional de Alto Desempenho coordenado pelo SINAPAD (Sistema Nacional de Processamento de Alto Desempenho). Resultados preliminares mostram um futuro promissor para utilização desta nova tecnologia em meteorologia