accelerating wrf i/o performance with adios2 and network-based streaming

With the approach of Exascale computing power for large-scale High Performance Computing (HPC) clusters, the gap between compute capabilities and storage systems is growing larger. This is particularly problematic for the Weather Research and Forecasting Model (WRF), a widely-used HPC application for high-resolution forecasting and research that produces sizable datasets, especially when analyzing transient weather phenomena. Despite this issue, the I/O modules within WRF have not been updated in the past ten years, resulting in subpar parallel I/O performance. This research paper demonstrates the positive impact of integrating ADIOS2, a next-generation parallel I/O framework, as a new I/O backend option in WRF. It goes into detail about the challenges encountered during the integration process and how they were addressed. The resulting I/O times show an over tenfold improvement when using ADIOS2 compared to traditional MPI-I/O based solutions. Furthermore, the study highlights the new features available to WRF users worldwide, such as the Sustainable Staging Transport (SST) enabling Unified Communication X (UCX) DataTransport, the node-local burst buffer write capabilities and in-line lossless compression capabilities of ADIOS2. Additionally, the research shows how ADIOS2's in-situ analysis capabilities can be smoothly integrated with a simple WRF forecasting pipeline, resulting in a significant improvement in overall time to solution. This study serves as a reminder to legacy HPC applications that incorporating modern libraries and tools can lead to considerable performance enhancements with minimal changes to the core application.Comment: arXiv admin note: text overlap with arXiv:2201.0822

Combining Litter Observations with a Regional Ocean Model to Identify Sources and Sinks of Floating Debris in a Semi-enclosed Basin: The Adriatic Sea

Visual ship transect surveys provide crucial information about the density, and spatial distribution of floating anthropogenic litter in a basin. However, such observations provide a ‘snapshot’ of local conditions at a given time and cannot be used to deduce the provenance of the litter or to predict its fate, crucial information for management and mitigation policies. Particle tracking techniques have seen extensive use in these roles, however, most previous studies have used simplistic initial conditions based on bulk average inputs of debris to the system. Here, observations of floating anthropogenic macro debris in the Adriatic Sea are used to define initial conditions (number of particles, location, and time) in a Lagrangian particle tracking model. Particles are advected backward and forward in time for 60 days (120 days total) using surface velocities from an operational regional ocean model. Sources and sinks for debris observed in the central and southern Adriatic in May 2013 and March 2015 included the Italian coastline from Pescara to Brindisi, the Croatian island of Mljet, and the coastline from Dubrovnik through Montenegro to Albania. Debris observed in the northern Adriatic originated from the Istrian peninsula to the Italian city of Termoli, as well as the Croatian island of Cres and the Kornati archipelago. Particles spent a total of roughly 47 days afloat. Coastal currents, notably the eastern and western Adriatic currents, resulted in large alongshore displacements. Our results indicate that anthropogenic macro debris originates largely from coastal sources near population centers and is advected by the cyclonic surface circulation until it strands on the southwest (Italian) coast, exits the Adriatic, or recirculates in the southern gyreVersión del edito

Central place foragers select ocean surface convergent features despite differing foraging strategies

Discovering the predictors of foraging locations can be challenging, and is often the critical missing piece for interpreting the ecological significance of observed movement patterns of predators. This is especially true in dynamic coastal marine systems, where planktonic food resources are diffuse and must be either physically or biologically concentrated to support upper trophic levels. In the Western Antarctic Peninsula, recent climate change has created new foraging sympatry between Adélie (Pygoscelis adeliae) and gentoo (P. papua) penguins in a known biological hotspot near Palmer Deep canyon. We used this recent sympatry as an opportunity to investigate how dynamic local oceanographic features affect aspects of the foraging ecology of these two species. Simulated particle trajectories from measured surface currents were used to investigate the co-occurrence of convergent ocean features and penguin foraging locations. Adélie penguin diving activity was restricted to the upper mixed layer, while gentoo penguins often foraged much deeper than the mixed layer, suggesting that Adélie penguins may be more responsive to dynamic surface convergent features compared to gentoo penguins. We found that, despite large differences in diving and foraging behavior, both shallow-diving Adélie and deeper-diving gentoo penguins strongly selected for surface convergent features. Furthermore, there was no difference in selectivity for shallow- versus deep-diving gentoo penguins. Our results suggest that these two mesopredators are selecting surface convergent features, however, how these surface signals are related to subsurface prey fields is unknown

Surface Ocean Dispersion Observations From the Ship-Tethered Aerostat Remote Sensing System

Oil slicks and sheens reside at the air-sea interface, a region of the ocean that is notoriously difficult to measure. Little is known about the velocity field at the sea surface in general, making predictions of oil dispersal difficult. The Ship-Tethered Aerostat Remote Sensing System (STARSS) was developed to measure Lagrangian velocities at the air-sea interface by tracking the transport and dispersion of bamboo dinner plates in the field of view of a high-resolution aerial imaging system. The camera had a field of view of approximately 300 × 200 m and images were obtained every 15 s over periods of up to 3 h. A series of experiments were conducted in the northern Gulf of Mexico in January-February 2016. STARSS was equipped with a GPS and inertial navigation system (INS) that was used to directly georectify the aerial images. A relative rectification technique was developed that translates and rotates the plates to minimize their total movement from one frame to the next. Rectified plate positions were used to quantify scale-dependent dispersion by computing relative dispersion, relative diffusivity, and velocity structure functions. STARSS was part of a nested observational framework, which included deployments of large numbers of GPS-tracked surface drifters from two ships, in situ ocean measurements, X-band radar observations of surface currents, and synoptic maps of sea surface temperature from a manned aircraft. Here we describe the STARSS system and image analysis techniques, and present results from an experiment that was conducted on a density front that was approximately 130 km offshore. These observations are the first of their kind and the methodology presented here can be adopted into existing and planned oceanographic campaigns to improve our understanding of small-scale and high-frequency variability at the air-sea interface and to provide much-needed benchmarks for numerical simulations

A single-point modeling approach for the intercomparison and evaluation of ozone dry deposition across chemical transport models (Activity 2 of AQMEII4)

A primary sink of air pollutants and their precursors is dry deposition. Dry deposition estimates differ across chemical transport models, yet an understanding of the model spread is incomplete. Here, we introduce Activity 2 of the Air Quality Model Evaluation International Initiative Phase 4 (AQMEII4). We examine 18 dry deposition schemes from regional and global chemical transport models as well as standalone models used for impact assessments or process understanding. We configure the schemes as single-point models at eight Northern Hemisphere locations with observed ozone fluxes. Single-point models are driven by a common set of site-specific meteorological and environmental conditions. Five of eight sites have at least 3 years and up to 12 years of ozone fluxes. The interquartile range across models in multiyear mean ozone deposition velocities ranges from a factor of 1.2 to 1.9 annually across sites and tends to be highest during winter compared with summer. No model is within 50 % of observed multiyear averages across all sites and seasons, but some models perform well for some sites and seasons. For the first time, we demonstrate how contributions from depositional pathways vary across models. Models can disagree with respect to relative contributions from the pathways, even when they predict similar deposition velocities, or agree with respect to the relative contributions but predict different deposition velocities. Both stomatal and nonstomatal uptake contribute to the large model spread across sites. Our findings are the beginning of results from AQMEII4 Activity 2, which brings scientists who model air quality and dry deposition together with scientists who measure ozone fluxes to evaluate and improve dry deposition schemes in the chemical transport models used for research, planning, and regulatory purposes

The 50- and 100-year Exceedance Probabilities as New and Convenient Statistics for a Frequency Analysis of Extreme Events: An Example of Extreme Precipitation in Israel

The misinterpreted statistics of extreme events’ probability in drainage design codes can lead to an insufficient capacity of drainage systems. This is one of the main factors of frequent floods, especially in coastal cities. Heavy rainfalls, the so-called “1-in-50-year” or “1-in-100-year” events followed by floods, may occur there every 2–3 years. We hope to contribute to the correct understanding of extreme events’ statistics. To achieve this goal, we first recall the technique of calculating the exceedance probability of the extreme event over a period of years. Then, to illustrate the theoretical results (among which is the well-known 63.2% minimal exceedance probability over a period of N years for an event with a 1/N annual exceedance probability), we use the rainfall intensity data archived at the Israel Meteorological Service for 1938–2021. We estimate their exceedance probabilities over periods of 50 and 100 years and offer the latter as convenient measures for comparative analysis of the intensity of extreme events. These illustrations highlight the importance of considering return periods that are significantly longer than the 50- or 100-year periods currently used in drainage design, to reduce the risk of flooding and damage caused by heavy rainfalls

Improved Gridded Precipitation Data Derived from Microwave Link Attenuation

The motivation for improving gridded precipitation data lies in weather now-casting and flood forecasting. Therefore, over the past decade, Commercial Microwave Link (CML) attenuation data have been used to determine rain rates between microwave antennas, and to produce more accurate countrywide precipitation grids. CML networks offer a unique advantage for precipitation measurements due to their high density. However, these data experience uncertainty from several sources as reported in earlier research. This current work determines the reliability of rainfall measurements for each link by comparing CML-derived rain rates to adjusted weather radar rainfall at the link location, over three months. Dynamic Time Warping (DTW) is applied to the pair of CML/radar time-series data in two study areas, Israel and Netherlands. Based on the DTW amplitude and temporal distance, unreliable links are identified and flagged, and interpolated gridded precipitation data are derived in each country after filtering out those unreliable links. Correlations between CML-derived grids and rain observations from an independent set of gauges, tested over several rain events in both study areas, are higher for the reliable subset of CML than the full set. For certain storm events, the Kendall rank correlation for the set of reliable CML is almost double that of the complete set, demonstrating that improved gridded precipitation data can be obtained by removing unreliable links