The impact of observations in convective-scale numerical weather prediction

The accuracy of the initial conditions strongly determines the skill of numerical weather prediction (NWP). Data assimilation systems combine millions of observations with the latest short-range forecast to provide optimal initial conditions. Only recently, NWP centers are capable of performing high-resolution, convection-permitting forecasts on a regional scale. However, moving to a higher model resolution involves several challenges concerning observations and the underlying data assimilation algorithm. The chaotic nature and limited predictability of convection calls for spatially and temporally high resolved observations. However, limited knowledge exists on which observations are most important for high-resolution NWP. Hence, a better understanding of the impact of different observations on these scales is required to improve current data assimilation, forecasting, and observing systems. Furthermore, knowledge of the potential impact of observations is needed to develop advanced observation and data assimilation strategies for future convective-scale NWP. This thesis, therefore, investigates the impact of observations in convective-scale ensemble forecasting. The impact of assimilated observation and the potential impact of future observations is evaluated by applying two complementary ensemble-based methods. Both methods rely on sample correlations that are estimated with an ensemble. However, state of the art ensemble prediction systems usually provide ensembles with only 20-250 members for estimating the uncertainty of the forecast and its spatial and temporal covariance. Given that the degrees of freedom of atmospheric models are several magnitudes higher, sample correlations are significantly affected by sampling errors. Therefore, sampling errors pose an issue for the impact assessment and in many other ensemble applications. Thus, it is essential to quantify sampling errors on convective-scales and to find methods to mitigate sampling errors. To address the previously discussed challenges, this dissertation aims to estimate the impact of observations and to reduce the issue of sampling error in convective-scale modeling and ensemble diagnostics. The first part of this thesis evaluates the impact of about 3 million conventional observations in the regional ensemble forecasting system of Deutscher Wetterdienst. This study presents the first evaluation of ensemble-based estimates of observation impact over an extended period of six weeks in a convection-permitting modeling system. Nearly all previous observation impact studies used the difference between the forecast and subsequent analysis of the same modeling system for verification. However, this kind of verification does not adequately reflect relevant forecast aspects of convective-scale forecasting. Hence, the observation impact is examined for different observation-based verification norms. The second part introduces an approach for estimating the relative potential impact of different observable quantities in convective-scale modeling. The approach is based on ensemble sensitivity analysis and uses accumulated squared spatiotemporal correlations as a proxy for the potential impact. To obtain reliable spatiotemporal correlations, a very large ensemble is required. Therefore, an unprecedented convective-scale 1000-member ensemble was computed in collaboration with the RIKEN Institute for computational science. This simulation allows to examine the sensitivity of the approach to the ensemble size. The present study further highlights the scale dependence of the potential impact and provides the basis for developing better observation and data assimilation strategies. The third part uses the 1000-member ensemble simulation as truth to quantify sampling errors on convective-scales and to evaluate a statistical sampling error correction. The sampling error correction is a simple look-up table based approach and aims to reduce spurious correlations. A detailed evaluation for spatiotemporal correlations shows that the sampling error correction significantly reduces sampling errors in sample correlations that are required for estimating the impact of observations. Additionally, the study demonstrates the great potential of the sampling error correction method for data assimilation where it could replace distance-based localization techniques and thereby increase the impact of observations

A convective-scale 1,000-member ensemble simulation and potential applications

This study presents the first convective-scale 1,000-member ensemble simulation over central Europe, which provides a unique data set for various applications. A comparison with the operational regional 40-member ensemble of Deutscher Wetterdienst shows that the 1,000-member simulation exhibits realistic spread properties overall. Based on this, we discuss two potential applications. First, we quantify the sampling error of spatial covariances of smaller subsets compared with the 1,000-member simulation. Knowledge about sampling errors and their dependence on ensemble size is crucial for ensemble and hybrid data assimilation and for developing better approaches for localization in this context. Secondly, we present an approach for estimating the relative potential impact of different observable quantities using ensemble sensitivity analysis. This will provide the basis for consecutive studies developing future observation and data assimilation strategies. Sensitivity studies on the ensemble size indicate that about 200 ensemble members are required to estimate the potential impact of observable quantities with respect to precipitation forecasts.Fil: Necker, Tobias. Ludwig Maximilians Universitat; Alemania. Universidad de Viena; AustriaFil: Geiss, Stefan. Ludwig Maximilians Universitat; AlemaniaFil: Weissmann, Martin. Ludwig Maximilians Universitat; AlemaniaFil: Ruiz, Juan Jose. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Centro de Investigaciones del Mar y la Atmósfera. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Centro de Investigaciones del Mar y la Atmósfera; ArgentinaFil: Miyoshi, Takemasa. RIKEN Center for Computational Science; JapónFil: Lien, Guo Yuan. RIKEN Center for Computational Science; Japó

Distributions and convergence of forecast variables in a 1,000-member convection-permitting ensemble

The errors in numerical weather forecasts resulting from limited ensemble size are explored using 1,000-member forecasts of convective weather over Germany at 3-km resolution. A large number of forecast variables at different lead times were examined, and their distributions could be classified into three categories: quasi-normal (e.g., tropospheric temperature), highly skewed (e.g. precipitation), and mixtures (e.g., humidity). Dependence on ensemble size was examined in comparison to the asymptotic convergence law that the sampling error decreases proportional to N−1/2 for large enough ensemble size N, independent of the underlying distribution shape. The asymptotic convergence behavior was observed for the ensemble mean of all forecast variables, even for ensemble sizes less than 10. For the ensemble standard deviation, sizes of up to 100 were required for the convergence law to apply. In contrast, there was no clear sign of convergence for the 95th percentile even with 1,000 members. Methods such as neighborhood statistics or prediction of area-averaged quantities were found to improve accuracy, but only for variables with random small-scale variability, such as convective precipitation.Fil: Craig, George C.. Ludwig Maximilians Universitat; AlemaniaFil: Puh, Matjaž. Ludwig Maximilians Universitat; AlemaniaFil: Keil, Christian. Ludwig Maximilians Universitat; AlemaniaFil: Tempest, Kirsten. Ludwig Maximilians Universitat; AlemaniaFil: Necker, Tobias. Universidad de Viena; AustriaFil: Ruiz, Juan Jose. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Centro de Investigaciones del Mar y la Atmósfera. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Centro de Investigaciones del Mar y la Atmósfera; Argentina. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Ciencias de la Atmósfera y los Océanos; ArgentinaFil: Weissmann, Martin. Universidad de Viena; AustriaFil: Miyoshi, Takemasa. Riken Center For Computational Science; Japó

Federated learning enables big data for rare cancer boundary detection.

Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing

Author Correction: Federated learning enables big data for rare cancer boundary detection.

10.1038/s41467-023-36188-7NATURE COMMUNICATIONS14

Non-standard neutrino interactions in IceCube

Non-standard neutrino interactions (NSI) may arise in various types of new physics. Their existence would change the potential that atmospheric neutrinos encounter when traversing Earth matter and hence alter their oscillation behavior. This imprint on coherent neutrino forward scattering can be probed using high-statistics neutrino experiments such as IceCube and its low-energy extension, DeepCore. Both provide extensive data samples that include all neutrino flavors, with oscillation baselines between tens of kilometers and the diameter of the Earth. DeepCore event energies reach from a few GeV up to the order of 100 GeV - which marks the lower threshold for higher energy IceCube atmospheric samples, ranging up to 10 TeV. In DeepCore data, the large sample size and energy range allow us to consider not only flavor-violating and flavor-nonuniversal NSI in the μ−τ sector, but also those involving electron flavor. The effective parameterization used in our analyses is independent of the underlying model and the new physics mass scale. In this way, competitive limits on several NSI parameters have been set in the past. The 8 years of data available now result in significantly improved sensitivities. This improvement stems not only from the increase in statistics but also from substantial improvement in the treatment of systematic uncertainties, background rejection and event reconstruction

IceCube Search for Earth-traversing ultra-high energy Neutrinos

The search for ultra-high energy neutrinos is more than half a century old. While the hunt for these neutrinos has led to major leaps in neutrino physics, including the detection of astrophysical neutrinos, neutrinos at the EeV energy scale remain undetected. Proposed strategies for the future have mostly been focused on direct detection of the first neutrino interaction, or the decay shower of the resulting charged particle. Here we present an analysis that uses, for the first time, an indirect detection strategy for EeV neutrinos. We focus on tau neutrinos that have traversed Earth, and show that they reach the IceCube detector, unabsorbed, at energies greater than 100 TeV for most trajectories. This opens up the search for ultra-high energy neutrinos to the entire sky. We use ten years of IceCube data to perform an analysis that looks for secondary neutrinos in the northern sky, and highlight the promise such a strategy can have in the next generation of experiments when combined with direct detection techniques