Interpretable structural model error discovery from sparse assimilation increments using spectral bias-reduced neural networks: A quasi-geostrophic turbulence test case

Earth system models suffer from various structural and parametric errors in their representation of nonlinear, multi-scale processes, leading to uncertainties in their long-term projections. The effects of many of these errors (particularly those due to fast physics) can be quantified in short-term simulations, e.g., as differences between the predicted and observed states (analysis increments). With the increase in the availability of high-quality observations and simulations, learning nudging from these increments to correct model errors has become an active research area. However, most studies focus on using neural networks, which while powerful, are hard to interpret, are data-hungry, and poorly generalize out-of-distribution. Here, we show the capabilities of Model Error Discovery with Interpretability and Data Assimilation (MEDIDA), a general, data-efficient framework that uses sparsity-promoting equation-discovery techniques to learn model errors from analysis increments. Using two-layer quasi-geostrophic turbulence as the test case, MEDIDA is shown to successfully discover various linear and nonlinear structural/parametric errors when full observations are available. Discovery from spatially sparse observations is found to require highly accurate interpolation schemes. While NNs have shown success as interpolators in recent studies, here, they are found inadequate due to their inability to accurately represent small scales, a phenomenon known as spectral bias. We show that a general remedy, adding a random Fourier feature layer to the NN, resolves this issue enabling MEDIDA to successfully discover model errors from sparse observations. These promising results suggest that with further development, MEDIDA could be scaled up to models of the Earth system and real observations.Comment: 26 pages, 5+1 figure

Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES

We demonstrate how incorporating physics constraints into convolutional neural networks (CNNs) enables learning subgrid-scale (SGS) closures for stable and accurate large-eddy simulations (LES) in the small-data regime (i.e., when the availability of high-quality training data is limited). Using several setups of forced 2D turbulence as the testbeds, we examine the {\it a priori} and {\it a posteriori} performance of three methods for incorporating physics: 1) data augmentation (DA), 2) CNN with group convolutions (GCNN), and 3) loss functions that enforce a global enstrophy-transfer conservation (EnsCon). While the data-driven closures from physics-agnostic CNNs trained in the big-data regime are accurate and stable, and outperform dynamic Smagorinsky (DSMAG) closures, their performance substantially deteriorate when these CNNs are trained with 40x fewer samples (the small-data regime). We show that CNN with DA and GCNN address this issue and each produce accurate and stable data-driven closures in the small-data regime. Despite its simplicity, DA, which adds appropriately rotated samples to the training set, performs as well or in some cases even better than GCNN, which uses a sophisticated equivariance-preserving architecture. EnsCon, which combines structural modeling with aspect of functional modeling, also produces accurate and stable closures in the small-data regime. Overall, GCNN+EnCon, which combines these two physics constraints, shows the best {\it a posteriori} performance in this regime. These results illustrate the power of physics-constrained learning in the small-data regime for accurate and stable LES.Comment: 23 pages, 9 figure

Next Generation Exa-Scale Capable Software for High Fidelity Physics Aware Simulations

Physics aware simulations, often arising in problems of fluid dynamics, aerodynamics and multi-physics areas have demanded the need for computing software that has the capability to resolve the complexities at multiple scales to analyze and visualize the effects of their interactions with the surroundings. Usually the governing dynamics of these phenomenon appear in the form of complex partial differential equations whose numerical solutions impose various constraints on computational complexity, programming time and efficient throughput. In this scenario, the need of computing software that can solve very large problems resolving the physics of these phenomenon at multiple scales is imperative. Despite traditional computing capabilities in today\u27s hardware through massively parallel systems, optimization and tuning of legacy physics code are usually constrained to specific super-computing clusters and often fail to reproduce similar efficiencies on others. With the dawn of heterogeneous computing systems equipped with accelerators, optimized code that is portable on different systems with varying architectures is a necessity. Such code exploits the advantages of specific hardware capabilities and scales sufficiently for very large and highly nonlinear problems. In this context Sandia National Labs\u27 Trilinos and Kokkos libraries with inherently optimized parallelism and performance portability layers provide a suitable abstraction to build APIs (application programming interfaces) that can model complex physics at multiple scales with a very high degree of fidelity while scaling on massively parallel computers and heterogeneous computing architectures requiring little to no modification of source code. This Thesis discusses the development of two such APIs that have been built to solve a range of different fluid dynamics problems and demonstrate the physics that they can simulate. At the same time the different performance metrics obtained from testing these APIs on different supercomputing platforms have been discussed

Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems

International audienceData assimilation (DA) is a key component of many forecasting models in science and engineering. DA allows one to estimate better initial conditions using an imperfect dynamical model of the system and noisy/sparse observations available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that is widely used in applications involving high-dimensional nonlinear dynamical systems. However, EnKF requires evolving large ensembles of forecasts using the dynamical model of the system. This often becomes computationally intractable, especially when the number of states of the system is very large, e.g., for weather prediction. With small ensembles, the estimated background error covariance matrix in the EnKF algorithm suffers from sampling error, leading to an erroneous estimate of the analysis state (initial condition for the next forecast cycle). In this work, we propose hybrid ensemble Kalman filter (H-EnKF), which is applied to a two-layer quasi-geostrophic turbulent flow as a test case. This framework utilizes a pre-trained deep learning-based data-driven surrogate that inexpensively generates and evolves a large data-driven ensemble of the states to accurately compute the background error covariance matrix with smaller sampling errors. The H-EnKF framework outperforms EnKF with only dynamical model or only the data-driven surrogate, and estimates a better initial condition without the need for any ad-hoc localization strategies. H-EnKF can be extended to any ensemble-based DA algorithm, e.g., particle filters, which are currently too expensive to use for high-dimensional systems