A Fortran-Keras Deep Learning Bridge for Scientific Computing

Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. These software libraries come preloaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model’s emergent behavior to be assessed, i.e., when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of the optimizer proves unexpectedly critical. This in turn reveals many new neural network architectures that produce considerable improvements in climate model stability including some with reduced error, for an especially challenging training dataset

A Fortran-Keras Deep Learning Bridge for Scientific Computing

Implementing artificial neural networks is commonly achieved via high-level programming languages like Python and easy-to-use deep learning libraries like Keras. These software libraries come pre-loaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful, with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model's emergent behavior to be assessed, i.e. when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of optimizer proves unexpectedly critical. This reveals many neural network architectures that produce considerable improvements in stability including some with reduced error, for an especially challenging training dataset

Towards Exascale CFD Simulations Using the Discontinuous Galerkin Solver FLEXI

Modern high-order discretizations bear considerable potential for the exascale era due to their high fidelity and the high, local computational load that allows for computational efficiency in massively parallel simulations. To this end, the discontinuous Galerkin (DG) framework FLEXI was selected to demonstrate exascale readiness within the Center of Excellence for Exascale CFD (CEEC) by simulating shock buffet on a three-dimensional wing segment at transsonic flight conditions. This paper summarizes the recent progress made to enable the simulation of this challenging exascale problem. For this, it is first demonstrated that FLEXI scales excellently to over 500 000 CPU cores on HAWK at the HLRS. To tackle the considerable resolution requirements near the wall, a novel wall model is proposed that takes compressibility effects into account and yields decent results for the simulation of a NACA 64A-110 airfoil. To address the shocks in the domain, a finite-volume-based shock capturing method was implemented in FLEXI, which is validated here using the simulation of a linear compressor cascade at supersonic flow conditions, where the method is demonstrated to yield efficient, robust and accurate results. Lastly, we present the TensorFlow-Fortran-Binding (TFFB) as an easy-to-use library to deploy trained machine learning models in Fortran solvers such as FLEXI.Comment: 15 pages, 5 figure

Investigating Model-Data Inconsistency in Data-Informed Turbulence Closure Terms

In the present work, we investigate the stability of turbulence closure predictions from neural network models and highlight the role of model-data-inconsistency during inference. We quantify this inconsistency by applying the Mahalanobis distance and demonstrate that the instability of the model predictions in practical large eddy simulations (LES) correlates with a deviation of the input data between the training dataset and actual simulation data. Moreover, the method of 'stability training' is applied to increase the robustness of recurrent artificial neural networks (ANN) against small perturbations in the input, which are typically unavoidable in any practical scenario. We show that this method can increase the stability of simulations with ANN-based closure term predictions significantly. The models also achieve good accuracy on the blind testing set in comparison to the baseline model trained without stability training. The work presented here can thus be seen as a building block towards long-term stable data-driven models for dynamical systems and highlights methods to detect and counter model-data-inconsistencies

A Comprehensive Survey of Deep Learning Models Based on Keras Framework

Python is one of the most widely adopted programming languages, having replaced a number of those in the field. Python is popular with developers for a variety of reasons, one of which is because it has an incredibly diverse collection of libraries in which users can run. This paper provides the most current survey on Keras, which is a Python-based deep learning Application Programming Interface (API) that runs on top of the machine learning framework TensorFlow. The mentions library is used in conjunction with TensorFlow, PyTorch, CODEEPNEATM, and Pygame to allow integration of deep learning models such as cardiovascular disease diagnostics, graph neural networks, identify health issues, COVID-19 recognition, skin tumors, image detection, and so on, in the applied area. Furthermore, the author used Keras details, goals, challenges, and significant outcomes, as well as the findings, obtained using this method.   Keywords

Sensitivity of air pollution exposure and disease burden to emission changes in China using machine learning emulation

Machine learning models can emulate chemical transport models, reducing computational costs and enabling more experimentation. We developed emulators to predict annual−mean fine particulate matter (PM(2.5)) and ozone (O(3)) concentrations and their associated chronic health impacts from changes in five major emission sectors (residential, industrial, land transport, agriculture, and power generation) in China. The emulators predicted 99.9% of the variance in PM(2.5) and O(3) concentrations. We used these emulators to estimate how emission reductions can attain air quality targets. In 2015, we estimate that PM(2.5) exposure was 47.4 μg m(−3) and O(3) exposure was 43.8 ppb, associated with 2,189,700 (95% uncertainty interval, 95UI: 1,948,000–2,427,300) premature deaths per year, primarily from PM(2.5) exposure (98%). PM(2.5) exposure and the associated disease burden were most sensitive to industry and residential emissions. We explore the sensitivity of exposure and health to different combinations of emission reductions. The National Air Quality Target (35 μg m(−3)) for PM(2.5) concentrations can be attained nationally with emission reductions of 72% in industrial, 57% in residential, 36% in land transport, 35% in agricultural, and 33% in power generation emissions. We show that complete removal of emissions from these five sectors does not enable the attainment of the WHO Annual Guideline (5 μg m(−3)) due to remaining air pollution from other sources. Our work provides the first assessment of how air pollution exposure and disease burden in China varies as emissions change across these five sectors and highlights the value of emulators in air quality research

Assessing the Potential of Deep Learning for Emulating Cloud Superparameterization in Climate Models with Real-Geography Boundary Conditions

We explore the potential of feed-forward deep neural networks (DNNs) for emulating cloud superparameterization in realistic geography, using offline fits to data from the Super Parameterized Community Atmospheric Model. To identify the network architecture of greatest skill, we formally optimize hyperparameters using ~250 trials. Our DNN explains over 70 percent of the temporal variance at the 15-minute sampling scale throughout the mid-to-upper troposphere. Autocorrelation timescale analysis compared against DNN skill suggests the less good fit in the tropical, marine boundary layer is driven by neural network difficulty emulating fast, stochastic signals in convection. However, spectral analysis in the temporal domain indicates skillful emulation of signals on diurnal to synoptic scales. A close look at the diurnal cycle reveals correct emulation of land-sea contrasts and vertical structure in the heating and moistening fields, but some distortion of precipitation. Sensitivity tests targeting precipitation skill reveal complementary effects of adding positive constraints vs. hyperparameter tuning, motivating the use of both in the future. A first attempt to force an offline land model with DNN emulated atmospheric fields produces reassuring results further supporting neural network emulation viability in real-geography settings. Overall, the fit skill is competitive with recent attempts by sophisticated Residual and Convolutional Neural Network architectures trained on added information, including memory of past states. Our results confirm the parameterizability of superparameterized convection with continents through machine learning and we highlight advantages of casting this problem locally in space and time for accurate emulation and hopefully quick implementation of hybrid climate models.Comment: 32 Pages, 13 Figures, Revised Version Submitted to Journal of Advances in Modeling Earth Systems April 202