Deep Convolutional Architectures for Extrapolative Forecast in Time-dependent Flow Problems

Physical systems whose dynamics are governed by partial differential equations (PDEs) find applications in numerous fields, from engineering design to weather forecasting. The process of obtaining the solution from such PDEs may be computationally expensive for large-scale and parameterized problems. In this work, deep learning techniques developed especially for time-series forecasts, such as LSTM and TCN, or for spatial-feature extraction such as CNN, are employed to model the system dynamics for advection dominated problems. These models take as input a sequence of high-fidelity vector solutions for consecutive time-steps obtained from the PDEs and forecast the solutions for the subsequent time-steps using auto-regression; thereby reducing the computation time and power needed to obtain such high-fidelity solutions. The models are tested on numerical benchmarks (1D Burgers' equation and Stoker's dam break problem) to assess the long-term prediction accuracy, even outside the training domain (extrapolation). Non-intrusive reduced-order modelling techniques such as deep auto-encoder networks are utilized to compress the high-fidelity snapshots before feeding them as input to the forecasting models in order to reduce the complexity and the required computations in the online and offline stages. Deep ensembles are employed to perform uncertainty quantification of the forecasting models, which provides information about the variance of the predictions as a result of the epistemic uncertainties

Embedding Physics to Learn Spatiotemporal Dynamics from Sparse Data

Modeling nonlinear spatiotemporal dynamical systems has primarily relied on partial differential equations (PDEs) that are typically derived from first principles. However, the explicit formulation of PDEs for many underexplored processes, such as climate systems, biochemical reaction and epidemiology, remains uncertain or partially unknown, where very sparse measurement data is yet available. To tackle this challenge, we propose a novel deep learning architecture that forcibly embedded known physics knowledge in a residual-recurrent $\Pi$-block network, to facilitate the learning of the spatiotemporal dynamics in a data-driven manner. The coercive embedding mechanism of physics, fundamentally different from physics-informed neural networks based on loss penalty, ensures the network to rigorously obey given physics. Numerical experiments demonstrate that the resulting learning paradigm that embeds physics possesses remarkable accuracy, robustness, interpretability and generalizability for learning spatiotemporal dynamics.Comment: 18 pages. arXiv admin note: substantial text overlap with arXiv:2105.0055

NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition

Neural networks have shown great potential in accelerating the solution of partial differential equations (PDEs). Recently, there has been a growing interest in introducing physics constraints into training neural PDE solvers to reduce the use of costly data and improve the generalization ability. However, these physics constraints, based on certain finite dimensional approximations over the function space, must resolve the smallest scaled physics to ensure the accuracy and stability of the simulation, resulting in high computational costs from large input, output, and neural networks. This paper proposes a general acceleration methodology called NeuralStagger by spatially and temporally decomposing the original learning tasks into several coarser-resolution subtasks. We define a coarse-resolution neural solver for each subtask, which requires fewer computational resources, and jointly train them with the vanilla physics-constrained loss by simply arranging their outputs to reconstruct the original solution. Due to the perfect parallelism between them, the solution is achieved as fast as a coarse-resolution neural solver. In addition, the trained solvers bring the flexibility of simulating with multiple levels of resolution. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations, which leads to an additional $10\sim100\times$ speed-up. Moreover, the experiment also shows that the learned model could be well used for optimal control.Comment: ICML 2023 accepte

Étude de la propagation acoustique en milieu complexe par des réseaux de neurones profonds

Abstract : Predicting the propagation of aerocoustic noise is a challenging task in the presence of complex mean flows and geometry installation effects. The design of future quiet propul- sion systems requires tools that are able to perform many accurate evaluations with a low computational cost. Analytical models or hybrid numerical approaches have tradition- ally been employed for that purpose. However, such methods are typically constrained by simplifying hypotheses that are not easily relaxed. Thus, the main objective of this thesis is to develop and validate novel methods for the fast and accurate prediction of aeroacoustic propagation in complex mean flows and geometries. For that, data-driven deep convolutional neural networks acting as auto-regressive spatio-temporal predictors are considered. These surrogates are trained on high-fidelity data, generated by direct aeroacoustic numerical solvers. Such datasets are able to model complex flow phenomena, along with complex geometrical parameters. The neural network is designed to substitute the high-fidelity solver at a much lower computational cost once the training is finished, while predicting the time-domain acoustic propagation with sufficient accuracy. Three test cases of growing complexity are employed to test the approach, where the learned surrogate is compared to analytical and numerical solutions. The first one corresponds to the two-dimensional propagation of Gaussian pulses in closed domains, which allows understanding the fundamental behavior of the employed convolution neural networks. Second, the approach is extended in order to consider a variety of boundary conditions, from non-reflecting to curved reflecting obstacles, including the reflection and scattering of waves at obstacles. This allows the prediction of acoustic propagation in configurations closer to industrial problems. Finally, the effects of complex mean flows is investigated through a dataset of acoustic waves propagating inside sheared flows. These applications highlight the flexibility of the employed data-driven methods using convolutional neural networks. They allow a significant acceleration of the acoustic predictions, while keeping an adequate accuracy and being also able to correctly predict the acoustic propagation outside the range of the training data. For that, prior knowledge about the wave propa- gation physics is included during and after the neural network training phase, allowing an increased control over the error performed by the surrogate. Among this prior knowledge, the conservation of physics quantities and the correct treatment of boundary conditions are identified as key parameters that improve the surrogate predictions.Prédire la propagation du bruit aéroacoustique est une tâche difficile en présence d’écoulements moyens complexes et d’effets géométriques d’installation. La conception des futurs systèmes de propulsion silencieux appelle au développement d’outils capables d’effectuer de nombreuses évaluations avec une faible erreur et un faible coût de calcul. Traditionnellement, des modèles analytiques ou des approches numériques hybrides ont été utilisés à cette fin. Cependant, ces méthodes sont généralement contraintes par des hypothèses simplificatrices qui ne sont pas facilement assouplies. Ainsi, l’objectif principal de cette thèse est de développer et de valider de nouvelles méthodes pour la prédiction rapide et précise de la propagation aéroacoustique dans des écoulements moyens et des géométries complexes. Pour cela, des réseaux de neurones profonds à convolution, entraînés sur des données, et agissant comme prédicteurs spatio-temporels sont considérés. Ces modèles par substitution sont entraînés sur des données de haute fidélité, générées par des solveurs numériques aérocoustiques directs. De telles bases de données sont capables de modéliser des phénomènes d’écoulement, ainsi que des paramètres géométriques complexes. Le réseau de neurones est conçu pour remplacer le solveur haute fidélité à un coût de calcul beaucoup plus faible une fois la phase d’entraînement terminée, tout en prédisant la propagation acoustique dans le domaine temporel avec une précision suffisante. Trois cas de test, de complexité croissante, sont utilisés pour tester l’approche, où le substitut appris est comparé à des solutions analytiques et numériques. Le premier cas correspond à la propagation acoustique bidimensionnelle dans des domaines fermés, où des sources impulsionnelles Gaussiennes sont considérées. Ceci permet de comprendre le comportement fondamental des réseaux de neurones à convolution étudiés. Deuxièmement, l’approche est étendue afin de prendre en compte une variété de conditions aux limites, notamment des conditions aux limites non réfléchissantes et des obstacles réfléchissants de géométrie arbitraire, modélisant la réflexion et la diffusion des ondes acoustiques sur ces obstacles. Cela permet de prédire la propagation acoustique dans des configurations plus proches des problématiques industrielles. Enfin, les effets des écoulements moyens complexes sont étudiés à travers une base de données d’ondes acoustiques qui se propagent à l’intérieur d’écoulements cisaillés. Ces applications mettent en évidence la flexibilité des méthodes basées sur les données, utilisant des réseaux de neurones à convolution. Ils permettent une accélération significative des prédictions acoustiques, tout en gardant une précision adéquate et en étant également capables de prédire correctement la propagation acoustique en dehors de la gamme de paramètres des données d’apprentissage. Pour cela, des connaissances préalables sur la physique de propagation des ondes sont incluses pendant et après la phase d’apprentissage du réseau de neurones, permettant un contrôle accru sur l’erreur effectuée par le substitut. Parmi ces connaissances préalables, la conservation des grandeurs physiques et le traitement correct des conditions aux limites sont identifiés comme des paramètres clés qui améliorent les prédictions du modèle proposé

Probabilistic Physics-integrated Neural Differentiable Modeling for Isothermal Chemical Vapor Infiltration Process

Chemical vapor infiltration (CVI) is a widely adopted manufacturing technique used in producing carbon-carbon and carbon-silicon carbide composites. These materials are especially valued in the aerospace and automotive industries for their robust strength and lightweight characteristics. The densification process during CVI critically influences the final performance, quality, and consistency of these composite materials. Experimentally optimizing the CVI processes is challenging due to long experimental time and large optimization space. To address these challenges, this work takes a modeling-centric approach. Due to the complexities and limited experimental data of the isothermal CVI densification process, we have developed a data-driven predictive model using the physics-integrated neural differentiable (PiNDiff) modeling framework. An uncertainty quantification feature has been embedded within the PiNDiff method, bolstering the model's reliability and robustness. Through comprehensive numerical experiments involving both synthetic and real-world manufacturing data, the proposed method showcases its capability in modeling densification during the CVI process. This research highlights the potential of the PiNDiff framework as an instrumental tool for advancing our understanding, simulation, and optimization of the CVI manufacturing process, particularly when faced with sparse data and an incomplete description of the underlying physics

Bi-fidelity Variational Auto-encoder for Uncertainty Quantification

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. This model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space that is integrated within the VAE's probabilistic encoder-decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available