Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation

Turbulent flows have historically presented formidable challenges to predictive computational modeling. Traditional numerical simulations often require vast computational resources, making them infeasible for numerous engineering applications. As an alternative, deep learning-based surrogate models have emerged, offering data-drive solutions. However, these are typically constructed within deterministic settings, leading to shortfall in capturing the innate chaotic and stochastic behaviors of turbulent dynamics. We introduce a novel generative framework grounded in probabilistic diffusion models for versatile generation of spatiotemporal turbulence. Our method unifies both unconditional and conditional sampling strategies within a Bayesian framework, which can accommodate diverse conditioning scenarios, including those with a direct differentiable link between specified conditions and generated unsteady flow outcomes, and scenarios lacking such explicit correlations. A notable feature of our approach is the method proposed for long-span flow sequence generation, which is based on autoregressive gradient-based conditional sampling, eliminating the need for cumbersome retraining processes. We showcase the versatile turbulence generation capability of our framework through a suite of numerical experiments, including: 1) the synthesis of LES simulated instantaneous flow sequences from URANS inputs; 2) holistic generation of inhomogeneous, anisotropic wall-bounded turbulence, whether from given initial conditions, prescribed turbulence statistics, or entirely from scratch; 3) super-resolved generation of high-speed turbulent boundary layer flows from low-resolution data across a range of input resolutions. Collectively, our numerical experiments highlight the merit and transformative potential of the proposed methods, making a significant advance in the field of turbulence generation.Comment: 37 pages, 31 figure

Variational Bayesian inference with complex geostatistical priors using inverse autoregressive flows

We combine inverse autoregressive flows (IAF) and variational Bayesian inference (variational Bayes) in the context of geophysical inversion parameterized with deep generative models encoding complex priors. Variational Bayes approximates the unnormalized posterior distribution parametrically within a given family of distributions by solving an optimization problem. Although prone to bias if the chosen family of distributions is too limited, it provides a computationally-efficient approach that scales well to high-dimensional inverse problems. To enhance the expressiveness of the variational distribution, we explore its combination with IAFs that allow samples from a simple base distribution to be pushed forward through a series of invertible transformations onto an approximate posterior. The IAF is learned by maximizing the lower bound of the evidence (marginal likelihood), which is equivalent to minimizing the Kullback–Leibler divergence between the approximation and the target posterior distribution. In our examples, we use either a deep generative adversarial network (GAN) or a variational autoencoder (VAE) to parameterize complex geostatistical priors. Although previous attempts to perform Gauss–Newton inversion in combination with GANs of the same architecture were proven unsuccessful, the trained IAF provides a good reconstruction of channelized subsurface models for both GAN- and VAE-based inversions using synthetic crosshole ground-penetrating-radar data. For the considered examples, the computational cost of our approach is seven times lower than for Markov chain Monte Carlo (MCMC) inversion. Furthermore, the VAE-based approximations in the latent space are in good agreement. The VAE-based inversion requires only one sample to estimate gradients with respect to the IAF parameters at each iteration, while the GAN-based inversions need more samples and the corresponding posterior approximation is less accurate

Turbulent Flow Simulation using Autoregressive Conditional Diffusion Models

Simulating turbulent flows is crucial for a wide range of applications, and machine learning-based solvers are gaining increasing relevance. However, achieving stability when generalizing to longer rollout horizons remains a persistent challenge for learned PDE solvers. We address this challenge by introducing a fully data-driven fluid solver that utilizes an autoregressive rollout based on conditional diffusion models. We show that this approach offers clear advantages in terms of rollout stability compared to other learned baselines. Remarkably, these improvements in stability are achieved without compromising the quality of generated samples, and our model successfully generalizes to flow parameters beyond the training regime. Additionally, the probabilistic nature of the diffusion approach allows for inferring predictions that align with the statistics of the underlying physics. We quantitatively and qualitatively evaluate the performance of our method on a range of challenging scenarios, including incompressible and transonic flows, as well as isotropic turbulence.Comment: Source code available at https://github.com/tum-pbs/autoreg-pde-diffusio

Sensing coherent structures from the wall

Any device or application in which a solid body and a fluid are in relative motion has to deal with the presence of drag forces. Counteract such forces in an effcient and environmentally-friendly way has become very important in recent years, in view of the sustainable-development challenges imposed by global authorities. The interaction between any engineering device and a surrounding uid occurs most of the time in a turbulent regime. Despite the chaotic appearance of any turbulent ow, a close inspection reveals the presence of patterns that maintain their structure over a period of time. These patterns, known as coherent structures, are of utmost importance, since they are a low-dimensional expression of a complex highly-dimensional dynamical system that can be used to target affordable control strategies. Taking into account that control models rely on proper identi cation of the ow state, it is necessary to acquire this information in a way that is su ciently robust for real applications. For example, the experimental tools normally used in wind tunnels (for instance, velocity eld measurement techniques like Particle Image Velocimetry) cannot be used in an airborne ight. It becomes necessary to acquire this information from sensors that can be embedded in the device itself. The present thesis attempts to provide a technique based on deep neural networks that allows to reconstruct turbulent ows and their energy-carrying coherent structures from sensors embedded in the wall. The choice of neural networks, characterized by their ability to approximate non-linear relations, is based on the development of new algorithms that has occurred in recent years due to the availability of new computational resources. The rst part the dissertation aims to develop a model that combines the ability of prediction in nonlinear systems of deep neural networks with the compression capabilities of proper orthogonal decomposition (POD). For that purpose, a direct numerical simulation (DNS) of a turbulent channel ow of Ret = 1000 is used. Using wall-shear-stress information, the proposed architecture extracts information into feature maps through a series of convolutional neural networks, which are later converted in POD modes coefficients by using fully-connected neural networks. Since the rst POD modes can be ascribed to the most energetic structures in any ow, the proposed networks aim to provide an estimation of the fi rst 10 POD modes. This approach allows to reconstruct wall-parallel ow elds containing the most energetic structures in the ow. This approach is compared with a baseline linear method, proving that neural networks are able to trace better the nonlinear effect of turbulent ows in the wall-normal direction. The next step is to improve this model to reconstruct as much ow scales as possible. For such purpose, a DNS of turbulent open-channel ow is used, with Ret = [180-550]. The aforementioned network is transformed into a fully convolutional network, which allows to exploit better the spatial organization of the POD modes and retrieve a signifi cant larger amount of flow energy, surpassing 90%. The results show that the prediction quality decays with increasing wall distance. This is because the imprint of the flow structures at the wall is related in a linear way for all scales that have a size of the order of the distance to the wall. In contrast, the smaller scales do not impose their imprint directly on the wall, and the nonlinear relationship between the two attenuates with distance. For these results, the energy spectrum of the flow has also been evaluated, showing that the accuracy loss of the predictions begins with the smallest scales. However, the predictions are capable of recovering the largest structures of the ow, which are the ones that transport the bulk of energy. Furthermore, this network is compared with a network that does not exploit the compressibility advantages of POD. The results are overall similar, although the network based on POD appear slightly more robust for increasing distance from the wall. While the previous networks have used fully-resolved wall information coming from DNS simulation, n real applications it is often difficult to embed such a ne mesh of sensors in the wall. Among the different types of architecture that comprise neural networks, generative adversarial networks (GANs) have stood out for their ability to recover the small-scale details of low-resolution images. In fact, they have already been proposed as a tool to recover resolution in flow measurements. In this thesis the use of GANs to retrieve high-resolution wall information is presented. GANs performance is evaluated for different information loss ratios, showing that even in extreme cases GANs are capable of recovering the most energetic scales of the flow footprint at the wall. Finally, as the use of GANs has been proven successful for both wall and flow fields, it is proposed to use them to make a direct estimation of the flow from wall quantities. The results offer a signi cant improvement in flow reconstruction when compared to the previous results. This improvement can be ascribed to the complexity of GAN architectures, which allows to optimize better the amount of information fed into the network without suffering vanishing-gradient problems. The reconstruction performances are also tested with low-resolution input data. Although the predictions get worse when moving away from the wall also in this case, the rate is lower than in the previous cases. This allows to have reconstructions that capture the coherent structures away from the wall.Cualquier dispositivo o aplicación en la que un cuerpo sólido y un fluido estén en movimiento relativo tiene que lidiar con la presencia de fuerzas de arrastre. A la vista de los desafíos de desarrollo sostenible que han impuesto las autoridades mundiales, contrarrestar estas fuerzas de una manera e ficiente y amistosa con el medio ambiente se ha convertido en un tema de suma importancia. La interacción entre un dispositivo de carácter industrial y el fluido que lo rodea ocurre la mayor parte del tiempo en el régimen turbulento. A pesar de la apariencia caótica que tienen los flujos turbulentos, una inspección de cerca revela la presencia de patrones que mantienen su estructura durante algún tiempo. Estos patrones, conocidos como estructuras coherentes, son de suma importancia, ya que son una expresión de baja dimensión de un sistema dinámico complejo de alta dimensionalidad que puede usarse para desarrollar estrategias de control asequibles. Teniendo en cuenta que los modelos de control se basan en la identi cación adecuada del estado del flujo, es necesario adquirir esta información de una manera que sea lo su ficientemente robusta para aplicaciones reales. Por ejemplo, las herramientas experimentales que se utilizan normalmente en los túneles de viento, como la velocimetría de imágenes de partículas para la medición del campo de velocidad, no se pueden utilizar durante el vuelo de un avión regular. Es necesario adquirir esta información a partir de sensores que se puedan incrustar en el propio dispositivo. La presente tesis intenta proporcionar una técnica basada en redes neuronales profundas que permita reconstruir flujos turbulentos y sus estructuras coherentes portadoras de energía a partir de sensores embebidos en la pared. La elección de las redes neuronales, caracterizadas por su capacidad para aproximar relaciones no lineales, se basa en el desarrollo de nuevos algoritmos que han aparecido en los últimos años, gracias a los avances de los nuevos recursos computacionales. La primera parte de esta disertación tiene como objetivo desarrollar un modelo que combine la capacidad de predicción en sistemas no lineales de redes neuronales profundas con las capacidades de compresión de la Descomposición Modal Ortogonal (Proper Orthogonal Decomposition, POD). Para ello, se utiliza una simulación numérica directa (Direct Numerical Simulation, DNS) de un flujo de canal turbulento, con Ret = 1000. Midiendo los esfuerzos de cortadura en la pared, la arquitectura propuesta extrae información en mapas de características a través de una serie de redes neuronales convolucionales, que luego se convierten en coe cientes de modos POD mediante el uso de redes neuronales completamente conectadas. Dado que los primeros modos POD se pueden atribuir a las estructuras más energéticas en cualquier flujo, las redes propuestas tienen como objetivo proporcionar una estimación de los primeros 10 modos POD. Este enfoque permite reconstruir campos de ujo paralelos a la pared que contienen sus estructuras más energéticas. Este método propuesto se compara con un método lineal de referencia, lo que demuestra que las redes neuronales pueden rastrear mejor el efecto no lineal de los flujos turbulentos en la dirección normal de la pared. El siguiente paso es mejorar este modelo para reconstruir tantas escalas del flujo como sea posible. Para ello se utiliza un DNS de flujo turbulento en canal abierto, con Ret = [180-550]. La mencionada red se transforma en una red totalmente convolucional, lo que permite aprovechar mejor la organización espacial de los modos POD y recuperar una cantidad signi ficativamente mayor de energía de flujo, superando el 90%. Los resultados muestran que la calidad de la predicción decae a medida que aumenta la distancia a la pared. Esto se debe a que la huella de las estructuras del flujo en la pared esáa relacionada de forma lineal para todas las escalas que tienen un tamaño del orden de la distancia a la pared. Por el contrario, las escalas más pequeñas no imponen su huella directamente en la pared y la relación no lineal entre las dos se atenúa con la distancia. Para estos resultados, también se ha evaluado el espectro de energía del flujo, mostrando que la pérdida de precisión de las predicciones comienza con las escalas más pequeñas. Sin embargo, las predicciones son capaces de recuperar las estructuras más grandes del flujo, que son las que transportan la mayor parte de la energía. Además, esta red se compara con una red que no aprovecha las ventajas de compresibilidad de la POD. Los resultados son en general similares, aunque la red basada en la POD parece ligeramente más robusta cuando aumenta la distancia desde la pared. Si bien las redes anteriores han utilizado información de pared completamente resuelta proveniente de la simulación de DNS, en aplicaciones reales es difícil incrustar una malla tan fi na de sensores en la pared. Entre los distintos tipos de arquitectura que componen las redes neuronales, las redes de generación por confrontación (Generative Adversarial Networks, GAN) se han destacado por su capacidad para recuperar detalles a pequeña escala en imágenes de baja resolución. De hecho, ya se han propuesto como herramienta para recuperar la resolución en medidas de flujo. En esta tesis se presenta una alternativa basada en GANs para recuperar información de pared de alta resolución. Se han evaluado las prestaciones de las GANs para diferentes ratios de pérdida de información, demostrando que incluso en casos extremos las GANs son capaces de recuperar las escalas más energéticas de la huella de flujo en la pared. Finalmente, dado que el uso de las GANs ha demostrado ser exitoso tanto para los campos de pared como para los de flujo, se propone usarlos para hacer una estimación directa del flujo a partir de las cantidades de pared. Los resultados ofrecen una mejora signifi cativa en la reconstrucción del flujo en comparación con los resultados anteriores. Esta mejora se puede atribuir a la complejidad de las arquitecturas GAN, que permite optimizar mejor la cantidad de información introducida en la red sin sufrir problemas de desvanecimiento de gradiente. Las prestaciones de reconstrucción también se han probado con datos de entrada de baja resolución. Aunque en este caso las predicciones también empeoran al alejarse del muro, la tasa de disminución es menor que en los casos anteriores. Esto permite tener reconstrucciones que capturan las estructuras coherentes lejos de la pared.Programa de Doctorado en Mecánica de Fluidos por la Universidad Carlos III de Madrid, la Universidad de Jaén, la Universidad de Zaragoza, la Universidad Nacional de Educación a Distancia, la Universidad Politécnica de Madrid y la Universidad Rovira i VirgiliPresidente: Javier Jiménez Sendín.- Secretario: Manuel García-Villalba Navaridas.- Vocal: Sergio Hoyas Calv

Semi-conditional variational auto-encoder for flow reconstruction and uncertainty quantification from limited observations

We present a new data-driven model to reconstruct nonlinear flow from spatially sparse observations. The proposed model is a version of a Conditional Variational Auto-Encoder (CVAE), which allows for probabilistic reconstruction and thus uncertainty quantification of the prediction. We show that in our model, conditioning on measurements from the complete flow data leads to a CVAE where only the decoder depends on the measurements. For this reason, we call the model semi-conditional variational autoencoder. The method, reconstructions, and associated uncertainty estimates are illustrated on the velocity data from simulations of 2D flow around a cylinder and bottom currents from a simulation of the southern North Sea by the Bergen Ocean Model. The reconstruction errors are compared to those of the Gappy proper orthogonal decomposition method.publishedVersio