17,493 research outputs found
Deep Learning of Turbulent Scalar Mixing
Based on recent developments in physics-informed deep learning and deep
hidden physics models, we put forth a framework for discovering turbulence
models from scattered and potentially noisy spatio-temporal measurements of the
probability density function (PDF). The models are for the conditional expected
diffusion and the conditional expected dissipation of a Fickian scalar
described by its transported single-point PDF equation. The discovered model
are appraised against exact solution derived by the amplitude mapping closure
(AMC)/ Johnsohn-Edgeworth translation (JET) model of binary scalar mixing in
homogeneous turbulence.Comment: arXiv admin note: text overlap with arXiv:1808.04327,
arXiv:1808.0895
Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
In deep latent Gaussian models, the latent variable is generated by a
time-inhomogeneous Markov chain, where at each time step we pass the current
state through a parametric nonlinear map, such as a feedforward neural net, and
add a small independent Gaussian perturbation. This work considers the
diffusion limit of such models, where the number of layers tends to infinity,
while the step size and the noise variance tend to zero. The limiting latent
object is an It\^o diffusion process that solves a stochastic differential
equation (SDE) whose drift and diffusion coefficient are implemented by neural
nets. We develop a variational inference framework for these \textit{neural
SDEs} via stochastic automatic differentiation in Wiener space, where the
variational approximations to the posterior are obtained by Girsanov
(mean-shift) transformation of the standard Wiener process and the computation
of gradients is based on the theory of stochastic flows. This permits the use
of black-box SDE solvers and automatic differentiation for end-to-end
inference. Experimental results with synthetic data are provided
Monge-Amp\`ere Flow for Generative Modeling
We present a deep generative model, named Monge-Amp\`ere flow, which builds
on continuous-time gradient flow arising from the Monge-Amp\`ere equation in
optimal transport theory. The generative map from the latent space to the data
space follows a dynamical system, where a learnable potential function guides a
compressible fluid to flow towards the target density distribution. Training of
the model amounts to solving an optimal control problem. The Monge-Amp\`ere
flow has tractable likelihoods and supports efficient sampling and inference.
One can easily impose symmetry constraints in the generative model by designing
suitable scalar potential functions. We apply the approach to unsupervised
density estimation of the MNIST dataset and variational calculation of the
two-dimensional Ising model at the critical point. This approach brings
insights and techniques from Monge-Amp\`ere equation, optimal transport, and
fluid dynamics into reversible flow-based generative models
Deep Latent-Variable Kernel Learning
Deep kernel learning (DKL) leverages the connection between Gaussian process
(GP) and neural networks (NN) to build an end-to-end, hybrid model. It combines
the capability of NN to learn rich representations under massive data and the
non-parametric property of GP to achieve automatic regularization that
incorporates a trade-off between model fit and model complexity. However, the
deterministic encoder may weaken the model regularization of the following GP
part, especially on small datasets, due to the free latent representation. We
therefore present a complete deep latent-variable kernel learning (DLVKL) model
wherein the latent variables perform stochastic encoding for regularized
representation. We further enhance the DLVKL from two aspects: (i) the
expressive variational posterior through neural stochastic differential
equation (NSDE) to improve the approximation quality, and (ii) the hybrid prior
taking knowledge from both the SDE prior and the posterior to arrive at a
flexible trade-off. Intensive experiments imply that the DLVKL-NSDE performs
similarly to the well calibrated GP on small datasets, and outperforms existing
deep GPs on large datasets.Comment: 13 pages, 8 figures, preprint under revie
Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
We present hidden fluid mechanics (HFM), a physics informed deep learning
framework capable of encoding an important class of physical laws governing
fluid motions, namely the Navier-Stokes equations. In particular, we seek to
leverage the underlying conservation laws (i.e., for mass, momentum, and
energy) to infer hidden quantities of interest such as velocity and pressure
fields merely from spatio-temporal visualizations of a passive scaler (e.g.,
dye or smoke), transported in arbitrarily complex domains (e.g., in human
arteries or brain aneurysms). Our approach towards solving the aforementioned
data assimilation problem is unique as we design an algorithm that is agnostic
to the geometry or the initial and boundary conditions. This makes HFM highly
flexible in choosing the spatio-temporal domain of interest for data
acquisition as well as subsequent training and predictions. Consequently, the
predictions made by HFM are among those cases where a pure machine learning
strategy or a mere scientific computing approach simply cannot reproduce. The
proposed algorithm achieves accurate predictions of the pressure and velocity
fields in both two and three dimensional flows for several benchmark problems
motivated by real-world applications. Our results demonstrate that this
relatively simple methodology can be used in physical and biomedical problems
to extract valuable quantitative information (e.g., lift and drag forces or
wall shear stresses in arteries) for which direct measurements may not be
possible
ODEVAE: Deep generative second order ODEs with Bayesian neural networks
We present Ordinary Differential Equation Variational Auto-Encoder
(ODEVAE), a latent second order ODE model for high-dimensional sequential
data. Leveraging the advances in deep generative models, ODEVAE can
simultaneously learn the embedding of high dimensional trajectories and infer
arbitrarily complex continuous-time latent dynamics. Our model explicitly
decomposes the latent space into momentum and position components and solves a
second order ODE system, which is in contrast to recurrent neural network (RNN)
based time series models and recently proposed black-box ODE techniques. In
order to account for uncertainty, we propose probabilistic latent ODE dynamics
parameterized by deep Bayesian neural networks. We demonstrate our approach on
motion capture, image rotation and bouncing balls datasets. We achieve
state-of-the-art performance in long term motion prediction and imputation
tasks
Deep Learning of Vortex Induced Vibrations
Vortex induced vibrations of bluff bodies occur when the vortex shedding
frequency is close to the natural frequency of the structure. Of interest is
the prediction of the lift and drag forces on the structure given some limited
and scattered information on the velocity field. This is an inverse problem
that is not straightforward to solve using standard computational fluid
dynamics (CFD) methods, especially since no information is provided for the
pressure. An even greater challenge is to infer the lift and drag forces given
some dye or smoke visualizations of the flow field. Here we employ deep neural
networks that are extended to encode the incompressible Navier-Stokes equations
coupled with the structure's dynamic motion equation. In the first case, given
scattered data in space-time on the velocity field and the structure's motion,
we use four coupled deep neural networks to infer very accurately the
structural parameters, the entire time-dependent pressure field (with no prior
training data), and reconstruct the velocity vector field and the structure's
dynamic motion. In the second case, given scattered data in space-time on a
concentration field only, we use five coupled deep neural networks to infer
very accurately the vector velocity field and all other quantities of interest
as before. This new paradigm of inference in fluid mechanics for coupled
multi-physics problems enables velocity and pressure quantification from flow
snapshots in small subdomains and can be exploited for flow control
applications and also for system identification.Comment: arXiv admin note: text overlap with arXiv:1808.0432
Data recovery in computational fluid dynamics through deep image priors
One of the challenges encountered by computational simulations at exascale is
the reliability of simulations in the face of hardware and software faults.
These faults, expected to increase with the complexity of the computational
systems, will lead to the loss of simulation data and simulation failure and
are currently addressed through a checkpoint-restart paradigm. Focusing
specifically on computational fluid dynamics simulations, this work proposes a
method that uses a deep convolutional neural network to recover simulation
data. This data recovery method (i) is agnostic to the flow configuration and
geometry, (ii) does not require extensive training data, and (iii) is accurate
for very different physical flows. Results indicate that the use of deep image
priors for data recovery is more accurate than standard recovery techniques,
such as the Gaussian process regression, also known as Kriging. Data recovery
is performed for two canonical fluid flows: laminar flow around a cylinder and
homogeneous isotropic turbulence. For data recovery of the laminar flow around
a cylinder, results indicate similar performance between the proposed method
and Gaussian process regression across a wide range of mask sizes. For
homogeneous isotropic turbulence, data recovery through the deep convolutional
neural network exhibits an error in relevant turbulent quantities approximately
three times smaller than that for the Gaussian process regression,. Forward
simulations using recovered data illustrate that the enstrophy decay is
captured within 10% using the deep convolutional neural network approach.
Although demonstrated specifically for data recovery of fluid flows, this
technique can be used in a wide range of applications, including particle image
velocimetry, visualization, and computational simulations of physical processes
beyond the Navier-Stokes equations
On Wasserstein Reinforcement Learning and the Fokker-Planck equation
Policy gradients methods often achieve better performance when the change in
policy is limited to a small Kullback-Leibler divergence. We derive policy
gradients where the change in policy is limited to a small Wasserstein distance
(or trust region). This is done in the discrete and continuous multi-armed
bandit settings with entropy regularisation. We show that in the small steps
limit with respect to the Wasserstein distance , policy dynamics are
governed by the Fokker-Planck (heat) equation, following the
Jordan-Kinderlehrer-Otto result. This means that policies undergo diffusion and
advection, concentrating near actions with high reward. This helps elucidate
the nature of convergence in the probability matching setup, and provides
justification for empirical practices such as Gaussian policy priors and
additive gradient noise
A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems
We propose a new composite neural network (NN) that can be trained based on
multi-fidelity data. It is comprised of three NNs, with the first NN trained
using the low-fidelity data and coupled to two high-fidelity NNs, one with
activation functions and another one without, in order to discover and exploit
nonlinear and linear correlations, respectively, between the low-fidelity and
the high-fidelity data. We first demonstrate the accuracy of the new
multi-fidelity NN for approximating some standard benchmark functions but also
a 20-dimensional function. Subsequently, we extend the recently developed
physics-informed neural networks (PINNs) to be trained with multi-fidelity data
sets (MPINNs). MPINNs contain four fully-connected neural networks, where the
first one approximates the low-fidelity data, while the second and third
construct the correlation between the low- and high-fidelity data and produce
the multi-fidelity approximation, which is then used in the last NN that
encodes the partial differential equations (PDEs). Specifically, in the two
high-fidelity NNs a relaxation parameter is introduced, which can be optimized
to combine the linear and nonlinear sub-networks. By optimizing this parameter,
the present model is capable of learning both the linear and complex nonlinear
correlations between the low- and high-fidelity data adaptively. By training
the MPINNs, we can:(1) obtain the correlation between the low- and
high-fidelity data, (2) infer the quantities of interest based on a few
scattered data, and (3) identify the unknown parameters in the PDEs. In
particular, we employ the MPINNs to learn the hydraulic conductivity field for
unsaturated flows as well as the reactive models for reactive transport. The
results demonstrate that MPINNs can achieve relatively high accuracy based on a
very small set of high-fidelity data
- …