47 research outputs found
Efficient Numerical Method for Models Driven by L\'evy Process via Hierarchical Matrices
Modeling via fractional partial differential equations or a L\'evy process
has been an active area of research and has many applications. However, the
lack of efficient numerical computation methods for general nonlocal operators
impedes people from adopting such modeling tools. We proposed an efficient
solver for the convection-diffusion equation whose operator is the
infinitesimal generator of a L\'evy process based on -matrix
technique. The proposed Crank Nicolson scheme is unconditionally stable and has
a theoretical convergence rate. The
-matrix technique has theoretical space and
computational complexity compared to and
respectively for the direct method. Numerical experiments demonstrate the
efficiency of the new algorithm.Comment: 40 pages, 16 figure
Calibrating Multivariate L\'evy Processes with Neural Networks
Calibrating a L\'evy process usually requires characterizing its jump
distribution. Traditionally this problem can be solved with nonparametric
estimation using the empirical characteristic functions (ECF), assuming certain
regularity, and results to date are mostly in 1D. For multivariate L\'evy
processes and less smooth L\'evy densities, the problem becomes challenging as
ECFs decay slowly and have large uncertainty because of limited observations.
We solve this problem by approximating the L\'evy density with a parametrized
functional form; the characteristic function is then estimated using numerical
integration. In our benchmarks, we used deep neural networks and found that
they are robust and can capture sharp transitions in the L\'evy density. They
perform favorably compared to piecewise linear functions and radial basis
functions. The methods and techniques developed here apply to many other
problems that involve nonparametric estimation of functions embedded in a
system model.Comment: 10 pages, 7 figure
The Neural Network Approach to Inverse Problems in Differential Equations
We proposed a framework for solving inverse problems in differential
equations based on neural networks and automatic differentiation. Neural
networks are used to approximate hidden fields. We analyze the source of errors
in the framework and derive an error estimate for a model diffusion equation
problem. Besides, we propose a way for sensitivity analysis, utilizing the
automatic differentiation mechanism embedded in the framework. It frees people
from the tedious and error-prone process of deriving the gradients. Numerical
examples exhibit consistency with the convergence analysis and error saturation
is noteworthily predicted. We also demonstrate the unique benefits neural
networks offer at the same time: universal approximation ability, regularizing
the solution, bypassing the curse of dimensionality and leveraging efficient
computing frameworks.Comment: 32 pages, 9 figure
ADCME: Learning Spatially-varying Physical Fields using Deep Neural Networks
ADCME is a novel computational framework to solve inverse problems involving
physical simulations and deep neural networks (DNNs). This paper benchmarks its
capability to learn spatially-varying physical fields using DNNs. We
demonstrate that our approach has superior accuracy compared to the
discretization approach on a variety of problems, linear or nonlinear, static
or dynamic. Technically, we formulate our inverse problem as a PDE-constrained
optimization problem. We express both the numerical simulations and DNNs using
computational graphs and therefore, we can calculate the gradients using
reverse-mode automatic differentiation. We apply a physics constrained learning
algorithm (PCL) to efficiently back-propagate gradients through iterative
solvers for nonlinear equations. The open-source software which accompanies the
present paper can be found at https://github.com/kailaix/ADCME.jl
Learning Constitutive Relations from Indirect Observations Using Deep Neural Networks
We present a new approach for predictive modeling and its uncertainty
quantification for mechanical systems, where coarse-grained models such as
constitutive relations are derived directly from observation data. We explore
the use of a neural network to represent the unknown constitutive relations,
compare the neural networks with piecewise linear functions, radial basis
functions, and radial basis function networks, and show that the neural network
outperforms the others in certain cases. We analyze the approximation error of
the neural networks using a scaling argument. The training and predicting
processes in our framework combine the finite element method, automatic
differentiation, and neural networks (or other function approximators). Our
framework also allows uncertainty quantification in the form of confidence
intervals. Numerical examples on a multiscale fiber-reinforced plate problem
and a nonlinear rubbery membrane problem from solid mechanics demonstrate the
effectiveness of our framework.Comment: 40 pages, 21 figure
Spectral Method for the Fractional Laplacian in 2D and 3D
A spectral method is considered for approximating the fractional Laplacian
and solving the fractional Poisson problem in 2D and 3D unit balls. The method
is based on the explicit formulation of the eigenfunctions and eigenvalues of
the fractional Laplacian in the unit balls under the weighted space. The
resulting method enjoys spectral accuracy for all fractional index and is computationally efficient due to the orthogonality of the basis
functions. We also proposed a numerical integration strategy for computing the
coefficients. Numerical examples in 2D and 3D are shown to demonstrate the
effectiveness of the proposed methods.Comment: 34 pages, 7 figure
Distributed Machine Learning for Computational Engineering using MPI
We propose a framework for training neural networks that are coupled with
partial differential equations (PDEs) in a parallel computing environment.
Unlike most distributed computing frameworks for deep neural networks, our
focus is to parallelize both numerical solvers and deep neural networks in
forward and adjoint computations. Our parallel computing model views data
communication as a node in the computational graph for numerical simulations.
The advantage of our model is that data communication and computing are cleanly
separated and thus provide better flexibility, modularity, and testability. We
demonstrate using various large-scale problems that we can achieve substantial
acceleration by using parallel solvers for PDEs in training deep neural
networks that are coupled with PDEs.Comment: 24 pages, 25 figure
Learning Constitutive Relations using Symmetric Positive Definite Neural Networks
We present the Cholesky-factored symmetric positive definite neural network
(SPD-NN) for modeling constitutive relations in dynamical equations. Instead of
directly predicting the stress, the SPD-NN trains a neural network to predict
the Cholesky factor of a tangent stiffness matrix, based on which the stress is
calculated in the incremental form. As a result of the special structure,
SPD-NN weakly imposes convexity on the strain energy function, satisfies time
consistency for path-dependent materials, and therefore improves numerical
stability, especially when the SPD-NN is used in finite element simulations.
Depending on the types of available data, we propose two training methods,
namely direct training for strain and stress pairs and indirect training for
loads and displacement pairs. We demonstrate the effectiveness of SPD-NN on
hyperelastic, elasto-plastic, and multiscale fiber-reinforced plate problems
from solid mechanics. The generality and robustness of the SPD-NN make it a
promising tool for a wide range of constitutive modeling applications.Comment: 31 pages, 20 figure
Coupled Time-lapse Full Waveform Inversion for Subsurface Flow Problems using Intrusive Automatic Differentiation
We describe a novel framework for estimating subsurface properties, such as
rock permeability and porosity, from time-lapse observed seismic data by
coupling full-waveform inversion, subsurface flow processes, and rock physics
models. For the inverse modeling, we handle the back-propagation of gradients
by an intrusive automatic differentiation strategy that offers three levels of
user control: (1) at the wave physics level, we adopted the discrete adjoint
method in order to use our existing high-performance FWI code; (2) at the rock
physics level, we used built-in operators from the
backend; (3) at the flow physics level, we implemented customized PDE operators
for the potential and nonlinear saturation equations. These three levels of
gradient computation strike a good balance between computational efficiency and
programming efficiency, and when chained together, constitute a coupled inverse
system. We use numerical experiments to demonstrate that (1) the three-level
coupled inverse problem is superior in terms of accuracy to a traditional
decoupled inversion strategy; (2) it is able to simultaneously invert for
parameters in empirical relationships such as the rock physics models; and (3)
the inverted model can be used for reservoir performance prediction and
reservoir management/optimization purposes.Comment: 27 pages, 14 figure
Learning Hidden Dynamics using Intelligent Automatic Differentiation
Many engineering problems involve learning hidden dynamics from indirect
observations, where the physical processes are described by systems of partial
differential equations (PDE). Gradient-based optimization methods are
considered scalable and efficient to learn hidden dynamics. However, one of the
most time-consuming and error-prone tasks is to derive and implement the
gradients, especially in systems of PDEs where gradients from different systems
must be correctly integrated together. To that purpose, we present a novel
technique, called intelligent automatic differentiation (IAD), to leverage the
modern machine learning tool for computing gradients
automatically and conducting optimization efficiently. Moreover, IAD allows us
to integrate specially designed state adjoint method codes to achieve better
performance. Numerical tests demonstrate the feasibility of IAD for learning
hidden dynamics in complicated systems of PDEs; additionally, by incorporating
custom built state adjoint method codes in IAD, we significantly accelerate the
forward and inverse simulation.Comment: 25 pages, 10 figure