18 research outputs found
Randomized Forward Mode of Automatic Differentiation for Optimization Algorithms
Backpropagation within neural networks leverages a fundamental element of
automatic differentiation, which is referred to as the reverse mode
differentiation, or vector Jacobian Product (VJP) or, in the context of
differential geometry, known as the pull-back process. The computation of
gradient is important as update of neural network parameters is performed using
gradient descent method. In this study, we present a genric randomized method,
which updates the parameters of neural networks by using directional
derivatives of loss functions computed efficiently by using forward mode AD or
Jacobian vector Product (JVP). These JVP are computed along the random
directions sampled from different probability distributions e.g., Bernoulli,
Normal, Wigner, Laplace and Uniform distributions. The computation of gradient
is performed during the forward pass of the neural network. We also present a
rigorous analysis of the presented methods providing the rate of convergence
along with the computational experiments deployed in scientific Machine
learning in particular physics-informed neural networks and Deep Operator
Networks.Comment: 23 Pages, 8 Figure
Seismic wave propagation, attenuation and scattering in porous media across various scales
The theory of poroelasticity describes the mechanics of fluid saturated deformable porous solids. Poroelastic theory accurately model the seismic wave propagation in oil and gas reservoirs with an extension to other porous medium e.g. living bones and soft tissues. The poroelastic theory also incorporates the attenuation of energy caused by the relative motion between solid and fluid excited by a point source perturbation.First and foremost, this thesis presents a comprehensive description of seismic attenuation in a viscoacoustic medium by using a fractional derivative approach. Representation of attenuation by using a fractional derivative avoids the solution of augmented system represented by memory process i.e. convolution. In this thesis, a highly accurate time integration scheme, known as rapid expansion method, with a spectral accuracy is implemented to solve the viscoacoustic wave equation.Next, this thesis describes the implementation of a high-order weight-adjusted discontinuous Galerkin (WADG) scheme for the numerical solution of two and three-dimensional (3D) wave propagation problems in anisotropic porous media. The use of a penalty-based numerical flux avoids the diagonalization of Jacobian matrices into polarized wave constituents necessary when solving element-wise Riemann problems.Additionally, a system of hyperbolic partial differential equations describing Biot's poroelastic wave equation for quasi-static Poisseulle and potential flow is also introduced. To incorporate effects from micro-heterogeneities due to pores, we have used the Johnson-Koplik-Dashen (JKD) model of dynamic permeability, which also account for frequency-dependent viscous dissipation caused by wave-induced pore fluids.Next, this thesis uses a model to quantify capillary effects on velocity and attenuation. Studies that have attempted to extend Biot's poroelasticity to include capillary effects found changes in fast P-wave velocity of up to 5 % between the sonic and ultrasonic frequency ranges
High-Order Methods for Hypersonic Flows with Strong Shocks and Real Chemistry
We compare high-order methods including spectral difference (SD), flux
reconstruction (FR), the entropy-stable discontinuous Galerkin spectral element
method (ES-DGSEM), modal discontinuous Galerkin methods, and WENO to select the
best candidate to simulate strong shock waves characteristic of hypersonic
flows. We consider several benchmarks, including the Leblanc and modified
shock-density wave interaction problems that require robust stabilization and
positivity-preserving properties for a successful flow realization. We also
perform simulations of the three-species Sod problem with simplified chemistry
with the chemical reaction source terms introduced in the Euler equations. The
ES-DGSEM scheme exhibits the highest stability, negligible numerical
oscillations, and requires the least computational effort in resolving reactive
flow regimes with strong shock waves. Therefore, we extend the ES-DGSEM to
hypersonic Euler equations by deriving a new set of two-point entropy
conservative fluxes for a five-species gas model. Stabilization for capturing
strong shock waves occurs by blending high-order entropy conservative fluxes
with low-order finite volume fluxes constructed using the HLLC Riemann solver.
The hypersonic Euler solver is verified using the non-equilibrium chemistry Sod
problem. To this end, we adopt the Mutation++ library to compute the reaction
source terms, thermodynamic properties, and transport coefficients. We also
investigate the effect of real chemistry versus ideal chemistry, and the
results demonstrate that the ideal chemistry assumption fails at high
temperatures, hence real chemistry must be employed for accurate predictions.
Finally, we consider a viscous hypersonic flow problem to verify the transport
coefficients and reaction source terms determined by the Mutation++ library
A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data
We propose a framework and an algorithm to uncover the unknown parts of
nonlinear equations directly from data. The framework is based on eXtended
Physics-Informed Neural Networks (X-PINNs), domain decomposition in space-time,
but we augment the original X-PINN method by imposing flux continuity across
the domain interfaces. The well-known Allen-Cahn equation is used to
demonstrate the approach. The Frobenius matrix norm is used to evaluate the
accuracy of the X-PINN predictions and the results show excellent performance.
In addition, symbolic regression is employed to determine the closed form of
the unknown part of the equation from the data, and the results confirm the
accuracy of the X-PINNs based approach. To test the framework in a situation
resembling real-world data, random noise is added to the datasets to mimic
scenarios such as the presence of thermal noise or instrument errors. The
results show that the framework is stable against significant amount of noise.
As the final part, we determine the minimal amount of data required for
training the neural network. The framework is able to predict the correct form
and coefficients of the underlying dynamical equation when at least 50\% data
is used for training
Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
The curse-of-dimensionality (CoD) taxes computational resources heavily with
exponentially increasing computational cost as the dimension increases. This
poses great challenges in solving high-dimensional PDEs as Richard Bellman
first pointed out over 60 years ago. While there has been some recent success
in solving numerically partial differential equations (PDEs) in high
dimensions, such computations are prohibitively expensive, and true scaling of
general nonlinear PDEs to high dimensions has never been achieved. In this
paper, we develop a new method of scaling up physics-informed neural networks
(PINNs) to solve arbitrary high-dimensional PDEs. The new method, called
Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs
into pieces corresponding to different dimensions and samples randomly a subset
of these dimensional pieces in each iteration of training PINNs. We
theoretically prove the convergence guarantee and other desired properties of
the proposed method. We experimentally demonstrate that the proposed method
allows us to solve many notoriously hard high-dimensional PDEs, including the
Hamilton-Jacobi-Bellman and the Schr\"{o}dinger equations in thousands of
dimensions very fast on a single GPU using the PINNs mesh-free approach. For
example, we solve nontrivial nonlinear PDEs (the HJB-Lin equation and the BSB
equation) in 100,000 dimensions in 6 hours on a single GPU using SDGD with
PINNs. Since SDGD is a general training methodology of PINNs, SDGD can be
applied to any current and future variants of PINNs to scale them up for
arbitrary high-dimensional PDEs.Comment: 32 pages, 5 figure
High order entropy stable schemes for the quasi-one-dimensional shallow water and compressible Euler equations
High order schemes are known to be unstable in the presence of shock
discontinuities or under-resolved solution features for nonlinear conservation
laws. Entropy stable schemes address this instability by ensuring that
physically relevant solutions satisfy a semi-discrete entropy inequality
independently of discretization parameters. This work extends high order
entropy stable schemes to the quasi-1D shallow water equations and the quasi-1D
compressible Euler equations, which model one-dimensional flows through
channels or nozzles with varying width.
We introduce new non-symmetric entropy conservative finite volume fluxes for
both sets of quasi-1D equations, as well as a generalization of the entropy
conservation condition to non-symmetric fluxes. When combined with an entropy
stable interface flux, the resulting schemes are high order accurate,
conservative, and semi-discretely entropy stable. For the quasi-1D shallow
water equations, the resulting schemes are also well-balanced
CrunchGPT: A chatGPT assisted framework for scientific machine learning
Scientific Machine Learning (SciML) has advanced recently across many
different areas in computational science and engineering. The objective is to
integrate data and physics seamlessly without the need of employing elaborate
and computationally taxing data assimilation schemes. However, preprocessing,
problem formulation, code generation, postprocessing and analysis are still
time consuming and may prevent SciML from wide applicability in industrial
applications and in digital twin frameworks. Here, we integrate the various
stages of SciML under the umbrella of ChatGPT, to formulate CrunchGPT, which
plays the role of a conductor orchestrating the entire workflow of SciML based
on simple prompts by the user. Specifically, we present two examples that
demonstrate the potential use of CrunchGPT in optimizing airfoils in
aerodynamics, and in obtaining flow fields in various geometries in interactive
mode, with emphasis on the validation stage. To demonstrate the flow of the
CrunchGPT, and create an infrastructure that can facilitate a broader vision,
we built a webapp based guided user interface, that includes options for a
comprehensive summary report. The overall objective is to extend CrunchGPT to
handle diverse problems in computational mechanics, design, optimization and
controls, and general scientific computing tasks involved in SciML, hence using
it as a research assistant tool but also as an educational tool. While here the
examples focus in fluid mechanics, future versions will target solid mechanics
and materials science, geophysics, systems biology and bioinformatics.Comment: 20 pages, 26 figure