18 research outputs found

    Randomized Forward Mode of Automatic Differentiation for Optimization Algorithms

    Full text link
    Backpropagation within neural networks leverages a fundamental element of automatic differentiation, which is referred to as the reverse mode differentiation, or vector Jacobian Product (VJP) or, in the context of differential geometry, known as the pull-back process. The computation of gradient is important as update of neural network parameters is performed using gradient descent method. In this study, we present a genric randomized method, which updates the parameters of neural networks by using directional derivatives of loss functions computed efficiently by using forward mode AD or Jacobian vector Product (JVP). These JVP are computed along the random directions sampled from different probability distributions e.g., Bernoulli, Normal, Wigner, Laplace and Uniform distributions. The computation of gradient is performed during the forward pass of the neural network. We also present a rigorous analysis of the presented methods providing the rate of convergence along with the computational experiments deployed in scientific Machine learning in particular physics-informed neural networks and Deep Operator Networks.Comment: 23 Pages, 8 Figure

    Seismic wave propagation, attenuation and scattering in porous media across various scales

    Get PDF
    The theory of poroelasticity describes the mechanics of fluid saturated deformable porous solids. Poroelastic theory accurately model the seismic wave propagation in oil and gas reservoirs with an extension to other porous medium e.g. living bones and soft tissues. The poroelastic theory also incorporates the attenuation of energy caused by the relative motion between solid and fluid excited by a point source perturbation.First and foremost, this thesis presents a comprehensive description of seismic attenuation in a viscoacoustic medium by using a fractional derivative approach. Representation of attenuation by using a fractional derivative avoids the solution of augmented system represented by memory process i.e. convolution. In this thesis, a highly accurate time integration scheme, known as rapid expansion method, with a spectral accuracy is implemented to solve the viscoacoustic wave equation.Next, this thesis describes the implementation of a high-order weight-adjusted discontinuous Galerkin (WADG) scheme for the numerical solution of two and three-dimensional (3D) wave propagation problems in anisotropic porous media. The use of a penalty-based numerical flux avoids the diagonalization of Jacobian matrices into polarized wave constituents necessary when solving element-wise Riemann problems.Additionally, a system of hyperbolic partial differential equations describing Biot's poroelastic wave equation for quasi-static Poisseulle and potential flow is also introduced. To incorporate effects from micro-heterogeneities due to pores, we have used the Johnson-Koplik-Dashen (JKD) model of dynamic permeability, which also account for frequency-dependent viscous dissipation caused by wave-induced pore fluids.Next, this thesis uses a model to quantify capillary effects on velocity and attenuation. Studies that have attempted to extend Biot's poroelasticity to include capillary effects found changes in fast P-wave velocity of up to 5 % between the sonic and ultrasonic frequency ranges

    High-Order Methods for Hypersonic Flows with Strong Shocks and Real Chemistry

    Full text link
    We compare high-order methods including spectral difference (SD), flux reconstruction (FR), the entropy-stable discontinuous Galerkin spectral element method (ES-DGSEM), modal discontinuous Galerkin methods, and WENO to select the best candidate to simulate strong shock waves characteristic of hypersonic flows. We consider several benchmarks, including the Leblanc and modified shock-density wave interaction problems that require robust stabilization and positivity-preserving properties for a successful flow realization. We also perform simulations of the three-species Sod problem with simplified chemistry with the chemical reaction source terms introduced in the Euler equations. The ES-DGSEM scheme exhibits the highest stability, negligible numerical oscillations, and requires the least computational effort in resolving reactive flow regimes with strong shock waves. Therefore, we extend the ES-DGSEM to hypersonic Euler equations by deriving a new set of two-point entropy conservative fluxes for a five-species gas model. Stabilization for capturing strong shock waves occurs by blending high-order entropy conservative fluxes with low-order finite volume fluxes constructed using the HLLC Riemann solver. The hypersonic Euler solver is verified using the non-equilibrium chemistry Sod problem. To this end, we adopt the Mutation++ library to compute the reaction source terms, thermodynamic properties, and transport coefficients. We also investigate the effect of real chemistry versus ideal chemistry, and the results demonstrate that the ideal chemistry assumption fails at high temperatures, hence real chemistry must be employed for accurate predictions. Finally, we consider a viscous hypersonic flow problem to verify the transport coefficients and reaction source terms determined by the Mutation++ library

    A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data

    Full text link
    We propose a framework and an algorithm to uncover the unknown parts of nonlinear equations directly from data. The framework is based on eXtended Physics-Informed Neural Networks (X-PINNs), domain decomposition in space-time, but we augment the original X-PINN method by imposing flux continuity across the domain interfaces. The well-known Allen-Cahn equation is used to demonstrate the approach. The Frobenius matrix norm is used to evaluate the accuracy of the X-PINN predictions and the results show excellent performance. In addition, symbolic regression is employed to determine the closed form of the unknown part of the equation from the data, and the results confirm the accuracy of the X-PINNs based approach. To test the framework in a situation resembling real-world data, random noise is added to the datasets to mimic scenarios such as the presence of thermal noise or instrument errors. The results show that the framework is stable against significant amount of noise. As the final part, we determine the minimal amount of data required for training the neural network. The framework is able to predict the correct form and coefficients of the underlying dynamical equation when at least 50\% data is used for training

    Tackling the Curse of Dimensionality with Physics-Informed Neural Networks

    Full text link
    The curse-of-dimensionality (CoD) taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional PDEs as Richard Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerically partial differential equations (PDEs) in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. In this paper, we develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs into pieces corresponding to different dimensions and samples randomly a subset of these dimensional pieces in each iteration of training PINNs. We theoretically prove the convergence guarantee and other desired properties of the proposed method. We experimentally demonstrate that the proposed method allows us to solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman and the Schr\"{o}dinger equations in thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. For example, we solve nontrivial nonlinear PDEs (the HJB-Lin equation and the BSB equation) in 100,000 dimensions in 6 hours on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, SDGD can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.Comment: 32 pages, 5 figure

    High order entropy stable schemes for the quasi-one-dimensional shallow water and compressible Euler equations

    Full text link
    High order schemes are known to be unstable in the presence of shock discontinuities or under-resolved solution features for nonlinear conservation laws. Entropy stable schemes address this instability by ensuring that physically relevant solutions satisfy a semi-discrete entropy inequality independently of discretization parameters. This work extends high order entropy stable schemes to the quasi-1D shallow water equations and the quasi-1D compressible Euler equations, which model one-dimensional flows through channels or nozzles with varying width. We introduce new non-symmetric entropy conservative finite volume fluxes for both sets of quasi-1D equations, as well as a generalization of the entropy conservation condition to non-symmetric fluxes. When combined with an entropy stable interface flux, the resulting schemes are high order accurate, conservative, and semi-discretely entropy stable. For the quasi-1D shallow water equations, the resulting schemes are also well-balanced

    CrunchGPT: A chatGPT assisted framework for scientific machine learning

    Full text link
    Scientific Machine Learning (SciML) has advanced recently across many different areas in computational science and engineering. The objective is to integrate data and physics seamlessly without the need of employing elaborate and computationally taxing data assimilation schemes. However, preprocessing, problem formulation, code generation, postprocessing and analysis are still time consuming and may prevent SciML from wide applicability in industrial applications and in digital twin frameworks. Here, we integrate the various stages of SciML under the umbrella of ChatGPT, to formulate CrunchGPT, which plays the role of a conductor orchestrating the entire workflow of SciML based on simple prompts by the user. Specifically, we present two examples that demonstrate the potential use of CrunchGPT in optimizing airfoils in aerodynamics, and in obtaining flow fields in various geometries in interactive mode, with emphasis on the validation stage. To demonstrate the flow of the CrunchGPT, and create an infrastructure that can facilitate a broader vision, we built a webapp based guided user interface, that includes options for a comprehensive summary report. The overall objective is to extend CrunchGPT to handle diverse problems in computational mechanics, design, optimization and controls, and general scientific computing tasks involved in SciML, hence using it as a research assistant tool but also as an educational tool. While here the examples focus in fluid mechanics, future versions will target solid mechanics and materials science, geophysics, systems biology and bioinformatics.Comment: 20 pages, 26 figure
    corecore