722 research outputs found
An Expert's Guide to Training Physics-informed Neural Networks
Physics-informed neural networks (PINNs) have been popularized as a deep
learning framework that can seamlessly synthesize observational data and
partial differential equation (PDE) constraints. Their practical effectiveness
however can be hampered by training pathologies, but also oftentimes by poor
choices made by users who lack deep learning expertise. In this paper we
present a series of best practices that can significantly improve the training
efficiency and overall accuracy of PINNs. We also put forth a series of
challenging benchmark problems that highlight some of the most prominent
difficulties in training PINNs, and present comprehensive and fully
reproducible ablation studies that demonstrate how different architecture
choices and training strategies affect the test accuracy of the resulting
models. We show that the methods and guiding principles put forth in this study
lead to state-of-the-art results and provide strong baselines that future
studies should use for comparison purposes. To this end, we also release a
highly optimized library in JAX that can be used to reproduce all results
reported in this paper, enable future research studies, as well as facilitate
easy adaptation to new use-case scenarios.Comment: 36 pages, 25 figures, 13 table
Physics-informed neural networks for modeling rate- and temperature-dependent plasticity
This work presents a physics-informed neural network (PINN) based framework
to model the strain-rate and temperature dependence of the deformation fields
in elastic-viscoplastic solids. To avoid unbalanced back-propagated gradients
during training, the proposed framework uses a simple strategy with no added
computational complexity for selecting scalar weights that balance the
interplay between different terms in the physics-based loss function. In
addition, we highlight a fundamental challenge involving the selection of
appropriate model outputs so that the mechanical problem can be faithfully
solved using a PINN-based approach. We demonstrate the effectiveness of this
approach by studying two test problems modeling the elastic-viscoplastic
deformation in solids at different strain rates and temperatures, respectively.
Our results show that the proposed PINN-based approach can accurately predict
the spatio-temporal evolution of deformation in elastic-viscoplastic materials.Comment: 11 pages, 7 figures; Accepted in NeurIPS 2022, Machine Learning and
the Physical Sciences worksho
Modeling Power Systems Dynamics with Symbolic Physics-Informed Neural Networks
In recent years, scientific machine learning, particularly physic-informed
neural networks (PINNs), has introduced new innovative methods to understanding
the differential equations that describe power system dynamics, providing a
more efficient alternative to traditional methods. However, using a single
neural network to capture patterns of all variables requires a large enough
size of networks, leading to a long time of training and still high
computational costs. In this paper, we utilize the interfacing of PINNs with
symbolic techniques to construct multiple single-output neural networks by
taking the loss function apart and integrating it over the relevant domain.
Also, we reweigh the factors of the components in the loss function to improve
the performance of the network for instability systems. Our results show that
the symbolic PINNs provide higher accuracy with significantly fewer parameters
and faster training time. By using the adaptive weight method, the symbolic
PINNs can avoid the vanishing gradient problem and numerical instability
Physics-Informed Neural Networks for 2nd order ODEs with sharp gradients
In this work, four different methods based on Physics-Informed Neural Networks (PINNs) for solving Differential Equations (DE) are compared: Classic-PINN that makes use of Deep Neural Networks (DNNs) to approximate the DE solution;Deep-TFC improves the efficiency of classic-PINN by employing the constrained expression from the Theory of Functional Connections (TFC) so to analytically satisfy the DE constraints;PIELM that improves the accuracy of classic-PINN by employing a single-layer NN trained via Extreme Learning Machine (ELM) algorithm;X-TFC, which makes use of both constrained expression and ELM. The last has been recently introduced to solve challenging problems affected by discontinuity, learning solutions in cases where the other three methods fail. The four methods are compared by solving the boundary value problem arising from the 1D Steady-State Advection–Diffusion Equation for different values of the diffusion coefficient. The solutions of the DEs exhibit steep gradients as the value of the diffusion coefficient decreases, increasing the challenge of the problem
Neural Eikonal Solver: improving accuracy of physics-informed neural networks for solving eikonal equation in case of caustics
The concept of physics-informed neural networks has become a useful tool for
solving differential equations due to its flexibility. There are a few
approaches using this concept to solve the eikonal equation which describes the
first-arrival traveltimes of acoustic and elastic waves in smooth heterogeneous
velocity models. However, the challenge of the eikonal is exacerbated by the
velocity models producing caustics, resulting in instabilities and
deterioration of accuracy due to the non-smooth solution behaviour. In this
paper, we revisit the problem of solving the eikonal equation using neural
networks to tackle the caustic pathologies. We introduce the novel Neural
Eikonal Solver (NES) for solving the isotropic eikonal equation in two
formulations: the one-point problem is for a fixed source location; the
two-point problem is for an arbitrary source-receiver pair. We present several
techniques which provide stability in velocity models producing caustics:
improved factorization; non-symmetric loss function based on Hamiltonian;
gaussian activation; symmetrization. In our tests, NES showed the
relative-mean-absolute error of about 0.2-0.4% from the second-order factored
Fast Marching Method, and outperformed existing neural-network solvers giving
10-60 times lower errors and 2-30 times faster training. The inference time of
NES is comparable with the Fast Marching. The one-point NES provides the most
accurate solution, whereas the two-point NES provides slightly lower accuracy
but gives an extremely compact representation. It can be useful in various
seismic applications where massive computations are required (millions of
source-receiver pairs): ray modeling, traveltime tomography, hypocenter
localization, and Kirchhoff migration.Comment: The paper has 14 pages and 6 figures. Source code is available at
https://github.com/sgrubas/NE
About optimal loss function for training physics-informed neural networks under respecting causality
A method is presented that allows to reduce a problem described by
differential equations with initial and boundary conditions to the problem
described only by differential equations. The advantage of using the modified
problem for physics-informed neural networks (PINNs) methodology is that it
becomes possible to represent the loss function in the form of a single term
associated with differential equations, thus eliminating the need to tune the
scaling coefficients for the terms related to boundary and initial conditions.
The weighted loss functions respecting causality were modified and new weighted
loss functions based on generalized functions are derived. Numerical
experiments have been carried out for a number of problems, demonstrating the
accuracy of the proposed methods.Comment: 25 pages, 7 figures, 6 table
- …