On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning

We shed light on a pitfall and an opportunity in physics-informed neural networks (PINNs). We prove that a multilayer perceptron (MLP) only with ReLU (Rectified Linear Unit) or ReLU-like Lipschitz activation functions will always lead to a vanished Hessian. Such a network-imposed constraint contradicts any second- or higher-order partial differential equations (PDEs). Therefore, a ReLU-based MLP cannot form a permissible function space for the approximation of their solutions. Inspired by this pitfall, we prove that a linear PDE up to the $n$-th order can be strictly satisfied by an MLP with $C^n$ activation functions when the weights of its output layer lie on a certain hyperplane, as called the out-layer-hyperplane. An MLP equipped with the out-layer-hyperplane becomes "physics-enforced", no longer requiring a loss function for the PDE itself (but only those for the initial and boundary conditions). Such a hyperplane exists not only for MLPs but for any network architecture tailed by a fully-connected hidden layer. To our knowledge, this should be the first PINN architecture that enforces point-wise correctness of PDEs. We show a closed-form expression of the out-layer-hyperplane for second-order linear PDEs, which can be generalised to higher-order nonlinear PDEs.Comment: 12 pages, 3 figure

Zero Coordinate Shift: Whetted Automatic Differentiation for Physics-informed Operator Learning

Automatic differentiation (AD) is a critical step in physics-informed machine learning, required for computing the high-order derivatives of network output w.r.t. coordinates of collocation points. In this paper, we present a novel and lightweight algorithm to conduct AD for physics-informed operator learning, which we call the trick of Zero Coordinate Shift (ZCS). Instead of making all sampled coordinates as leaf variables, ZCS introduces only one scalar-valued leaf variable for each spatial or temporal dimension, simplifying the wanted derivatives from "many-roots-many-leaves" to "one-root-many-leaves" whereby reverse-mode AD becomes directly utilisable. It has led to an outstanding performance leap by avoiding the duplication of the computational graph along the dimension of functions (physical parameters). ZCS is easy to implement with current deep learning libraries; our own implementation is achieved by extending the DeepXDE package. We carry out a comprehensive benchmark analysis and several case studies, training physics-informed DeepONets to solve partial differential equations (PDEs) without data. The results show that ZCS has persistently reduced GPU memory consumption and wall time for training by an order of magnitude, and such reduction factor scales with the number of functions. As a low-level optimisation technique, ZCS imposes no restrictions on data, physics (PDE) or network architecture and does not compromise training results from any aspect.Comment: Published in Journal of Computational Physics. https://doi.org/10.1016/j.jcp.2024.11290

AxiSEM3D: broad-band seismic wavefields in 3-D global earth models with undulating discontinuities

We present a novel numerical method to simulate global seismic wave propagation in realisticaspherical 3-D earth models across the observable frequency band of global seismic data. Ourmethod, named AxiSEM3D, is a hybrid of spectral element method (SEM) and pseudospectralmethod. It describes the azimuthal dimension of global wavefields with a substantially reducednumber of degrees of freedom via a global Fourier series parametrization, of which the numberof terms can be locally adapted to the inherent azimuthal complexity of the wavefields.AxiSEM3D allows for material heterogeneities, such as velocity, density, anisotropy andattenuation, as well as for finite undulations on radial discontinuities, both solid–solid andsolid–fluid, and thereby a variety of aspherical Earth features such as ellipticity, surfacetopography, variable crustal thickness, undulating transition zone and core–mantle boundarytopography. Undulating discontinuities are honoured by means of the ‘particle relabellingtransformation’, so that the spectral element mesh can be kept spherical. The implementation ofthe particle relabelling transformation is verified by benchmark solutions against a discretized3-D SEM, considering ellipticity, topography and bathymetry (with the ocean approximatedas a hydrodynamic load) and a tomographic mantle model with an undulating transition zone.For the state-of-the-art global tomographic models with aspherical geometry but without a 3-Dcrust, efficiency comparisons suggest that AxiSEM3D can be two to three orders of magnitudefaster than a discretized 3-D method for a seismic period at 5 s or below, with the speed-upincreasing with frequency and decreasing with model complexity. We also verify AxiSEM3Dfor localized small-scale heterogeneities with strong perturbation strength. With reasonablecomputing resources, we have achieved a corner frequency of up to 1 Hz for 3-D mantlemodels

Global wave propagation in three-dimensional aspherical Earth models

Earth structure and dynamics are largely inferred by seismology as the main tool for data-informed probing of Earth’s interior. Seismic wave propagation in physically realistic Earth models is one of the most funda- mental topics in seismology, not only essential to the analysis and interpre- tation of observed ground motions in terms of their source characteristics and structure-induced propagation effects, but also indispensable for seis- mic inverse techniques that deliver Earth models by minimising misfits between observations and theoretical predictions. Nevertheless, seismic wave propagation at a global scale still remains as one of the most chal- lenging problems in scientific computing, because of Earth’s multiscale constituents and the observable frequency band of global seismic data for their resolution. In this thesis, I present a new, computationally efficient numerical ap- proach to simulate global seismic wave propagation in realistic three- dimensional (3-D) Earth models. It is a hybrid method of spectral el- ement method (SEM) and pseudo-spectral method, that is, the azimuth dimension of a 3-D wavefield is characterised in terms of a global Fourier series parameterisation, such that the 3-D wave equation reduces to an al- gebraic system of coupled 2-D meridian equations, which is then solved by a 2-D SEM named AxiSEM3D. Computational efficiency of such a hybrid method stems from the inherent azimuthal smoothness of 3-D global wave- fields, resulted from the predominance of long-wavelength heterogeneity in Earth’s mantle and the axial singularity of a point seismic source. AxiSEM3D allows for material heterogeneities such as velocity, density, anisotropy and attenuation, as well as for finite undulations on radial discontinuities, both solid-solid and solid-fluid, and thereby a variety of aspherical Earth features such as ellipticity, topography and bathymetry, variable crustal thickness, and core-mantle boundary topography. Such interface undulations are equivalently interpreted as material perturba- tions of the contiguous media, based on the “particle relabelling transfor- mation”. Ocean is currently modelled as a hydrodynamic load. Benchmarked in reference to a discretised 3-D SEM for a variety of Earth models, including 3-D mantle and crustal structures, topography and bathymetry, and ellipticity, AxiSEM3D proves to be a convergent and ac- curate numerical method. Efficiency comparisons suggest that AxiSEM3D can be 1 to 3 orders of magnitude faster than a conventional 3-D SEM across a period range from 10 s down to 1 s, with its speedup increasing with simulated frequency and decreasing with model complexity. It is sufficiently comprehensive to cover all considered applications at global scale, but is maximally efficient for deep Earth studies with body wave data. The observable frequency range of global seismic data (up to 1 Hz) has been achieved for wavefield modelling upon a 3-D mantle model with moderate computing resources. The MPI-based high-performance C++ code scales up to more than 10,000 cores, available open-source at https://github.com/kuangdai/AxiSEM3D. Three applications are carried out, with different foci from Earth’s surface to the core-mantle boundary. First, surface waves are simulated within a state-of-the-art crustal model; the synthetics are compared to real seismic data to assess two different implementations of a 3-D crust. Second, wave scattering effects and cost impact of a localised small-scale structure with sharp and strong material contrast are investigated, considering a wide range of structure-wavelength combinations. And last, diffracted waves through an ultra-low velocity zone atop the core-mantle boundary are modelled at a 1 Hz frequency

Global wave propagation in three-dimensional aspherical Earth models

Earth structure and dynamics are largely inferred by seismology as the main tool for data-informed probing of Earth’s interior. Seismic wave propagation in physically realistic Earth models is one of the most funda- mental topics in seismology, not only essential to the analysis and interpre- tation of observed ground motions in terms of their source characteristics and structure-induced propagation effects, but also indispensable for seis- mic inverse techniques that deliver Earth models by minimising misfits between observations and theoretical predictions. Nevertheless, seismic wave propagation at a global scale still remains as one of the most chal- lenging problems in scientific computing, because of Earth’s multiscale constituents and the observable frequency band of global seismic data for their resolution. In this thesis, I present a new, computationally efficient numerical ap- proach to simulate global seismic wave propagation in realistic three- dimensional (3-D) Earth models. It is a hybrid method of spectral el- ement method (SEM) and pseudo-spectral method, that is, the azimuth dimension of a 3-D wavefield is characterised in terms of a global Fourier series parameterisation, such that the 3-D wave equation reduces to an al- gebraic system of coupled 2-D meridian equations, which is then solved by a 2-D SEM named AxiSEM3D. Computational efficiency of such a hybrid method stems from the inherent azimuthal smoothness of 3-D global wave- fields, resulted from the predominance of long-wavelength heterogeneity in Earth’s mantle and the axial singularity of a point seismic source. AxiSEM3D allows for material heterogeneities such as velocity, density, anisotropy and attenuation, as well as for finite undulations on radial discontinuities, both solid-solid and solid-fluid, and thereby a variety of aspherical Earth features such as ellipticity, topography and bathymetry, variable crustal thickness, and core-mantle boundary topography. Such interface undulations are equivalently interpreted as material perturba- tions of the contiguous media, based on the “particle relabelling transfor- mation”. Ocean is currently modelled as a hydrodynamic load. Benchmarked in reference to a discretised 3-D SEM for a variety of Earth models, including 3-D mantle and crustal structures, topography and bathymetry, and ellipticity, AxiSEM3D proves to be a convergent and ac- curate numerical method. Efficiency comparisons suggest that AxiSEM3D can be 1 to 3 orders of magnitude faster than a conventional 3-D SEM across a period range from 10 s down to 1 s, with its speedup increasing with simulated frequency and decreasing with model complexity. It is sufficiently comprehensive to cover all considered applications at global scale, but is maximally efficient for deep Earth studies with body wave data. The observable frequency range of global seismic data (up to 1 Hz) has been achieved for wavefield modelling upon a 3-D mantle model with moderate computing resources. The MPI-based high-performance C++ code scales up to more than 10,000 cores, available open-source at https://github.com/kuangdai/AxiSEM3D. Three applications are carried out, with different foci from Earth’s surface to the core-mantle boundary. First, surface waves are simulated within a state-of-the-art crustal model; the synthetics are compared to real seismic data to assess two different implementations of a 3-D crust. Second, wave scattering effects and cost impact of a localised small-scale structure with sharp and strong material contrast are investigated, considering a wide range of structure-wavelength combinations. And last, diffracted waves through an ultra-low velocity zone atop the core-mantle boundary are modelled at a 1 Hz frequency.</p

Kilometer-scale structure on the core–mantle boundary near Hawaii

The lowermost mantle right above the core-mantle boundary is highly heterogeneous containing multiple poorly understood seismic features. The smallest but most extreme heterogeneities yet observed are ‘Ultra-Low Velocity Zones’ (ULVZ). We exploit seismic shear waves that diffract along the core-mantle boundary to provide new insight into these enigmatic structures. We measure a rare core-diffracted signal refracted by a ULVZ at the base of the Hawaiian mantle plume at unprecedentedly high frequencies. This signal shows remarkably longer time delays at higher compared to lower frequencies, indicating a pronounced internal variability inside the ULVZ. Utilizing the latest computational advances in 3D waveform modeling, here we show that we are able to model this high-frequency signal and constrain high-resolution ULVZ structure on the scale of kilometers, for the first time. This new observation suggests a chemically distinct ULVZ with increasing iron content towards the core-mantle boundary, which has implications for Earth’s early evolutionary history and core-mantle interaction

Kilometer-scale structure on the core-mantle boundary at the source of the Hawaiian mantle plume

Abstract The lowermost mantle right above the core-mantle boundary shows a complex and heterogeneous landscape containing multiple poorly understood seismic features visible across a wide range of length scales. The smallest, but most extreme, heterogeneities yet observed are 'Ultra-Low Velocity Zones' (ULVZ), several of which have recently been linked to the base of mantle plumes. We exploit seismic shear waves that diffract along the core-mantle boundary to provide new insight into these enigmatic structures. We demonstrate that these waves have a strong frequency-dependent sensitivity to structure at different length scales above the core-mantle boundary, similar to the dispersive characteristics of surface waves. We measure a rare core-diffracted signal refracted by a ULVZ at the base of the Hawaiian mantle plume at unprecedented high frequencies. This signal shows remarkably longer time delays compared to lower frequencies, indicating extreme internal variability within the Hawaiian ULVZ. Utilizing the latest computational advances in 3D synthetic waveform modeling, we are able to model this high frequency signal and constrain high-resolution structure on the scale of kilometers at the core-mantle boundary, for the first time. Results reveal that the lowermost part of the Hawaiian ULVZ is extremely reduced in shear wave velocity, by up to -40%. This new observation suggests a chemically distinct ULVZ with increasing iron content towards the core-mantle boundary, which has implications for Earth’s early evolutionary history and core-mantle interaction.</jats:p

New Candidate Ultralow-Velocity Zone Locations from Highly Anomalous SPdKS Waveforms

Ultralow-velocity zones (ULVZs) at the core&ndash;mantle boundary (CMB) represent some of the most preternatural features in Earth&rsquo;s mantle. These zones most likely contain partial melt, extremely high iron content ferropericlase, or combinations of both. We analyzed a new collection of 58,155 carefully processed and quality-controlled broadband recordings of the seismic phase SPdKS in the epicentral distance range from 106&deg; to 115&deg;. These data sample 56.9% of the CMB by surface area. From these recordings we searched for the most anomalous seismic waveforms that are indicative of ULVZ presence. We used a Bayesian approach to identify the regions of the CMB that have the highest probability of containing ULVZs, thereby identifying sixteen regions of interest. Of these regions, we corroborate well-known ULVZ existence beneath the South China Sea, southwest Pacific, the Samoa hotspot, the southwestern US/northern Mexico, and Iceland. We find good evidence for new ULVZs beneath North Africa, East Asia, and north of Papua New Guinea. We provide further evidence for ULVZs in regions where some evidence has been hinted at before beneath the Philippine Sea, the Pacific Northwest, and the Amazon Basin. Additional evidence is shown for potential ULVZs at the base of the Caroline, San Felix and Galapagos hotspots

AxiSEM3D: broadband seismic wavefields in 3-D global Earth models with undulating discontinuities

ISSN:0956-540XISSN:1365-246

Simulations of Seismic Wave Propagation on Mars

International audienceWe present global and regional synthetic seismograms computed for 1D and 3D Mars models based on the spectral-element method. For global simulations, we implemented a radially-symmetric Mars model with a 110 km thick crust (Sohl and Spohn in J. Geophys. Res., Planets 102(E1):1613–1635, 1997). For this 1D model, we successfully benchmarked the 3D seismic wave propagation solver SPECFEM3D_GLOBE (Komatitsch and Tromp in Geophys. J. Int. 149(2):390–412, 2002a; 150(1):303–318, 2002b) against the 2D axisymmetric wave propagation solver AxiSEM (Nissen-Meyer et al. in Solid Earth 5(1):425–445, 2014) at periods down to 10 s. We also present higher-resolution body-wave simulations with AxiSEM down to 1 s in a model with a more complex 1D crust, revealing wave propagation effects that would have been difficult to interpret based on ray theory. For 3D global simulations based on SPECFEM3D_GLOBE, we superimposed 3D crustal thickness variations capturing the distinct crustal dichotomy between Mars’ northern and southern hemispheres, as well as topography, ellipticity, gravity, and rotation. The global simulations clearly indicate that the 3D crust speeds up body waves compared to the reference 1D model, whereas it significantly changes surface waveforms and their dispersive character depending on its thickness. We also perform regional simulations with the solver SES3D (Fichtner et al. Geophys. J. Int. 179:1703–1725, 2009) based on 3D crustal models derived from surface composition, thereby addressing the effects of various distinct crustal features down to 2 s. The regional simulations confirm the strong effects of crustal variations on waveforms. We conclude that the numerical tools are ready for examining more scenarios, including various other seismic models and sources