The automatic solution of partial differential equations using a global spectral method

A spectral method for solving linear partial differential equations (PDEs) with variable coefficients and general boundary conditions defined on rectangular domains is described, based on separable representations of partial differential operators and the one-dimensional ultraspherical spectral method. If a partial differential operator is of splitting rank $2$, such as the operator associated with Poisson or Helmholtz, the corresponding PDE is solved via a generalized Sylvester matrix equation, and a bivariate polynomial approximation of the solution of degree $(n_x,n_y)$ is computed in $\mathcal{O}((n_x n_y)^{3/2})$ operations. Partial differential operators of splitting rank $\geq 3$ are solved via a linear system involving a block-banded matrix in $\mathcal{O}(\min(n_x^{3} n_y,n_x n_y^{3}))$ operations. Numerical examples demonstrate the applicability of our 2D spectral method to a broad class of PDEs, which includes elliptic and dispersive time-evolution equations. The resulting PDE solver is written in MATLAB and is publicly available as part of CHEBFUN. It can resolve solutions requiring over a million degrees of freedom in under $60$ seconds. An experimental implementation in the Julia language can currently perform the same solve in $10$ seconds.Comment: 22 page

Adaptive Meshfree Methods for Partial Differential Equations

There are many types of adaptive methods that have been developed with different algorithm schemes and definitions for solving Partial Differential Equations (PDE). Adaptive methods have been developed in mesh-based methods, and in recent years, they have been extended by using meshfree methods, such as the Radial Basis Function (RBF) collocation method and the Method of Fundamental Solutions (MFS). The purpose of this dissertation is to introduce an adaptive algorithm with a residual type of error estimator which has not been found in the literature for the adaptive MFS. Some modifications have been made in developing the algorithm schemes depending on the governing equations, the domains, and the boundary conditions. The MFS is used as the main meshfree method to solve the Laplace equation in this dissertation, and we propose adaptive algorithms in different versions based on the residual type of an error estimator in 2D and 3D domains. Popular techniques for handling parameters and different approaches are considered in each example to obtain satisfactory results. Dirichlet boundary conditions are carefully chosen to validate the efficiency of the adaptive method. The RBF collocation method and the Method of Approximate Particular Solutions (MAPS) are used for solving the Poisson equation. Due to the type of the PDE, different strategies for constructing the adaptive method had to be followed, and proper error estimators are considered for this part. This results in having a new point of view when observing the numerical results. Methodologies of meshfree methods that are employed in this dissertation are introduced, and numerical examples are presented with various boundary conditions to show how the adaptive method performs. We can observe the benefit of using the adaptive method and the improved error estimators provide better results in the experiments

Investigating and Mitigating Failure Modes in Physics-informed Neural Networks (PINNs)

This paper explores the difficulties in solving partial differential equations (PDEs) using physics-informed neural networks (PINNs). PINNs use physics as a regularization term in the objective function. However, a drawback of this approach is the requirement for manual hyperparameter tuning, making it impractical in the absence of validation data or prior knowledge of the solution. Our investigations of the loss landscapes and backpropagated gradients in the presence of physics reveal that existing methods produce non-convex loss landscapes that are hard to navigate. Our findings demonstrate that high-order PDEs contaminate backpropagated gradients and hinder convergence. To address these challenges, we introduce a novel method that bypasses the calculation of high-order derivative operators and mitigates the contamination of backpropagated gradients. Consequently, we reduce the dimension of the search space and make learning PDEs with non-smooth solutions feasible. Our method also provides a mechanism to focus on complex regions of the domain. Besides, we present a dual unconstrained formulation based on Lagrange multiplier method to enforce equality constraints on the model's prediction, with adaptive and independent learning rates inspired by adaptive subgradient methods. We apply our approach to solve various linear and non-linear PDEs

General-purpose kernel regularization of boundary integral equations via density interpolation

This paper presents a general high-order kernel regularization technique applicable to all four integral operators of Calder\'on calculus associated with linear elliptic PDEs in two and three spatial dimensions. Like previous density interpolation methods, the proposed technique relies on interpolating the density function around the kernel singularity in terms of solutions of the underlying homogeneous PDE, so as to recast singular and nearly singular integrals in terms of bounded (or more regular) integrands. We present here a simple interpolation strategy which, unlike previous approaches, does not entail explicit computation of high-order derivatives of the density function along the surface. Furthermore, the proposed approach is kernel- and dimension-independent in the sense that the sought density interpolant is constructed as a linear combination of point-source fields, given by the same Green's function used in the integral equation formulation, thus making the procedure applicable, in principle, to any PDE with known Green's function. For the sake of definiteness, we focus here on Nystr\"om methods for the (scalar) Laplace and Helmholtz equations and the (vector) elastostatic and time-harmonic elastodynamic equations. The method's accuracy, flexibility, efficiency, and compatibility with fast solvers are demonstrated by means of a variety of large-scale three-dimensional numerical examples

Direct solution of Navier-Stokes equations by using an upwind local RBF-DQ method

The differential quadrature (DQ) method is able to obtain quite accurate numerical solutions of differential equations with few grid points and less computational effort. However, the traditional DQ method is convenient only for regular regions and lacks upwind mechanism to characterize the convection of the fluid flow. In this paper, an upwind local radial basis function-based DQ (RBF-DQ) method is applied to solve the Navier-Stokes equations, instead of using an iterative algorithm for the primitive variables. The non-linear collocated equations are solved using the Levenberg-Marquardt method. The irregular regions of 2D channel flow with different obstructions situations are considered. Finally, the approach is validated by comparing the results with those obtained using the well-validated Fluent commercial package

Local Radial Basis Function Methods for Solving Partial Differential Equations

Meshless methods are relatively new numerical methods which have gained popularity in computational and engineering sciences during the last two decades. This dissertation develops two new localized meshless methods for solving a variety partial differential equations. Recently, some localized meshless methods have been introduced in order to handle large-scale problems, or to avoid ill-conditioned problems involving global radial basis function approximations. This dissertation explains two new localized meshelss methods, each derived from the global Method of Approximate Particular Solutions (MAPS). One method, the Localized Method of Approximate Particular Solutions (LMAPS), is used for elliptic and parabolic partial differential equations (PDEs) using a global sparse linear system of equations. The second method, the Explicit Localized Method of Approximate Particular Solutions (ELMAPS), is constructed for solving parabolic types of partial differential equations by inverting a finite number of small linear systems. For both methods, the only information that is needed in constructing the approximating solution to PDEs, consists of the local nodes that fall within the domain of influence of the data. Since the methods are completely mesh free, they can be used for irregularly shaped domains. Both methods are tested and compared with existing global and local meshless methods. The results illustrate the accuracy and efficiency of our proposed methods

Adaptive Method of Approximate Particular Solution for One-Dimensional Differential Equations

An adaptive algorithm for the Method Approximate Particular Solution (MAPS) using radial basis functions for solving boundary value problems is discussed in this work. The goal of the adaptive algorithm is to construct an optimal collocation points distribution that gives the required accuracy with the smallest number of degrees of freedom. I proposed the formulation of the adaptive MAPS for second order boundary value problems in an arbitrary dimensional setting. Then I applied this method to three different boundary value problems in one-dimensional setting. The performance of the adaptive method has been demonstrated by numerical experiments

SlabLU: A Two-Level Sparse Direct Solver for Elliptic PDEs

The paper describes a sparse direct solver for the linear systems that arise from the discretization of an elliptic PDE on a two dimensional domain. The solver is designed to reduce communication costs and perform well on GPUs; it uses a two-level framework, which is easier to implement and optimize than traditional multi-frontal schemes based on hierarchical nested dissection orderings. The scheme decomposes the domain into thin subdomains, or "slabs". Within each slab, a local factorization is executed that exploits the geometry of the local domain. A global factorization is then obtained through the LU factorization of a block-tridiagonal reduced coefficient matrix. The solver has complexity $O(N^{5/3})$ for the factorization step, and $O(N^{7/6})$ for each solve once the factorization is completed. The solver described is compatible with a range of different local discretizations, and numerical experiments demonstrate its performance for regular discretizations of rectangular and curved geometries. The technique becomes particularly efficient when combined with very high-order convergent multi-domain spectral collocation schemes. With this discretization, a Helmholtz problem on a domain of size $1000 \lambda \times 1000 \lambda$ (for which N=100 \mbox{M}) is solved in 15 minutes to 6 correct digits on a high-powered desktop with GPU acceleration