Less Emphasis on Difficult Layer Regions: Curriculum Learning for Singularly Perturbed Convection-Diffusion-Reaction Problems

Although Physics-Informed Neural Networks (PINNs) have been successfully applied in a wide variety of science and engineering fields, they can fail to accurately predict the underlying solution in slightly challenging convection-diffusion-reaction problems. In this paper, we investigate the reason of this failure from a domain distribution perspective, and identify that learning multi-scale fields simultaneously makes the network unable to advance its training and easily get stuck in poor local minima. We show that the widespread experience of sampling more collocation points in high-loss layer regions hardly help optimize and may even worsen the results. These findings motivate the development of a novel curriculum learning method that encourages neural networks to prioritize learning on easier non-layer regions while downplaying learning on harder layer regions. The proposed method helps PINNs automatically adjust the learning emphasis and thereby facilitate the optimization procedure. Numerical results on typical benchmark equations show that the proposed curriculum learning approach mitigates the failure modes of PINNs and can produce accurate results for very sharp boundary and interior layers. Our work reveals that for equations whose solutions have large scale differences, paying less attention to high-loss regions can be an effective strategy for learning them accurately.Comment: 22 page

Robust error estimates in weak norms for advection dominated transport problems with rough data

We consider mixing problems in the form of transient convection--diffusion equations with a velocity vector field with multiscale character and rough data. We assume that the velocity field has two scales, a coarse scale with slow spatial variation, which is responsible for advective transport and a fine scale with small amplitude that contributes to the mixing. For this problem we consider the estimation of filtered error quantities for solutions computed using a finite element method with symmetric stabilization. A posteriori error estimates and a priori error estimates are derived using the multiscale decomposition of the advective velocity to improve stability. All estimates are independent both of the P\'eclet number and of the regularity of the exact solution

Stabilization arising from PGEM : a review and further developments

The aim of this paper is twofold. First, we review the recent Petrov-Galerkin enriched method (PGEM) to stabilize numerical solutions of BVP's in primal and mixed forms. Then, we extend such enrichment technique to a mixed singularly perturbed problem, namely, the generalized Stokes problem, and focus on a stabilized finite element method arising in a natural way after performing static condensation. The resulting stabilized method is shown to lead to optimal convergences, and afterward, it is numerically validated

Multiscale stabilization for convection-dominated diffusion in heterogeneous media

We develop a Petrov-Galerkin stabilization method for multiscale convection-diffusion transport systems. Existing stabilization techniques add a limited number of degrees of freedom in the form of bubble functions or a modified diffusion, which may not be sufficient to stabilize multiscale systems. We seek a local reduced-order model for this kind of multiscale transport problems and thus, develop a systematic approach for finding reduced-order approximations of the solution. We start from a Petrov-Galerkin framework using optimal weighting functions. We introduce an auxiliary variable to a mixed formulation of the problem. The auxiliary variable stands for the optimal weighting function. The problem reduces to finding a test space (a dimensionally reduced space for this auxiliary variable), which guarantees that the error in the primal variable (representing the solution) is close to the projection error of the full solution on the dimensionally reduced space that approximates the solution. To find the test space, we reformulate some recent mixed Generalized Multiscale Finite Element Methods. We introduce snapshots and local spectral problems that appropriately define local weight and trial spaces. In particular, we use energy minimizing snapshots and local spectral decompositions in the natural norm associated with the auxiliary variable. The resulting spectral decomposition adaptively identifies and builds the optimal multiscale space to stabilize the system. We discuss the stability and its relation to the approximation property of the test space. We design online basis functions, which accelerate convergence in the test space, and consequently, improve stability. We present several numerical examples and show that one needs a few test functions to achieve an error similar to the projection error in the primal variable irrespective of the Peclet number