The solution to partial differential equations using deep learning approaches
has shown promising results for several classes of initial and boundary-value
problems. However, their ability to surpass, particularly in terms of accuracy,
classical discretization methods such as the finite element methods, remains a
significant challenge. Deep learning methods usually struggle to reliably
decrease the error in their approximate solution. A new methodology to better
control the error for deep learning methods is presented here. The main idea
consists in computing an initial approximation to the problem using a simple
neural network and in estimating, in an iterative manner, a correction by
solving the problem for the residual error with a new network of increasing
complexity. This sequential reduction of the residual of the partial
differential equation allows one to decrease the solution error, which, in some
cases, can be reduced to machine precision. The underlying explanation is that
the method is able to capture at each level smaller scales of the solution
using a new network. Numerical examples in 1D and 2D are presented to
demonstrate the effectiveness of the proposed approach. This approach applies
not only to physics informed neural networks but to other neural network
solvers based on weak or strong formulations of the residual.Comment: 34 pages, 20 figure