4,459 research outputs found
Machine Learning for semi linear PDEs
Recent machine learning algorithms dedicated to solving semi-linear PDEs are
improved by using different neural network architectures and different
parameterizations. These algorithms are compared to a new one that solves a
fixed point problem by using deep learning techniques. This new algorithm
appears to be competitive in terms of accuracy with the best existing
algorithms.Comment: 38 page
A learning scheme by sparse grids and Picard approximations for semilinear parabolic PDEs
Relying on the classical connection between Backward Stochastic Differential
Equations (BSDEs) and non-linear parabolic partial differential equations
(PDEs), we propose a new probabilistic learning scheme for solving
high-dimensional semi-linear parabolic PDEs. This scheme is inspired by the
approach coming from machine learning and developed using deep neural networks
in Han and al. [32]. Our algorithm is based on a Picard iteration scheme in
which a sequence of linear-quadratic optimisation problem is solved by means of
stochastic gradient descent (SGD) algorithm. In the framework of a linear
specification of the approximation space, we manage to prove a convergence
result for our scheme, under some smallness condition. In practice, in order to
be able to treat high-dimensional examples, we employ sparse grid approximation
spaces. In the case of periodic coefficients and using pre-wavelet basis
functions, we obtain an upper bound on the global complexity of our method. It
shows in particular that the curse of dimensionality is tamed in the sense that
in order to achieve a root mean squared error of order , for a
prescribed precision , the complexity of the Picard algorithm grows
polynomially in up to some logarithmic factor which grows linearly with respect to the PDE dimension.
Various numerical results are presented to validate the performance of our
method and to compare them with some recent machine learning schemes proposed
in Han and al. [20] and Hur\'e and al. [37]
MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
We develop a unified model, known as MgNet, that simultaneously recovers some
convolutional neural networks (CNN) for image classification and multigrid (MG)
methods for solving discretized partial differential equations (PDEs). This
model is based on close connections that we have observed and uncovered between
the CNN and MG methodologies. For example, pooling operation and feature
extraction in CNN correspond directly to restriction operation and iterative
smoothers in MG, respectively. As the solution space is often the dual of the
data space in PDEs, the analogous concept of feature space and data space
(which are dual to each other) is introduced in CNN. With such connections and
new concept in the unified model, the function of various convolution
operations and pooling used in CNN can be better understood. As a result,
modified CNN models (with fewer weights and hyper parameters) are developed
that exhibit competitive and sometimes better performance in comparison with
existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.Comment: 30 page
Harmonic Extension
In this paper, we consider the harmonic extension problem, which is widely
used in many applications of machine learning. We find that the transitional
method of graph Laplacian fails to produce a good approximation of the
classical harmonic function. To tackle this problem, we propose a new method
called the point integral method (PIM). We consider the harmonic extension
problem from the point of view of solving PDEs on manifolds. The basic idea of
the PIM method is to approximate the harmonicity using an integral equation,
which is easy to be discretized from points. Based on the integral equation, we
explain the reason why the transitional graph Laplacian may fail to approximate
the harmonicity in the classical sense and propose a different approach which
we call the volume constraint method (VCM). Theoretically, both the PIM and the
VCM computes a harmonic function with convergence guarantees, and practically,
they are both simple, which amount to solve a linear system. One important
application of the harmonic extension in machine learning is semi-supervised
learning. We run a popular semi-supervised learning algorithm by Zhu et al.
over a couple of well-known datasets and compare the performance of the
aforementioned approaches. Our experiments show the PIM performs the best.Comment: 10 pages, 2 figure
Using Deep Learning to Extend the Range of Air-Pollution Monitoring and Forecasting
Across numerous applications, forecasting relies on numerical solvers for
partial differential equations (PDEs). Although the use of deep-learning
techniques has been proposed, actual applications have been restricted by the
fact the training data are obtained using traditional PDE solvers. Thereby, the
uses of deep-learning techniques were limited to domains, where the PDE solver
was applicable.
We demonstrate a deep-learning framework for air-pollution monitoring and
forecasting that provides the ability to train across different model domains,
as well as a reduction in the run-time by two orders of magnitude. It presents
a first-of-a-kind implementation that combines deep-learning and
domain-decomposition techniques to allow model deployments extend beyond the
domain(s) on which the it has been trained.Comment: 14 pages, 10 figure
Probabilistic Numerical Methods for Partial Differential Equations and Bayesian Inverse Problems
This paper develops a probabilistic numerical method for solution of partial
differential equations (PDEs) and studies application of that method to
PDE-constrained inverse problems. This approach enables the solution of
challenging inverse problems whilst accounting, in a statistically principled
way, for the impact of discretisation error due to numerical solution of the
PDE. In particular, the approach confers robustness to failure of the numerical
PDE solver, with statistical inferences driven to be more conservative in the
presence of substantial discretisation error. Going further, the problem of
choosing a PDE solver is cast as a problem in the Bayesian design of
experiments, where the aim is to minimise the impact of solver error on
statistical inferences; here the challenge of non-linear PDEs is also
considered. The method is applied to parameter inference problems in which
discretisation error in non-negligible and must be accounted for in order to
reach conclusions that are statistically valid
Solving Nonlinear and High-Dimensional Partial Differential Equations via Deep Learning
In this work we apply the Deep Galerkin Method (DGM) described in Sirignano
and Spiliopoulos (2018) to solve a number of partial differential equations
that arise in quantitative finance applications including option pricing,
optimal execution, mean field games, etc. The main idea behind DGM is to
represent the unknown function of interest using a deep neural network. A key
feature of this approach is the fact that, unlike other commonly used numerical
approaches such as finite difference methods, it is mesh-free. As such, it does
not suffer (as much as other numerical methods) from the curse of
dimensionality associated with highdimensional PDEs and PDE systems. The main
goals of this paper are to elucidate the features, capabilities and limitations
of DGM by analyzing aspects of its implementation for a number of different
PDEs and PDE systems. Additionally, we present: (1) a brief overview of PDEs in
quantitative finance along with numerical methods for solving them; (2) a brief
overview of deep learning and, in particular, the notion of neural networks;
(3) a discussion of the theoretical foundations of DGM with a focus on the
justification of why this method is expected to perform well
IMEXnet: A Forward Stable Deep Neural Network
Deep convolutional neural networks have revolutionized many machine learning
and computer vision tasks, however, some remaining key challenges limit their
wider use. These challenges include improving the network's robustness to
perturbations of the input image and the limited ``field of view'' of
convolution operators. We introduce the IMEXnet that addresses these challenges
by adapting semi-implicit methods for partial differential equations. Compared
to similar explicit networks, such as residual networks, our network is more
stable, which has recently shown to reduce the sensitivity to small changes in
the input features and improve generalization. The addition of an implicit step
connects all pixels in each channel of the image and therefore addresses the
field of view problem while still being comparable to standard convolutions in
terms of the number of parameters and computational complexity. We also present
a new dataset for semantic segmentation and demonstrate the effectiveness of
our architecture using the NYU Depth dataset
A mesh-free method for interface problems using the deep learning approach
In this paper, we propose a mesh-free method to solve interface problems
using the deep learning approach. Two interface problems are considered. The
first one is an elliptic PDE with a discontinuous and high-contrast
coefficient. While the second one is a linear elasticity equation with
discontinuous stress tensor. In both cases, we formulate the PDEs into
variational problems, which can be solved via the deep learning approach. To
deal with the inhomogeneous boundary conditions, we use a shallow neuron
network to approximate the boundary conditions. Instead of using an adaptive
mesh refinement method or specially designed basis functions or numerical
schemes to compute the PDE solutions, the proposed method has the advantages
that it is easy to implement and mesh-free. Finally, we present numerical
results to demonstrate the accuracy and efficiency of the proposed method for
interface problems
The Random Feature Model for Input-Output Maps between Banach Spaces
Well known to the machine learning community, the random feature model, originally introduced by Rahimi and Recht in 2008, is a parametric approximation to kernel interpolation or regression methods. It is typically used to approximate functions mapping a finite-dimensional input space to the real line. In this paper, we instead propose a methodology for use of the random feature model as a data-driven surrogate for operators that map an input Banach space to an output Banach space. Although the methodology is quite general, we consider operators defined by partial differential equations (PDEs); here, the inputs and outputs are themselves functions, with the input parameters being functions required to specify the problem, such as initial data or coefficients, and the outputs being solutions of the problem. Upon discretization, the model inherits several desirable attributes from this infinite-dimensional, function space viewpoint, including mesh-invariant approximation error with respect to the true PDE solution map and the capability to be trained at one mesh resolution and then deployed at different mesh resolutions. We view the random feature model as a non-intrusive data-driven emulator, provide a mathematical framework for its interpretation, and demonstrate its ability to efficiently and accurately approximate the nonlinear parameter-to-solution maps of two prototypical PDEs arising in physical science and engineering applications: viscous Burgers' equation and a variable coefficient elliptic equation
- …