43 research outputs found
Ensemble Kalman filter for neural network based one-shot inversion
We study the use of novel techniques arising in machine learning for inverse
problems. Our approach replaces the complex forward model by a neural network,
which is trained simultaneously in a one-shot sense when estimating the unknown
parameters from data, i.e. the neural network is trained only for the unknown
parameter. By establishing a link to the Bayesian approach to inverse problems,
an algorithmic framework is developed which ensures the feasibility of the
parameter estimate w.r. to the forward model. We propose an efficient,
derivative-free optimization method based on variants of the ensemble Kalman
inversion. Numerical experiments show that the ensemble Kalman filter for
neural network based one-shot inversion is a promising direction combining
optimization and machine learning techniques for inverse problems
ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units using Chebyshev Approximations
In a recent paper [B. Li, S. Tang and H. Yu, arXiv:1903.05858], it was shown
that deep neural networks built with rectified power units (RePU) can give
better approximation for sufficient smooth functions than those with rectified
linear units, by converting polynomial approximation given in power series into
deep neural networks with optimal complexity and no approximation error.
However, in practice, power series are not easy to compute. In this paper, we
propose a new and more stable way to construct deep RePU neural networks based
on Chebyshev polynomial approximations. By using a hierarchical structure of
Chebyshev polynomial approximation in frequency domain, we build efficient and
stable deep neural network constructions. In theory, ChebNets and the deep RePU
nets based on Power series have the same upper error bounds for general
function approximations. But numerically, ChebNets are much more stable.
Numerical results show that the constructed ChebNets can be further trained and
obtain much better results than those obtained by training deep RePU nets
constructed basing on power series.Comment: 18 pages, 6 figures, 2 table
Model Reduction and Neural Networks for Parametric PDEs
We develop a general framework for data-driven approximation of input-output maps between infinite-dimensional spaces. The proposed approach is motivated by the recent successes of neural networks and deep learning, in combination with ideas from model reduction. This combination results in a neural network approximation which, in principle, is defined on infinite-dimensional spaces and, in practice, is robust to the dimension of finite-dimensional approximations of these spaces required for computation. For a class of input-output maps, and suitably chosen probability measures on the inputs, we prove convergence of the proposed approximation methodology. Numerically we demonstrate the effectiveness of the method on a class of parametric elliptic PDE problems, showing convergence and robustness of the approximation scheme with respect to the size of the discretization, and compare our method with existing algorithms from the literature