An adaptive multi-fidelity PC-based ensemble Kalman inversion for inverse problems

The ensemble Kalman inversion (EKI), as a derivative-free methodology, has been widely used in the parameter estimation of inverse problems. Unfortunately, its cost may become moderately large for systems described by high dimensional nonlinear PDEs, as EKI requires a relatively large ensemble size to guarantee its performance. In this paper, we propose an adaptive multi-fidelity polynomial chaos (PC) based EKI technique to address this challenge. Our new strategy combines a large number of low-order PC surrogate model evaluations and a small number of high-fidelity forward model evaluations, yielding a multi-fidelity approach. Especially, we present a new approach that adaptively constructs and refines a multi-fidelity PC surrogate during the EKI simulation. Since the forward model evaluations are only required for updating the low-order multi-fidelity PC model, whose number can be much smaller than the total ensemble size of the classic EKI, the entire computational costs are thus significantly reduced. The new algorithm was tested through the two-dimensional time fractional inverse diffusion problems and demonstrated great effectiveness in comparison with PC based EKI and classic EKI.Comment: arXiv admin note: text overlap with arXiv:1807.0061

Transfer learning based multi-fidelity physics informed deep neural network

For many systems in science and engineering, the governing differential equation is either not known or known in an approximate sense. Analyses and design of such systems are governed by data collected from the field and/or laboratory experiments. This challenging scenario is further worsened when data-collection is expensive and time-consuming. To address this issue, this paper presents a novel multi-fidelity physics informed deep neural network (MF-PIDNN). The framework proposed is particularly suitable when the physics of the problem is known in an approximate sense (low-fidelity physics) and only a few high-fidelity data are available. MF-PIDNN blends physics informed and data-driven deep learning techniques by using the concept of transfer learning. The approximate governing equation is first used to train a low-fidelity physics informed deep neural network. This is followed by transfer learning where the low-fidelity model is updated by using the available high-fidelity data. MF-PIDNN is able to encode useful information on the physics of the problem from the {\it approximate} governing differential equation and hence, provides accurate prediction even in zones with no data. Additionally, no low-fidelity data is required for training this model. Applicability and utility of MF-PIDNN are illustrated in solving four benchmark reliability analysis problems. Case studies to illustrate interesting features of the proposed approach are also presented

Bayesian identification of discontinuous fields with an ensemble-based variable separation multiscale method

This work presents a multiscale model reduction approach to discontinuous fields identification problems in the framework of Bayesian inference. An ensemble-based variable separation (VS) method is proposed to approximate multiscale basis functions used to build a coarse model. The variable-separation expression is constructed for stochastic multiscale basis functions based on the random field, which is treated Gauss process as prior information. To this end, multiple local inhomogeneous Dirichlet boundary condition problems are required to be solved, and the ensemble-based method is used to obtain variable separation forms for the corresponding local functions. The local functions share the same interpolate rule for different physical basis functions in each coarse block. This approach significantly improves the efficiency of computation. We obtain the variable separation expression of multiscale basis functions, which can be used to the models with different boundary conditions and source terms, once the expression constructed. The proposed method is applied to discontinuous field identification problems where the hybrid of total variation and Gaussian (TG) densities are imposed as the penalty. We give a convergence analysis of the approximate posterior to the reference one with respect to the Kullback-Leibler (KL) divergence under the hybrid prior. The proposed method is applied to identify discontinuous structures in permeability fields. Two patterns of discontinuous structures are considered in numerical examples: separated blocks and nested blocks

Inverse scattering by a random periodic structure

This paper develops an efficient numerical method for the inverse scattering problem of a time-harmonic plane wave incident on a perfectly reflecting random periodic structure. The method is based on a novel combination of the Monte Carlo technique for sampling the probability space, a continuation method with respect to the wavenumber, and the Karhunen-Lo$\grave{e}$ve expansion of the random structure, which reconstructs key statistical properties of the profile for the unknown random periodic structure from boundary measurements of the scattered fields away from the structure. Numerical results are presented to demonstrate the reliability and efficiency of the proposed method

An acceleration strategy for randomize-then-optimize sampling via deep neural networks

Randomize-then-optimize (RTO) is widely used for sampling from posterior distributions in Bayesian inverse problems. However, RTO may be computationally intensive for complexity problems due to repetitive evaluations of the expensive forward model and its gradient. In this work, we present a novel strategy to substantially reduce the computation burden of RTO by using a goal-oriented deep neural networks (DNN) surrogate approach. In particular, the training points for the DNN-surrogate are drawn from a local approximated posterior distribution, and it is shown that the resulting algorithm can provide a flexible and efficient sampling algorithm, which converges to the direct RTO approach. We present a Bayesian inverse problem governed by a benchmark elliptic PDE to demonstrate the computational accuracy and efficiency of our new algorithm (i.e., DNN-RTO). It is shown that with our algorithm, one can significantly outperform the traditional RTO.Comment: to appear in J. Comput. Mat

An artificial neural network approximation for Cauchy inverse problems

A novel artificial neural network method is proposed for solving Cauchy inverse problems. It allows multiple hidden layers with arbitrary width and depth, which theoretically yields better approximations to the inverse problems. In this research, the existence and convergence are shown to establish the well-posedness of neural network method for Cauchy inverse problems, and various numerical examples are presented to illustrate its accuracy and stability. The numerical examples are from different points of view, including time-dependent and time-independent cases, high spatial dimension cases up to 8D, and cases with noisy boundary data and singular computational domain. Moreover, numerical results also show that neural networks with wider and deeper hidden layers could lead to better approximation for Cauchy inverse problems.Comment: 32 pages, 56 figure

An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems

In Bayesian inverse problems, surrogate models are often constructed to speed up the computational procedure, as the parameter-to-data map can be very expensive to evaluate. However, due to the curse of dimensionality and the nonlinear concentration of the posterior, traditional surrogate approaches (such us the polynomial-based surrogates) are still not feasible for large scale problems. To this end, we present in this work an adaptive multi-fidelity surrogate modeling framework based on deep neural networks (DNNs), motivated by the facts that the DNNs can potentially handle functions with limited regularity and are powerful tools for high dimensional approximations. More precisely, we first construct offline a DNNs-based surrogate according to the prior distribution, and then, this prior-based DNN-surrogate will be adaptively \& locally refined online using only a few high-fidelity simulations. In particular, in the refine procedure, we construct a new shallow neural network that view the previous constructed surrogate as an input variable -- yielding a composite multi-fidelity neural network approach. This makes the online computational procedure rather efficient. Numerical examples are presented to confirm that the proposed approach can obtain accurate posterior information with a limited number of forward simulations

A non-intrusive reduced basis EKI for time-fractional diffusion inverse problems

In this study, we consider an ensemble Kalman inversion (EKI) for the numerical solution of time-fractional diffusion inverse problems (TFDIPs). Computational challenges in the EKI arise from the need for repeated evaluations of the forward model. We address this challenge by introducing a non-intrusive reduced basis (RB) method for constructing surrogate models to reduce computational cost. In this method, a reduced basis is extracted from a set of full-order snapshots by the proper orthogonal decomposition (POD), and a doubly stochastic radial basis function (DSRBF) is used to learn the projection coefficients. The DSRBF is carried out in the offline stage with a stochastic leave-one-out cross-validation algorithm to select the shape parameter, and the outputs for new parameter values can be obtained rapidly during the online stage. Due to the complete decoupling of the offline and online stages, the proposed non-intrusive RB method -- referred to as POD-DSRBF -- provides a powerful tool to accelerate the EKI approach for TFDIPs. We demonstrate the practical performance of the proposed strategies through two nonlinear time-fractional diffusion inverse problems. The numerical results indicate that the new algorithm can achieve significant computational gains without sacrificing accuracy

Multilevel adaptive sparse Leja approximations for Bayesian inverse problems

Deterministic interpolation and quadrature methods are often unsuitable to address Bayesian inverse problems depending on computationally expensive forward mathematical models. While interpolation may give precise posterior approximations, deterministic quadrature is usually unable to efficiently investigate an informative and thus concentrated likelihood. This leads to a large number of required expensive evaluations of the mathematical model. To overcome these challenges, we formulate and test a multilevel adaptive sparse Leja algorithm. At each level, adaptive sparse grid interpolation and quadrature are used to approximate the posterior and perform all quadrature operations, respectively. Specifically, our algorithm uses coarse discretizations of the underlying mathematical model to investigate the parameter space and to identify areas of high posterior probability. Adaptive sparse grid algorithms are then used to place points in these areas, and ignore other areas of small posterior probability. The points are weighted Leja points. As the model discretization is coarse, the construction of the sparse grid is computationally efficient. On this sparse grid, the posterior measure can be approximated accurately with few expensive, fine model discretizations. The efficiency of the algorithm can be enhanced further by exploiting more than two discretization levels. We apply the proposed multilevel adaptive sparse Leja algorithm in numerical experiments involving elliptic inverse problems in 2D and 3D space, in which we compare it with Markov chain Monte Carlo sampling and a standard multilevel approximation.Comment: 24 pages, 9 figure

Parameter identification in uncertain scalar conservation laws discretized with the discontinuous stochastic Galerkin Scheme

We study an identification problem which estimates the parameters of the underlying random distribution for uncertain scalar conservation laws. The hyperbolic equations are discretized with the so-called discontinuous stochastic Galerkin method, i.e., using a spatial discontinuous Galerkin scheme and a Multielement stochastic Galerkin ansatz in the random space. We assume an uncertain flux or uncertain initial conditions and that a data set of an observed solution is given. The uncertainty is assumed to be uniformly distributed on an unknown interval and we focus on identifying the correct endpoints of this interval. The first-order optimality conditions from the discontinuous stochastic Galerkin discretization are computed on the time-continuous level. Then, we solve the resulting semi-discrete forward and backward schemes with the Runge Kutta method. To illustrate the feasibility of the approach, we apply the method to a stochastic advection and a stochastic equation of Burgers' type. The results show that the method is able to identify the distribution parameters of the random variable in the uncertain differential equation even if discontinuities are present