58 research outputs found
Optimization Methods for Inverse Problems
Optimization plays an important role in solving many inverse problems.
Indeed, the task of inversion often either involves or is fully cast as a
solution of an optimization problem. In this light, the mere non-linear,
non-convex, and large-scale nature of many of these inversions gives rise to
some very challenging optimization problems. The inverse problem community has
long been developing various techniques for solving such optimization tasks.
However, other, seemingly disjoint communities, such as that of machine
learning, have developed, almost in parallel, interesting alternative methods
which might have stayed under the radar of the inverse problem community. In
this survey, we aim to change that. In doing so, we first discuss current
state-of-the-art optimization methods widely used in inverse problems. We then
survey recent related advances in addressing similar challenges in problems
faced by the machine learning community, and discuss their potential advantages
for solving inverse problems. By highlighting the similarities among the
optimization challenges faced by the inverse problem and the machine learning
communities, we hope that this survey can serve as a bridge in bringing
together these two communities and encourage cross fertilization of ideas.Comment: 13 page
Semivariogram methods for modeling Whittle-Mat\'ern priors in Bayesian inverse problems
We present a new technique, based on semivariogram methodology, for obtaining
point estimates for use in prior modeling for solving Bayesian inverse
problems. This method requires a connection between Gaussian processes with
covariance operators defined by the Mat\'ern covariance function and Gaussian
processes with precision (inverse-covariance) operators defined by the Green's
functions of a class of elliptic stochastic partial differential equations
(SPDEs). We present a detailed mathematical description of this connection. We
will show that there is an equivalence between these two Gaussian processes
when the domain is infinite -- for us, -- which breaks down when
the domain is finite due to the effect of boundary conditions on Green's
functions of PDEs. We show how this connection can be re-established using
extended domains. We then introduce the semivariogram method for estimating the
Mat\'ern covariance parameters, which specify the Gaussian prior needed for
stabilizing the inverse problem. Results are extended from the isotropic case
to the anisotropic case where the correlation length in one direction is larger
than another. Finally, we consider the situation where the correlation length
is spatially dependent rather than constant. We implement each method in
two-dimensional image inpainting test cases to show that it works on practical
examples
Tensor-train methods for sequential state and parameter learning in state-space models
We consider sequential state and parameter learning in state-space models
with intractable state transition and observation processes. By exploiting
low-rank tensor-train (TT) decompositions, we propose new sequential learning
methods for joint parameter and state estimation under the Bayesian framework.
Our key innovation is the introduction of scalable function approximation tools
such as TT for recursively learning the sequentially updated posterior
distributions. The function approximation perspective of our methods offers
tractable error analysis and potentially alleviates the particle degeneracy
faced by many particle-based methods. In addition to the new insights into
algorithmic design, our methods complement conventional particle-based methods.
Our TT-based approximations naturally define conditional Knothe--Rosenblatt
(KR) rearrangements that lead to filtering, smoothing and path estimation
accompanying our sequential learning algorithms, which open the door to
removing potential approximation bias. We also explore several preconditioning
techniques based on either linear or nonlinear KR rearrangements to enhance the
approximation power of TT for practical problems. We demonstrate the efficacy
and efficiency of our proposed methods on several state-space models, in which
our methods achieve state-of-the-art estimation accuracy and computational
performance
Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems
One of the major challenges in the Bayesian solution of inverse problems
governed by partial differential equations (PDEs) is the computational cost of
repeatedly evaluating numerical PDE models, as required by Markov chain Monte
Carlo (MCMC) methods for posterior sampling. This paper proposes a data-driven
projection-based model reduction technique to reduce this computational cost.
The proposed technique has two distinctive features. First, the model reduction
strategy is tailored to inverse problems: the snapshots used to construct the
reduced-order model are computed adaptively from the posterior distribution.
Posterior exploration and model reduction are thus pursued simultaneously.
Second, to avoid repeated evaluations of the full-scale numerical model as in a
standard MCMC method, we couple the full-scale model and the reduced-order
model together in the MCMC algorithm. This maintains accurate inference while
reducing its overall computational cost. In numerical experiments considering
steady-state flow in a porous medium, the data-driven reduced-order model
achieves better accuracy than a reduced-order model constructed using the
classical approach. It also improves posterior sampling efficiency by several
orders of magnitude compared to a standard MCMC method
Bernstein approximation and beyond: proofs by means of elementary probability theory
Bernstein polynomials provide a constructive proof for the Weierstrass
approximation theorem, which states that every continuous function on a closed
bounded interval can be uniformly approximated by polynomials with arbitrary
accuracy. Interestingly the proof of this result can be done using elementary
probability theory. This way one can even get error bounds for Lipschitz
functions. In this note, we present these techniques and show how the method
can be extended naturally to other interesting situations. As examples, we
obtain in an elementary way results for the Sz\'{a}sz-Mirakjan operator and the
Baskakov operator
Certified dimension reduction in nonlinear Bayesian inverse problems
We propose a dimension reduction technique for Bayesian inverse problems with
nonlinear forward operators, non-Gaussian priors, and non-Gaussian observation
noise. The likelihood function is approximated by a ridge function, i.e., a map
which depends non-trivially only on a few linear combinations of the
parameters. We build this ridge approximation by minimizing an upper bound on
the Kullback--Leibler divergence between the posterior distribution and its
approximation. This bound, obtained via logarithmic Sobolev inequalities,
allows one to certify the error of the posterior approximation. Computing the
bound requires computing the second moment matrix of the gradient of the
log-likelihood function. In practice, a sample-based approximation of the upper
bound is then required. We provide an analysis that enables control of the
posterior approximation error due to this sampling. Numerical and theoretical
comparisons with existing methods illustrate the benefits of the proposed
methodology
- …