2 research outputs found
Learning the optimal Tikhonov regularizer for inverse problems
Publisher Copyright: © 2021 Neural information processing systems foundation. All rights reserved.In this work, we consider the linear inverse problem y = Ax+ε, where A: X → Y is a known linear operator between the separable Hilbert spaces X and Y, x is a random variable in X and ε is a zero-mean random process in Y . This setting covers several inverse problems in imaging including denoising, deblurring and X-ray tomography. Within the classical framework of regularization, we focus on the case where the regularization functional is not given a priori, but learned from data. Our first result is a characterization of the optimal generalized Tikhonov regularizer, with respect to the mean squared error. We find that it is completely independent of the forward operator A and depends only on the mean and covariance of x. Then, we consider the problem of learning the regularizer from a finite training set in two different frameworks: one supervised, based on samples of both x and y, and one unsupervised, based only on samples of x. In both cases we prove generalization bounds, under some weak assumptions on the distribution of x and ε, including the case of sub-Gaussian variables. Our bounds hold in infinite-dimensional spaces, thereby showing that finer and finer discretizations do not make this learning problem harder. The results are validated through numerical simulations.Peer reviewe
Convergence Rates for Learning Linear Operators from Noisy Data
This paper studies the learning of linear operators between
infinite-dimensional Hilbert spaces. The training data comprises pairs of
random input vectors in a Hilbert space and their noisy images under an unknown
self-adjoint linear operator. Assuming that the operator is diagonalizable in a
known basis, this work solves the equivalent inverse problem of estimating the
operator's eigenvalues given the data. Adopting a Bayesian approach, the
theoretical analysis establishes posterior contraction rates in the infinite
data limit with Gaussian priors that are not directly linked to the forward map
of the inverse problem. The main results also include learning-theoretic
generalization error guarantees for a wide range of distribution shifts. These
convergence rates quantify the effects of data smoothness and true eigenvalue
decay or growth, for compact or unbounded operators, respectively, on sample
complexity. Numerical evidence supports the theory in diagonal and non-diagonal
settings.Comment: To appear in SIAM/ASA Journal on Uncertainty Quantification (JUQ); 34
pages, 5 figures, 2 table