16,634 research outputs found
Convergence rates of general regularization methods for statistical inverse problems and applications
During the past the convergence analysis for linear statistical inverse problems has mainly focused on spectral cut-off and Tikhonov type estimators. Spectral cut-off estimators achieve minimax rates for a broad range of smoothness classes and operators, but their practical usefulness is limited by the fact that they require a complete spectral decomposition of the operator. Tikhonov estimators are simpler to compute, but still involve the inversion of an operator and achieve minimax rates only in restricted smoothness classes. In this paper we introduce a unifying technique to study the mean square error of a large class of regularization methods (spectral methods) including the aforementioned estimators as well as many iterative methods, such as í-methods and the Landweber iteration. The latter estimators converge at the same rate as spectral cut-off, but only require matrixvector products. Our results are applied to various problems, in particular we obtain precise convergence rates for satellite gradiometry, L2-boosting, and errors in variable problems. --Statistical inverse problems,iterative regularization methods,Tikhonov regularization,nonparametric regression,minimax convergence rates,satellite gradiometry,Hilbert scales,boosting,errors in variable
Convergence rates of general regularization methods for statistical inverse problems and applications
During the past the convergence analysis for linear statistical inverse problems has mainly focused
on spectral cut-off and Tikhonov type estimators. Spectral cut-off estimators achieve minimax rates for a broad
range of smoothness classes and operators, but their practical usefulness is limited by the fact that they require
a complete spectral decomposition of the operator. Tikhonov estimators are simpler to compute, but still involve
the inversion of an operator and achieve minimax rates only in restricted smoothness classes. In this paper we
introduce a unifying technique to study the mean square error of a large class of regularization methods (spectral
methods) including the aforementioned estimators as well as many iterative methods, such as ν-methods and the
Landweber iteration. The latter estimators converge at the same rate as spectral cut-off, but only require matrixvector
products. Our results are applied to various problems, in particular we obtain precise convergence rates for
satellite gradiometry, L2-boosting, and errors in variable problems.
AMS subject classifications: 62G05, 62J05, 62P35, 65J10, 35R3
Regularization of statistical inverse problems and the Bakushinskii veto
In the deterministic context Bakushinskii's theorem excludes the existence of
purely data driven convergent regularization for ill-posed problems. We will
prove in the present work that in the statistical setting we can either
construct a counter example or develop an equivalent formulation depending on
the considered class of probability distributions. Hence, Bakushinskii's
theorem does not generalize to the statistical context, although this has often
been assumed in the past. To arrive at this conclusion, we will deduce from the
classic theory new concepts for a general study of statistical inverse problems
and perform a systematic clarification of the key ideas of statistical
regularization.Comment: 20 page
Empirical risk minimization as parameter choice rule for general linear regularization methods.
We consider the statistical inverse problem to recover f from noisy measurements Y = Tf + sigma xi where xi is Gaussian white noise and T a compact operator between Hilbert spaces. Considering general reconstruction methods of the form (f) over cap (alpha) = q(alpha) (T*T)T*Y with an ordered filter q(alpha), we investigate the choice of the regularization parameter alpha by minimizing an unbiased estiate of the predictive risk E[parallel to T f - T (f) over cap (alpha)parallel to(2)]. The corresponding parameter alpha(pred) and its usage are well-known in the literature, but oracle inequalities and optimality results in this general setting are unknown. We prove a (generalized) oracle inequality, which relates the direct risk E[parallel to f - (f) over cap (alpha pred)parallel to(2)] with the oracle prediction risk inf(alpha>0) E[parallel to T f - T (f) over cap (alpha)parallel to(2)]. From this oracle inequality we are then able to conclude that the investigated parameter choice rule is of optimal order in the minimax sense. Finally we also present numerical simulations, which support the order optimality of the method and the quality of the parameter choice in finite sample situations
Low Complexity Regularization of Linear Inverse Problems
Inverse problems and regularization theory is a central theme in contemporary
signal processing, where the goal is to reconstruct an unknown signal from
partial indirect, and possibly noisy, measurements of it. A now standard method
for recovering the unknown signal is to solve a convex optimization problem
that enforces some prior knowledge about its structure. This has proved
efficient in many problems routinely encountered in imaging sciences,
statistics and machine learning. This chapter delivers a review of recent
advances in the field where the regularization prior promotes solutions
conforming to some notion of simplicity/low-complexity. These priors encompass
as popular examples sparsity and group sparsity (to capture the compressibility
of natural signals and images), total variation and analysis sparsity (to
promote piecewise regularity), and low-rank (as natural extension of sparsity
to matrix-valued data). Our aim is to provide a unified treatment of all these
regularizations under a single umbrella, namely the theory of partial
smoothness. This framework is very general and accommodates all low-complexity
regularizers just mentioned, as well as many others. Partial smoothness turns
out to be the canonical way to encode low-dimensional models that can be linear
spaces or more general smooth manifolds. This review is intended to serve as a
one stop shop toward the understanding of the theoretical properties of the
so-regularized solutions. It covers a large spectrum including: (i) recovery
guarantees and stability to noise, both in terms of -stability and
model (manifold) identification; (ii) sensitivity analysis to perturbations of
the parameters involved (in particular the observations), with applications to
unbiased risk estimation ; (iii) convergence properties of the forward-backward
proximal splitting scheme, that is particularly well suited to solve the
corresponding large-scale regularized optimization problem
Adaptive complexity regularization for linear inverse problems
We tackle the problem of building adaptive estimation procedures for
ill-posed inverse problems. For general regularization methods depending on
tuning parameters, we construct a penalized method that selects the optimal
smoothing sequence without prior knowledge of the regularity of the function to
be estimated. We provide for such estimators oracle inequalities and optimal
rates of convergence. This penalized approach is applied to Tikhonov
regularization and to regularization by projection.Comment: Published in at http://dx.doi.org/10.1214/07-EJS115 the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …