27,073 research outputs found
Discontinuous Parameter Estimates with Least Squares Estimators
We discuss weighted least squares estimates of ill-conditioned linear inverse problems where weights are chosen to be inverse error covariance matrices. Least squares estimators are the maximum likelihood estimate for normally distributed data and parameters, but here we do not assume particular probability distributions. Weights for the estimator are found by ensuring its minimum follows a χ2 distribution. Previous work with this approach has shown that it is competitive with regularization methods such as the L-curve and Generalized Cross Validation (GCV) [20]. In this work we extend the method to find diagonal weighting matrices, rather than a scalar regularization parameter. Diagonal weighting matrices are advantageous because they give piecewise smooth least squares estimates and hence are a mechanism through which least squares can be used to estimate discontinuous parameters. This is explained by viewing least squares estimation as a constrained optimization problem. Results with diagonal weighting matrices are given for a benchmark discontinuous inverse problem from [13]. In addition, the method is used to estimate soil moisture from data collected in the Dry Creek Watershed near Boise, Idaho. Parameter estimates are found that combine two different types of measurements, and weighting matrices are found that incorporate uncertainty due to spatial variation so that the parameters can be used over larger scales than those that were measured
Generalized SURE for Exponential Families: Applications to Regularization
Stein's unbiased risk estimate (SURE) was proposed by Stein for the
independent, identically distributed (iid) Gaussian model in order to derive
estimates that dominate least-squares (LS). In recent years, the SURE criterion
has been employed in a variety of denoising problems for choosing
regularization parameters that minimize an estimate of the mean-squared error
(MSE). However, its use has been limited to the iid case which precludes many
important applications. In this paper we begin by deriving a SURE counterpart
for general, not necessarily iid distributions from the exponential family.
This enables extending the SURE design technique to a much broader class of
problems. Based on this generalization we suggest a new method for choosing
regularization parameters in penalized LS estimators. We then demonstrate its
superior performance over the conventional generalized cross validation
approach and the discrepancy method in the context of image deblurring and
deconvolution. The SURE technique can also be used to design estimates without
predefining their structure. However, allowing for too many free parameters
impairs the performance of the resulting estimates. To address this inherent
tradeoff we propose a regularized SURE objective. Based on this design
criterion, we derive a wavelet denoising strategy that is similar in sprit to
the standard soft-threshold approach but can lead to improved MSE performance.Comment: to appear in the IEEE Transactions on Signal Processin
Recommended from our members
Obtaining sparse distributions in 2D inverse problems
The mathematics of inverse problems has relevance across numerous estimation problems in science and engineering. L1 regularization has attracted recent attention in reconstructing the system properties in the case of sparse inverse problems; i.e., when the true property sought is not adequately described by a continuous distribution, in particular in Compressed Sensing image reconstruction. In this work, we focus on the application of L1 regularization to a class of inverse problems; relaxation-relaxation, T1–T2, and diffusion-relaxation, D–T2, correlation experiments in NMR, which have found widespread applications in a number of areas including probing surface interactions in catalysis and characterizing fluid composition and pore structures in rocks. We introduce a robust algorithm for solving the L1 regularization problem and provide a guide to implementing it, including the choice of the amount of regularization used and the assignment of error estimates. We then show experimentally that L1 regularization has significant advantages over both the Non-Negative Least Squares (NNLS) algorithm and Tikhonov regularization. It is shown that the L1 regularization algorithm stably recovers a distribution at a signal to noise ratio < 20 and that it resolves relaxation time constants and diffusion coefficients differing by as little as 10%. The enhanced resolving capability is used to measure the inter and intra particle concentrations of a mixture of hexane and dodecane present within porous silica beads immersed within a bulk liquid phase; neither NNLS nor Tikhonov regularization are able to provide this resolution. This experimental study shows that the approach enables discrimination between different chemical species when direct spectroscopic discrimination is impossible, and hence measurement of chemical composition within porous media, such as catalysts or rocks, is possible while still being stable to high levels of noise.A.R. acknowledges Gates Trust Cambridge for financial support. A.J.S. and L.F.G. would like to acknowledge support from EPSRC (EP/N009304/1)
Primal-Dual Mixed Finite Element Methods for the Elliptic Cauchy Problem
We consider primal-dual mixed finite element methods for the solution of the elliptic
Cauchy problem, or other related data assimilation problems. The method has a local conservation property. We derive a priori error estimates using known conditional stability estimates and
determine the minimal amount of weakly consistent stabilization and Tikhonov regularization that
yields optimal convergence for smooth exact solutions. The effect of perturbations in data is also
accounted for. A reduced version of the method, obtained by choosing a special stabilization of the
dual variable, can be viewed as a variant of the least squares mixed finite element method introduced
by Dard´e, Hannukainen, and Hyv¨onen in [SIAM J. Numer. Anal., 51 (2013), pp. 2123–2148]. The
main difference is that our choice of regularization does not depend on auxiliary parameters, the
mesh size being the only asymptotic parameter. Finally, we show that the reduced method can
be used for defect correction iteration to determine the solution of the full method. The theory is
illustrated by some numerical examples
Structured penalties for functional linear models---partially empirical eigenvectors for regression
One of the challenges with functional data is incorporating spatial
structure, or local correlation, into the analysis. This structure is inherent
in the output from an increasing number of biomedical technologies, and a
functional linear model is often used to estimate the relationship between the
predictor functions and scalar responses. Common approaches to the ill-posed
problem of estimating a coefficient function typically involve two stages:
regularization and estimation. Regularization is usually done via dimension
reduction, projecting onto a predefined span of basis functions or a reduced
set of eigenvectors (principal components). In contrast, we present a unified
approach that directly incorporates spatial structure into the estimation
process by exploiting the joint eigenproperties of the predictors and a linear
penalty operator. In this sense, the components in the regression are
`partially empirical' and the framework is provided by the generalized singular
value decomposition (GSVD). The GSVD clarifies the penalized estimation process
and informs the choice of penalty by making explicit the joint influence of the
penalty and predictors on the bias, variance, and performance of the estimated
coefficient function. Laboratory spectroscopy data and simulations are used to
illustrate the concepts.Comment: 29 pages, 3 figures, 5 tables; typo/notational errors edited and
intro revised per journal review proces
- …