27,073 research outputs found

    Discontinuous Parameter Estimates with Least Squares Estimators

    Get PDF
    We discuss weighted least squares estimates of ill-conditioned linear inverse problems where weights are chosen to be inverse error covariance matrices. Least squares estimators are the maximum likelihood estimate for normally distributed data and parameters, but here we do not assume particular probability distributions. Weights for the estimator are found by ensuring its minimum follows a χ2 distribution. Previous work with this approach has shown that it is competitive with regularization methods such as the L-curve and Generalized Cross Validation (GCV) [20]. In this work we extend the method to find diagonal weighting matrices, rather than a scalar regularization parameter. Diagonal weighting matrices are advantageous because they give piecewise smooth least squares estimates and hence are a mechanism through which least squares can be used to estimate discontinuous parameters. This is explained by viewing least squares estimation as a constrained optimization problem. Results with diagonal weighting matrices are given for a benchmark discontinuous inverse problem from [13]. In addition, the method is used to estimate soil moisture from data collected in the Dry Creek Watershed near Boise, Idaho. Parameter estimates are found that combine two different types of measurements, and weighting matrices are found that incorporate uncertainty due to spatial variation so that the parameters can be used over larger scales than those that were measured

    Generalized SURE for Exponential Families: Applications to Regularization

    Full text link
    Stein's unbiased risk estimate (SURE) was proposed by Stein for the independent, identically distributed (iid) Gaussian model in order to derive estimates that dominate least-squares (LS). In recent years, the SURE criterion has been employed in a variety of denoising problems for choosing regularization parameters that minimize an estimate of the mean-squared error (MSE). However, its use has been limited to the iid case which precludes many important applications. In this paper we begin by deriving a SURE counterpart for general, not necessarily iid distributions from the exponential family. This enables extending the SURE design technique to a much broader class of problems. Based on this generalization we suggest a new method for choosing regularization parameters in penalized LS estimators. We then demonstrate its superior performance over the conventional generalized cross validation approach and the discrepancy method in the context of image deblurring and deconvolution. The SURE technique can also be used to design estimates without predefining their structure. However, allowing for too many free parameters impairs the performance of the resulting estimates. To address this inherent tradeoff we propose a regularized SURE objective. Based on this design criterion, we derive a wavelet denoising strategy that is similar in sprit to the standard soft-threshold approach but can lead to improved MSE performance.Comment: to appear in the IEEE Transactions on Signal Processin

    Primal-Dual Mixed Finite Element Methods for the Elliptic Cauchy Problem

    Get PDF
    We consider primal-dual mixed finite element methods for the solution of the elliptic Cauchy problem, or other related data assimilation problems. The method has a local conservation property. We derive a priori error estimates using known conditional stability estimates and determine the minimal amount of weakly consistent stabilization and Tikhonov regularization that yields optimal convergence for smooth exact solutions. The effect of perturbations in data is also accounted for. A reduced version of the method, obtained by choosing a special stabilization of the dual variable, can be viewed as a variant of the least squares mixed finite element method introduced by Dard´e, Hannukainen, and Hyv¨onen in [SIAM J. Numer. Anal., 51 (2013), pp. 2123–2148]. The main difference is that our choice of regularization does not depend on auxiliary parameters, the mesh size being the only asymptotic parameter. Finally, we show that the reduced method can be used for defect correction iteration to determine the solution of the full method. The theory is illustrated by some numerical examples

    Structured penalties for functional linear models---partially empirical eigenvectors for regression

    Get PDF
    One of the challenges with functional data is incorporating spatial structure, or local correlation, into the analysis. This structure is inherent in the output from an increasing number of biomedical technologies, and a functional linear model is often used to estimate the relationship between the predictor functions and scalar responses. Common approaches to the ill-posed problem of estimating a coefficient function typically involve two stages: regularization and estimation. Regularization is usually done via dimension reduction, projecting onto a predefined span of basis functions or a reduced set of eigenvectors (principal components). In contrast, we present a unified approach that directly incorporates spatial structure into the estimation process by exploiting the joint eigenproperties of the predictors and a linear penalty operator. In this sense, the components in the regression are `partially empirical' and the framework is provided by the generalized singular value decomposition (GSVD). The GSVD clarifies the penalized estimation process and informs the choice of penalty by making explicit the joint influence of the penalty and predictors on the bias, variance, and performance of the estimated coefficient function. Laboratory spectroscopy data and simulations are used to illustrate the concepts.Comment: 29 pages, 3 figures, 5 tables; typo/notational errors edited and intro revised per journal review proces
    • …
    corecore