Robust Least Squares Methods Under Bounded Data Uncertainties

Cataloged from PDF version of article.We study the problem of estimating an unknown deterministic signal that is observed through an unknown deterministic data matrix under additive noise. In particular, we present a minimax optimization framework to the least squares problems, where the estimator has imperfect data matrix and output vector information. We define the performance of an estimator relative to the performance of the optimal least squares (LS) estimator tuned to the underlying unknown data matrix and output vector, which is defined as the regret of the estimator. We then introduce an efficient robust LS estimation approach that minimizes this regret for the worst possible data matrix and output vector, where we refrain from any structural assumptions on the data. We demonstrate that minimizing this worst-case regret can be cast as a semi-definite programming (SDP) problem. We then consider the regularized and structured LS problems and present novel robust estimation methods by demonstrating that these problems can also be cast as SDP problems. We illustrate the merits of the proposed algorithms with respect to the well-known alternatives in the literature through our simulations

Estimation and control with bounded data uncertainties

AbstractThe paper describes estimation and control strategies for models with bounded data uncertainties. We shall refer to them as BDU estimation and BDU control methods, for brevity. They are based on constrained game-type formulations that allow the designer to explicitly incorporate into the problem statement a priori information about bounds on the sizes of the uncertainties. In this way, the effect of uncertainties is not unnecessarily over-emphasized beyond what is implied by the a priori bounds; consequently, overly conservative designs, as well as overly sensitive designs, are avoided. A feature of these new formulations is that geometric insights and recursive techniques, which are widely known and appreciated for classical quadratic-cost designs, can also be pursued in this new framework. Also, algorithms for computing the optimal solutions with the same computational effort as standard least-squares solutions exist, thus making the new formulations attractive for practical use. Moreover, the framework is broad enough to encompass applications across several disciplines, not just estimation and control. Examples will be given of a quadratic control design, an H∞ control design, a total-least-square design, image restoration, image separation, and co-channel interference cancellation. A major theme in this paper is the emphasis on geometric and linear algebraic arguments, which lead to useful insights about the nature of the new formulations. Despite the interesting results that will be discussed, several issues remain open and indicate potential future developments; these will be briefly discussed

Robust and Stable Predictive Control with Bounded Uncertainties

[EN] Min-Max optimization is often used for improving robustness in Model Predictive Control (MPC). An analogy to this optimization could be the BDU (Bounded Data Uncertainties) method, which is a regularization technique for least-squares problems that takes into account the uncertainty bounds. Stability of MPC can be achieved by using terminal constraints, such as in the CRHPC (Constrained Receding-Horizon Predictive Control) algorithm. By combining both BDU and CRHPC methods, a robust and stable MPC is obtained, which is the aim of this work. BDU also offers a guided method of tuning the empirically tuned penalization parameter for the control effort in MPC. (C) 2008 Elsevier Inc. All rights reserved.This work has been partially financed by DPI2005-07835 and DPI2004-08383-C03-02 MEC-FEDER.Ramos Fernández, C.; Martínez Iranzo, MA.; Sanchís Saez, J.; Herrero Durá, JM. (2008). Robust and Stable Predictive Control with Bounded Uncertainties. Journal of Mathematical Analysis and Applications. 342(2):1003-1014. https://doi.org/10.1016/j.jmaa.2007.12.073S10031014342

A non-adapted sparse approximation of PDEs with stochastic inputs

We propose a method for the approximation of solutions of PDEs with stochastic coefficients based on the direct, i.e., non-adapted, sampling of solutions. This sampling can be done by using any legacy code for the deterministic problem as a black box. The method converges in probability (with probabilistic error bounds) as a consequence of sparsity and a concentration of measure phenomenon on the empirical correlation between samples. We show that the method is well suited for truly high-dimensional problems (with slow decay in the spectrum)

Structured least squares problems and robust estimators

Cataloged from PDF version of article.A novel approach is proposed to provide robust and accurate estimates for linear regression problems when both the measurement vector and the coefficient matrix are structured and subject to errors or uncertainty. A new analytic formulation is developed in terms of the gradient flow of the residual norm to analyze and provide estimates to the regression. The presented analysis enables us to establish theoretical performance guarantees to compare with existing methods and also offers a criterion to choose the regularization parameter autonomously. Theoretical results and simulations in applications such as blind identification, multiple frequency estimation and deconvolution show that the proposed technique outperforms alternative methods in mean-squared error for a significant range of signal-to-noise ratio values

Generalized SURE for Exponential Families: Applications to Regularization

Stein's unbiased risk estimate (SURE) was proposed by Stein for the independent, identically distributed (iid) Gaussian model in order to derive estimates that dominate least-squares (LS). In recent years, the SURE criterion has been employed in a variety of denoising problems for choosing regularization parameters that minimize an estimate of the mean-squared error (MSE). However, its use has been limited to the iid case which precludes many important applications. In this paper we begin by deriving a SURE counterpart for general, not necessarily iid distributions from the exponential family. This enables extending the SURE design technique to a much broader class of problems. Based on this generalization we suggest a new method for choosing regularization parameters in penalized LS estimators. We then demonstrate its superior performance over the conventional generalized cross validation approach and the discrepancy method in the context of image deblurring and deconvolution. The SURE technique can also be used to design estimates without predefining their structure. However, allowing for too many free parameters impairs the performance of the resulting estimates. To address this inherent tradeoff we propose a regularized SURE objective. Based on this design criterion, we derive a wavelet denoising strategy that is similar in sprit to the standard soft-threshold approach but can lead to improved MSE performance.Comment: to appear in the IEEE Transactions on Signal Processin

A Probabilistic Approach to Robust Optimal Experiment Design with Chance Constraints

Accurate estimation of parameters is paramount in developing high-fidelity models for complex dynamical systems. Model-based optimal experiment design (OED) approaches enable systematic design of dynamic experiments to generate input-output data sets with high information content for parameter estimation. Standard OED approaches however face two challenges: (i) experiment design under incomplete system information due to unknown true parameters, which usually requires many iterations of OED; (ii) incapability of systematically accounting for the inherent uncertainties of complex systems, which can lead to diminished effectiveness of the designed optimal excitation signal as well as violation of system constraints. This paper presents a robust OED approach for nonlinear systems with arbitrarily-shaped time-invariant probabilistic uncertainties. Polynomial chaos is used for efficient uncertainty propagation. The distinct feature of the robust OED approach is the inclusion of chance constraints to ensure constraint satisfaction in a stochastic setting. The presented approach is demonstrated by optimal experimental design for the JAK-STAT5 signaling pathway that regulates various cellular processes in a biological cell.Comment: Submitted to ADCHEM 201

High-dimensional semi-supervised learning: in search for optimal inference of the mean

We provide a high-dimensional semi-supervised inference framework focused on the mean and variance of the response. Our data are comprised of an extensive set of observations regarding the covariate vectors and a much smaller set of labeled observations where we observe both the response as well as the covariates. We allow the size of the covariates to be much larger than the sample size and impose weak conditions on a statistical form of the data. We provide new estimators of the mean and variance of the response that extend some of the recent results presented in low-dimensional models. In particular, at times we will not necessitate consistent estimation of the functional form of the data. Together with estimation of the population mean and variance, we provide their asymptotic distribution and confidence intervals where we showcase gains in efficiency compared to the sample mean and variance. Our procedure, with minor modifications, is then presented to make important contributions regarding inference about average treatment effects. We also investigate the robustness of estimation and coverage and showcase widespread applicability and generality of the proposed method

Theory and Applications of Robust Optimization

In this paper we survey the primary research, both theoretical and applied, in the area of Robust Optimization (RO). Our focus is on the computational attractiveness of RO approaches, as well as the modeling power and broad applicability of the methodology. In addition to surveying prominent theoretical results of RO, we also present some recent results linking RO to adaptable models for multi-stage decision-making problems. Finally, we highlight applications of RO across a wide spectrum of domains, including finance, statistics, learning, and various areas of engineering.Comment: 50 page