On Degrees of Freedom of Projection Estimators with Applications to Multivariate Nonparametric Regression

In this paper, we consider the nonparametric regression problem with multivariate predictors. We provide a characterization of the degrees of freedom and divergence for estimators of the unknown regression function, which are obtained as outputs of linearly constrained quadratic optimization procedures, namely, minimizers of the least squares criterion with linear constraints and/or quadratic penalties. As special cases of our results, we derive explicit expressions for the degrees of freedom in many nonparametric regression problems, e.g., bounded isotonic regression, multivariate (penalized) convex regression, and additive total variation regularization. Our theory also yields, as special cases, known results on the degrees of freedom of many well-studied estimators in the statistics literature, such as ridge regression, Lasso and generalized Lasso. Our results can be readily used to choose the tuning parameter(s) involved in the estimation procedure by minimizing the Stein's unbiased risk estimate. As a by-product of our analysis we derive an interesting connection between bounded isotonic regression and isotonic regression on a general partially ordered set, which is of independent interest.Comment: 72 pages, 7 figures, Journal of the American Statistical Association (Theory and Methods), 201

Band Width Selection for High Dimensional Covariance Matrix Estimation

The banding estimator of Bickel and Levina (2008a) and its tapering version of Cai, Zhang and Zhou (2010), are important high dimensional covariance estimators. Both estimators require choosing a band width parameter. We propose a band width selector for the banding covariance estimator by minimizing an empirical estimate of the expected squared Frobenius norms of the estimation error matrix. The ratio consistency of the band width selector to the underlying band width is established. We also provide a lower bound for the coverage probability of the underlying band width being contained in an interval around the band width estimate. Extensions to the band width selection for the tapering estimator and threshold level selection for the thresholding covariance estimator are made. Numerical simulations and a case study on sonar spectrum data are conducted to confirm and demonstrate the proposed band width and threshold estimation approaches

Statistical Inference For High-Dimensional Linear Models

High-dimensional linear models play an important role in the analysis of modern data sets. Although the estimation problem has been well understood, there is still a paucity of methods and theories on the inference problem for high-dimensional linear models. This thesis focuses on statistical inference for high-dimensional linear models and consists of the following three parts. 1. The first part of the thesis considers confidence intervals for linear functionals in high-dimensional linear regression. We first establish the convergence rates of the minimax expected length for confidence intervals. Furthermore, we investigate the problem of adaptation to sparsity for the construction of confidence intervals and identify the regimes in which it is possible to construct adaptive confidence intervals. 2. In the second part of the thesis, we consider point and interval estimation of the $\ell_q$ loss of a given estimator in high-dimensional linear regression. For the class of rate-optimal estimators, we establish the minimax rates for estimating their $\ell_{q}$ losses, the minimax expected length of confidence intervals for their $\ell_{q}$ losses and the possibility of adaptivity of confidence intervals for their $\ell_q$ losses. 3. In the third part of the thesis, we consider the problem in the framework of high-dimensional instrumental variable regression and construct confidence intervals for the treatment effect in the presence of possibly invalid instrumental variables. We develop a novel selection procedure, Two-Stage Hard Thresholding (TSHT) to select valid instrumental variables and construct honest confidence intervals for the treatment effect using the selected instrumental variables

Robust high dimensional m-test using regularized geometric median covariance

Notes-System ETD not approved Abstract because Matrik/Symbol. Warning This item is temporarily locked to allow your modifications only