Robust local polynomial regression using M-estimator with adaptive bandwidth

In this paper, a new method for robust local polynomial regression (LPR) using M-estimator with adaptive bandwidth is proposed. This is motivated by the limitation of traditional LPR in detecting and removing impulsive noise or outlies. By using M-estimation technique and the intersection of confidence intervals (ICI) rule for choosing an adaptive local bandwidth, a robust LPR algorithm is developed. Simulation results show that the new M-estimation-based LPR performs considerably better than the traditional LS-based method in removing the impulsive noise as well as preserving the jump discontinuities, which are frequently found in image and video processing.published_or_final_versio

Adaptive Estimation of the Regression Discontinuity Model

In order to reduce the finite sample bias and improve the rate of convergence, local polynomial estimators have been introduced into the econometric literature to estimate the regression discontinuity model. In this paper, we show that, when the degree of smoothness is known, the local polynomial estimator achieves the optimal rate of convergence within the Hölder smoothness class. However, when the degree of smoothness is not known, the local polynomial estimator may actually inflate the finite sample bias and reduce the rate of convergence. We propose an adaptive version of the local polynomial estimator which selects both the bandwidth and the polynomial order adaptively and show that the adaptive estimator achieves the optimal rate of convergence up to a logarithm factor without knowing the degree of smoothness. Simulation results show that the finite sample performance of the locally cross-validated adaptive estimator is robust to the parameter combinations and data generating processes, reflecting the adaptive nature of the estimator. The root mean squared error of the adaptive estimator compares favorably to local polynomial estimators in the Monte Carlo experiments.Adaptive estimator, local cross validation, local polynomial, minimax rate, optimal bandwidth, optimal smoothness parameter

Orthogonalized smoothing for rescaled spike and slab models

Rescaled spike and slab models are a new Bayesian variable selection method for linear regression models. In high dimensional orthogonal settings such models have been shown to possess optimal model selection properties. We review background theory and discuss applications of rescaled spike and slab models to prediction problems involving orthogonal polynomials. We first consider global smoothing and discuss potential weaknesses. Some of these deficiencies are remedied by using local regression. The local regression approach relies on an intimate connection between local weighted regression and weighted generalized ridge regression. An important implication is that one can trace the effective degrees of freedom of a curve as a way to visualize and classify curvature. Several motivating examples are presented.Comment: Published in at http://dx.doi.org/10.1214/074921708000000192 the IMS Collections (http://www.imstat.org/publications/imscollections.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

Bandwidth selection in kernel empirical risk minimization via the gradient

In this paper, we deal with the data-driven selection of multidimensional and possibly anisotropic bandwidths in the general framework of kernel empirical risk minimization. We propose a universal selection rule, which leads to optimal adaptive results in a large variety of statistical models such as nonparametric robust regression and statistical learning with errors in variables. These results are stated in the context of smooth loss functions, where the gradient of the risk appears as a good criterion to measure the performance of our estimators. The selection rule consists of a comparison of gradient empirical risks. It can be viewed as a nontrivial improvement of the so-called Goldenshluger-Lepski method to nonlinear estimators. Furthermore, one main advantage of our selection rule is the nondependency on the Hessian matrix of the risk, usually involved in standard adaptive procedures.Comment: Published at http://dx.doi.org/10.1214/15-AOS1318 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

On weighted local fitting and its relation to the Horvitz-Thompson estimator

Weighting is a largely used concept in many fields of statistics and has frequently caused controversies on its justification and profit. In this paper, we analyze a weighted version of the well-known local polynomial regression estimators, derive their asymptotic bias and variance, and find that the conflict between the asymptotically optimal weighting scheme and the practical requirements has a surprising counterpart in sampling theory, leading us back to the discussion on Basu's (1971) elephants

Optimal inference in a class of regression models

We consider the problem of constructing confidence intervals (CIs) for a linear functional of a regression function, such as its value at a point, the regression discontinuity parameter, or a regression coefficient in a linear or partly linear regression. Our main assumption is that the regression function is known to lie in a convex function class, which covers most smoothness and/or shape assumptions used in econometrics. We derive finite-sample optimal CIs and sharp efficiency bounds under normal errors with known variance. We show that these results translate to uniform (over the function class) asymptotic results when the error distribution is not known. When the function class is centrosymmetric, these efficiency bounds imply that minimax CIs are close to efficient at smooth regression functions. This implies, in particular, that it is impossible to form CIs that are tighter using data-dependent tuning parameters, and maintain coverage over the whole function class. We specialize our results to inference on the regression discontinuity parameter, and illustrate them in simulations and an empirical application.Comment: 39 pages plus supplementary material