Adaptive orthogonal series estimation in additive stochastic regression models

In this paper, we consider additive stochastic nonparametric regression models. By approximating the nonparametric components by a class of orthogonal series and using a generalized cross-validation criterion, an adaptive and simultaneous estimation procedure for the nonparametric components is constructed. We illustrate the adaptive and simultaneous estimation procedure by a number of simulated and real examples.Adaptive estimation; additive model; dependent process; mixing condition; nonlinear time series; nonparametric regression; orthogonal series; strict stationarity; truncation parameter

Nonparametric ridge estimation

We study the problem of estimating the ridges of a density function. Ridge estimation is an extension of mode finding and is useful for understanding the structure of a density. It can also be used to find hidden structure in point cloud data. We show that, under mild regularity conditions, the ridges of the kernel density estimator consistently estimate the ridges of the true density. When the data are noisy measurements of a manifold, we show that the ridges are close and topologically similar to the hidden manifold. To find the estimated ridges in practice, we adapt the modified mean-shift algorithm proposed by Ozertem and Erdogmus [J. Mach. Learn. Res. 12 (2011) 1249-1286]. Some numerical experiments verify that the algorithm is accurate.Comment: Published in at http://dx.doi.org/10.1214/14-AOS1218 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Characterization of the asymptotic distribution of semiparametric M-estimators

This paper develops a concrete formula for the asymptotic distribution of two-step, possibly non-smooth semiparametric M-estimators under general misspecification. Our regularity conditions are relatively straightforward to verify and also weaker than those available in the literature. The first-stage nonparametric estimation may depend on finite dimensional parameters. We characterize: (1) conditions under which the first-stage estimation of nonparametric components do not affect the asymptotic distribution, (2) conditions under which the asymptotic distribution is affected by the derivatives of the first-stage nonparametric estimator with respect to the finite-dimensional parameters, and (3) conditions under which one can allow non-smooth objective functions. Our framework is illustrated by applying it to three examples: (1) profiled estimation of a single index quantile regression model, (2) semiparametric least squares estimation under model misspecification, and (3) a smoothed matching estimator. © 2010 Elsevier B.V. All rights reserved

Consistent estimation of the filtering and marginal smoothing distributions in nonparametric hidden Markov models

In this paper, we consider the filtering and smoothing recursions in nonparametric finite state space hidden Markov models (HMMs) when the parameters of the model are unknown and replaced by estimators. We provide an explicit and time uniform control of the filtering and smoothing errors in total variation norm as a function of the parameter estimation errors. We prove that the risk for the filtering and smoothing errors may be uniformly upper bounded by the risk of the estimators. It has been proved very recently that statistical inference for finite state space nonparametric HMMs is possible. We study how the recent spectral methods developed in the parametric setting may be extended to the nonparametric framework and we give explicit upper bounds for the L2-risk of the nonparametric spectral estimators. When the observation space is compact, this provides explicit rates for the filtering and smoothing errors in total variation norm. The performance of the spectral method is assessed with simulated data for both the estimation of the (nonparametric) conditional distribution of the observations and the estimation of the marginal smoothing distributions.Comment: 27 pages, 2 figures. arXiv admin note: text overlap with arXiv:1501.0478

Semi-parametric regression: Efficiency gains from modeling the nonparametric part

It is widely admitted that structured nonparametric modeling that circumvents the curse of dimensionality is important in nonparametric estimation. In this paper we show that the same holds for semi-parametric estimation. We argue that estimation of the parametric component of a semi-parametric model can be improved essentially when more structure is put into the nonparametric part of the model. We illustrate this for the partially linear model, and investigate efficiency gains when the nonparametric part of the model has an additive structure. We present the semi-parametric Fisher information bound for estimating the parametric part of the partially linear additive model and provide semi-parametric efficient estimators for which we use a smooth backfitting technique to deal with the additive nonparametric part. We also present the finite sample performances of the proposed estimators and analyze Boston housing data as an illustration.Comment: Published in at http://dx.doi.org/10.3150/10-BEJ296 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Estimation of semiparametric stochastic frontiers under shape constraints with application to pollution generating technologies

A number of studies have explored the semi- and nonparametric estimation of stochastic frontier models by using kernel regression or other nonparametric smoothing techniques. In contrast to popular deterministic nonparametric estimators, these approaches do not allow one to impose any shape constraints (or regularity conditions) on the frontier function. On the other hand, as many of the previous techniques are based on the nonparametric estimation of the frontier function, the convergence rate of frontier estimators can be sensitive to the number of inputs, which is generally known as “the curse of dimensionality” problem. This paper proposes a new semiparametric approach for stochastic frontier estimation that avoids the curse of dimensionality and allows one to impose shape constraints on the frontier function. Our approach is based on the singleindex model and applies both single-index estimation techniques and shape-constrained nonparametric least squares. In addition to production frontier and technical efficiency estimation, we show how the technique can be used to estimate pollution generating technologies. The new approach is illustrated by an empirical application to the environmental adjusted performance evaluation of U.S. coal-fired electric power plants.stochastic frontier analysis (SFA), nonparametric least squares, single-index model, sliced inverse regression, monotone rank correlation estimator, environmental efficiency

Yield Curve Estimation by Kernel Smoothing Methods

We introduce a new method for the estimation of discount functions, yield curves and forward curves from government issued coupon bonds. Our approach is nonparametric and does not assume a particular functional form for the discount function although we do show how to impose various restrictions in the estimation. Our method is based on kernel smoothing and is defined as the minimum of some localized population moment condition. The solution to the sample problem is not explicit and our estimation procedure is iterative, rather like the backfitting method of estimating additive nonparametric models. We establish the asymptotic normality of our methods using the asymptotic representation of our estimator as an infinite series with declining coefficients. The rate of convergence is standard for one dimensional nonparametric regression. We investigate the finite sample performance of our method, in comparison with other well-established methods, in a small simulation experiment.Coupon bonds, kernel estimation, Hilbert space, nonparametric regression, term structure estimation, yield curve, zero coupon.