44 research outputs found
Nonparametric inference in generalized functional linear models
We propose a roughness regularization approach in making nonparametric
inference for generalized functional linear models. In a reproducing kernel
Hilbert space framework, we construct asymptotically valid confidence intervals
for regression mean, prediction intervals for future response and various
statistical procedures for hypothesis testing. In particular, one procedure for
testing global behaviors of the slope function is adaptive to the smoothness of
the slope function and to the structure of the predictors. As a by-product, a
new type of Wilks phenomenon [Ann. Math. Stat. 9 (1938) 60-62; Ann. Statist. 29
(2001) 153-193] is discovered when testing the functional linear models.
Despite the generality, our inference procedures are easy to implement.
Numerical examples are provided to demonstrate the empirical advantages over
the competing methods. A collection of technical tools such as
integro-differential equation techniques [Trans. Amer. Math. Soc. (1927) 29
755-800; Trans. Amer. Math. Soc. (1928) 30 453-471; Trans. Amer. Math. Soc.
(1930) 32 860-868], Stein's method [Ann. Statist. 41 (2013) 2786-2819] [Stein,
Approximate Computation of Expectations (1986) IMS] and functional Bahadur
representation [Ann. Statist. 41 (2013) 2608-2638] are employed in this paper.Comment: Published at http://dx.doi.org/10.1214/15-AOS1322 in the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Joint asymptotics for semi-nonparametric regression models with partially linear structure
We consider a joint asymptotic framework for studying semi-nonparametric
regression models where (finite-dimensional) Euclidean parameters and
(infinite-dimensional) functional parameters are both of interest. The class of
models in consideration share a partially linear structure and are estimated in
two general contexts: (i) quasi-likelihood and (ii) true likelihood. We first
show that the Euclidean estimator and (pointwise) functional estimator, which
are re-scaled at different rates, jointly converge to a zero-mean Gaussian
vector. This weak convergence result reveals a surprising joint asymptotics
phenomenon: these two estimators are asymptotically independent. A major goal
of this paper is to gain first-hand insights into the above phenomenon.
Moreover, a likelihood ratio testing is proposed for a set of joint local
hypotheses, where a new version of the Wilks phenomenon [Ann. Math. Stat. 9
(1938) 60-62; Ann. Statist. 1 (2001) 153-193] is unveiled. A novel technical
tool, called a joint Bahadur representation, is developed for studying these
joint asymptotics results.Comment: Published at http://dx.doi.org/10.1214/15-AOS1313 in the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Computational Limits of A Distributed Algorithm For Smoothing Spline
In this paper, we explore statistical versus computational trade-off to
address a basic question in the application of a distributed algorithm: what is
the minimal computational cost in obtaining statistical optimality? In
smoothing spline setup, we observe a phase transition phenomenon for the number
of deployed machines that ends up being a simple proxy for computing cost.
Specifically, a sharp upper bound for the number of machines is established:
when the number is below this bound, statistical optimality (in terms of
nonparametric estimation or testing) is achievable; otherwise, statistical
optimality becomes impossible. These sharp bounds partly capture intrinsic
computational limits of the distributed algorithm considered in this paper, and
turn out to be fully determined by the smoothness of the regression function.
As a side remark, we argue that sample splitting may be viewed as an
alternative form of regularization, playing a similar role as smoothing
parameter.Comment: To Appear in Journal of Machine Learning Researc
Consistency of Bayesian Linear Model Selection With a Growing Number of Parameters
Linear models with a growing number of parameters have been widely used in
modern statistics. One important problem about this kind of model is the
variable selection issue. Bayesian approaches, which provide a stochastic
search of informative variables, have gained popularity. In this paper, we will
study the asymptotic properties related to Bayesian model selection when the
model dimension is growing with the sample size . We consider
and provide sufficient conditions under which: (1) with large probability, the
posterior probability of the true model (from which samples are drawn)
uniformly dominates the posterior probability of any incorrect models; and (2)
with large probability, the posterior probability of the true model converges
to one. Both (1) and (2) guarantee that the true model will be selected under a
Bayesian framework. We also demonstrate several situations when (1) holds but
(2) fails, which illustrates the difference between these two properties.
Simulated examples are provided to illustrate the main results