195 research outputs found
Nonparametric Independence Screening in Sparse Ultra-High Dimensional Varying Coefficient Models
The varying-coefficient model is an important nonparametric statistical model
that allows us to examine how the effects of covariates vary with exposure
variables. When the number of covariates is big, the issue of variable
selection arrives. In this paper, we propose and investigate marginal
nonparametric screening methods to screen variables in ultra-high dimensional
sparse varying-coefficient models. The proposed nonparametric independence
screening (NIS) selects variables by ranking a measure of the nonparametric
marginal contributions of each covariate given the exposure variable. The sure
independent screening property is established under some mild technical
conditions when the dimensionality is of nonpolynomial order, and the
dimensionality reduction of NIS is quantified. To enhance practical utility and
the finite sample performance, two data-driven iterative NIS methods are
proposed for selecting thresholding parameters and variables: conditional
permutation and greedy methods, resulting in Conditional-INIS and Greedy-INIS.
The effectiveness and flexibility of the proposed methods are further
illustrated by simulation studies and real data applications
Robust sure independence screening for nonpolynomial dimensional generalized linear models
We consider the problem of variable screening in
ultra-high-dimensional generalized linear models
(GLMs) of nonpolynomial orders. Since the popular
SIS approach is extremely unstable in the presence
of contamination and noise, we discuss a new robust
screening procedure based on the minimum density
power divergence estimator (MDPDE) of the marginal
regression coefficients. Our proposed screening procedure performs well under pure and contaminated data
scenarios. We provide a theoretical motivation for the
use of marginal MDPDEs for variable screening from
both population as well as sample aspects; in particular, we prove that the marginal MDPDEs are uniformly
consistent leading to the sure screening property of
our proposed algorithm. Finally, we propose an appropriate MDPDE-based extension for robust conditional
screening in GLMs along with the derivation of its sure
screening property. Our proposed methods are illustrated through extensive numerical studies along with
an interesting real data application
Generalization in Deep Learning
This paper provides theoretical insights into why and how deep learning can
generalize well, despite its large capacity, complexity, possible algorithmic
instability, nonrobustness, and sharp minima, responding to an open question in
the literature. We also discuss approaches to provide non-vacuous
generalization guarantees for deep learning. Based on theoretical observations,
we propose new open problems and discuss the limitations of our results.Comment: To appear in Mathematics of Deep Learning, Cambridge University
Press. All previous results remain unchange
- …