9 research outputs found
A One-Sample Test for Normality with Kernel Methods
We propose a new one-sample test for normality in a Reproducing Kernel
Hilbert Space (RKHS). Namely, we test the null-hypothesis of belonging to a
given family of Gaussian distributions. Hence our procedure may be applied
either to test data for normality or to test parameters (mean and covariance)
if data are assumed Gaussian. Our test is based on the same principle as the
MMD (Maximum Mean Discrepancy) which is usually used for two-sample tests such
as homogeneity or independence testing. Our method makes use of a special kind
of parametric bootstrap (typical of goodness-of-fit tests) which is
computationally more efficient than standard parametric bootstrap. Moreover, an
upper bound for the Type-II error highlights the dependence on influential
quantities. Experiments illustrate the practical improvement allowed by our
test in high-dimensional settings where common normality tests are known to
fail. We also consider an application to covariance rank selection through a
sequential procedure
Non-asymptotic Optimal Prediction Error for RKHS-based Partially Functional Linear Models
Under the framework of reproducing kernel Hilbert space (RKHS), we consider
the penalized least-squares of the partially functional linear models (PFLM),
whose predictor contains both functional and traditional multivariate part, and
the multivariate part allows a divergent number of parameters. From the
non-asymptotic point of view, we focus on the rate-optimal upper and lower
bounds of the prediction error. An exact upper bound for the excess prediction
risk is shown in a non-asymptotic form under a more general assumption known as
the effective dimension to the model, by which we also show the prediction
consistency when the number of multivariate covariates slightly increases
with the sample size . Our new finding implies a trade-off between the
number of non-functional predictors and the effective dimension of the kernel
principal components to ensure the prediction consistency in the
increasing-dimensional setting. The analysis in our proof hinges on the
spectral condition of the sandwich operator of the covariance operator and the
reproducing kernel, and on the concentration inequalities for the random
elements in Hilbert space. Finally, we derive the non-asymptotic minimax lower
bound under the regularity assumption of Kullback-Leibler divergence of the
models.Comment: 24 page
Variable Selection and Estimation in Multivariate Functional Linear Regression via the LASSO
In more and more applications, a quantity of interest may depend on several
covariates, with at least one of them infinite-dimensional (e.g. a curve). To
select the relevant covariates in this context, we propose an adaptation of the
Lasso method. Two estimation methods are defined. The first one consists in the
minimisation of a criterion inspired by classical Lasso inference under group
sparsity (Yuan and Lin, 2006; Lounici et al., 2011) on the whole multivariate
functional space H. The second one minimises the same criterion but on a
finite-dimensional subspace of H which dimension is chosen by a penalized
leasts-squares method base on the work of Barron et al. (1999). Sparsity-oracle
inequalities are proven in case of fixed or random design in our
infinite-dimensional context. To calculate the solutions of both criteria, we
propose a coordinate-wise descent algorithm, inspired by the glmnet algorithm
(Friedman et al., 2007). A numerical study on simulated and experimental
datasets illustrates the behavior of the estimators
VARIABLE SELECTION AND ESTIMATION IN MULTIVARIATE FUNCTIONAL LINEAR REGRESSION VIA THE LASSO
In more and more applications, a quantity of interest may depend on several covariates, with at least one of them infinite-dimensional (e.g. a curve). To select the relevant covariates in this context, we propose an adaptation of the Lasso method. Two estimation methods are defined. The first one consists in the minimisation of a criterion inspired by classical Lasso inference under group sparsity (Yuan and Lin, 2006; Lounici et al., 2011) on the whole multivariate functional space H. The second one minimises the same criterion but on a finite-dimensional subspace of H which dimension is chosen by a penalized leasts-squares method base on the work of Barron et al. (1999). Sparsity- oracle inequalities are proven in case of fixed or random design in our infinite-dimensional context. To calculate the solutions of both criteria, we propose a coordinate-wise descent algorithm, inspired by the glmnet algorithm (Friedman et al., 2007). A numerical study on simulated and experimental datasets illustrates the behavior of the estimators
Analyzing the discrepancy principle for kernelized spectral filter learning algorithms
We investigate the construction of early stopping rules in the nonparametric
regression problem where iterative learning algorithms are used and the optimal
iteration number is unknown. More precisely, we study the discrepancy
principle, as well as modifications based on smoothed residuals, for kernelized
spectral filter learning algorithms including gradient descent. Our main
theoretical bounds are oracle inequalities established for the empirical
estimation error (fixed design), and for the prediction error (random design).
From these finite-sample bounds it follows that the classical discrepancy
principle is statistically adaptive for slow rates occurring in the hard
learning scenario, while the smoothed discrepancy principles are adaptive over
ranges of faster rates (resp. higher smoothness parameters). Our approach
relies on deviation inequalities for the stopping rules in the fixed design
setting, combined with change-of-norm arguments to deal with the random design
setting.Comment: 68 pages, 4 figure
Non-asymptotic adaptive prediction in functional linear models
Functional linear regression has recently attracted considerable interest. Many works focus on asymptotic inference. In this paper we consider in a non asymptotic framework a simple estimation procedure based on functional Principal Regression. It revolves in the minimization of a least square contrast coupled with a classical projection on the space spanned by the m first empirical eigenvectors of the covariance operator of the functional sample. The novelty of our approach is to select automatically the crucial dimension m by minimization of a penalized least square contrast. Our method is based on model selection tools. Yet, since this kind of methods consists usually in projecting onto known non-random spaces, we need to adapt it to empirical eigenbasis made of data-dependent - hence random - vectors. The resulting estimator is fully adaptive and is shown to verify an oracle inequality for the risk associated to the prediction error and to attain optimal minimax rates of convergence over a certain class of ellipsoids. Our strategy of model selection is finally compared numerically with cross-validation