Data-driven rate-optimal specification testing in regression models

We propose new data-driven smooth tests for a parametric regression function. The smoothing parameter is selected through a new criterion that favors a large smoothing parameter under the null hypothesis. The resulting test is adaptive rate-optimal and consistent against Pitman local alternatives approaching the parametric model at a rate arbitrarily close to 1/\sqrtn. Asymptotic critical values come from the standard normal distribution and the bootstrap can be used in small samples. A general formalization allows one to consider a large class of linear smoothing methods, which can be tailored for detection of additive alternatives.Comment: Published at http://dx.doi.org/10.1214/009053604000001200 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

DATA-DRIVEN RATE-OPTIMAL SPECIFICATION TESTING IN REGRESSION MODELS

We propose new data-driven smooth tests for a parametric regression function. The smoothing parameter is selected through a new criterion that favors a large smoothing parameter under the null hypothesis. The resulting test is adaptive rate-optimal and consistent against Pitman local alternatives approaching the parametric model at a rate arbitrarily close to 1/\sqrt{n}. Asymptotic critical values come from the standard normal distribution and bootstrap can be used in small samples. A general formalization allows to consider a large class of linear smoothing methods, which can be tailored for detection of additive alternatives.Hypothesis testing, nonparametric adaptive tests, selection methods

Minimax testing of a composite null hypothesis defined via a quadratic functional in the model of regression

We consider the problem of testing a particular type of composite null hypothesis under a nonparametric multivariate regression model. For a given quadratic functional $Q$, the null hypothesis states that the regression function $f$ satisfies the constraint $Q[f]=0$, while the alternative corresponds to the functions for which $Q[f]$ is bounded away from zero. On the one hand, we provide minimax rates of testing and the exact separation constants, along with a sharp-optimal testing procedure, for diagonal and nonnegative quadratic functionals. We consider smoothness classes of ellipsoidal form and check that our conditions are fulfilled in the particular case of ellipsoids corresponding to anisotropic Sobolev classes. In this case, we present a closed form of the minimax rate and the separation constant. On the other hand, minimax rates for quadratic functionals which are neither positive nor negative makes appear two different regimes: "regular" and "irregular". In the "regular" case, the minimax rate is equal to $n^{-1/4}$ while in the "irregular" case, the rate depends on the smoothness class and is slower than in the "regular" case. We apply this to the issue of testing the equality of norms of two functions observed in noisy environments

An Improved Bernstein-type Inequality for C-Mixing-type Processes and Its Application to Kernel Smoothing

There are many processes, particularly dynamic systems, that cannot be described as strong mixing processes. \citet{maume2006exponential} introduced a new mixing coefficient called C-mixing, which includes a large class of dynamic systems. Based on this, \citet{hang2017bernstein} obtained a Bernstein-type inequality for a geometric C-mixing process, which, modulo a logarithmic factor and some constants, coincides with the standard result for the iid case. In order to honor this pioneering work, we conduct follow-up research in this paper and obtain an improved result under more general preconditions. We allow for a weaker requirement for the semi-norm condition, fully non-stationarity, non-isotropic sampling behavior. Our result covers the case in which the index set of processes lies in $\mathbf{Z}^{d+}$ for any given positive integer $d$. Here $\mathbf{Z}^{d+}$ denotes the collection of all nonnegative integer-valued $d$-dimensional vector. This setting of index set takes both time and spatial data into consideration. For our application, we investigate the theoretical guarantee of multiple kernel-based nonparametric curve estimators for C-Mixing-type processes. More specifically we firstly obtain the $L^{\infty}$-convergence rate of the kernel density estimator and then discuss the attainability of optimality, which can also be regarded as an upate of the result of \citet{hang2018kernel}. Furthermore, we investigate the uniform convergence of the kernel-based estimators of the conditional mean and variance function in a heteroscedastic nonparametric regression model. Under a mild smoothing condition, these estimators are optimal. At last, we obtain the uniform convergence rate of conditional mode function

Statistical inference in high-dimensional matrix models

Matrix models are ubiquitous in modern statistics. For instance, they are used in finance to assess interdependence of assets, in genomics to impute missing data and in movie recommender systems to model the relationship between users and movie ratings. Typically such models are either high-dimensional, meaning that the number of parameters may exceed the number of data points by many orders of magnitudes, or nonparametric in the sense that the quantity of interest is an infinite dimensional operator. This leads to new algorithms and also to new theoretical phenomena that may occur when estimating a parameter of interest or functionals of it or when constructing confidence sets. In this thesis, we will exemplarily consider three such matrix models and develop statistical theory for them: Matrix completion, Principal Component Analysis (PCA) with Gaussian data and transition operators of Markov chains. \\ \\ We start with matrix completion and investigate the existence of adaptive confidence sets in the 'Bernoulli' and 'trace-regression' models. In the 'Bernoulli' model we show that adaptive confidence sets do not exist when the variance of the errors is unknown, whereas we give an explicit construction in the ’trace-regression’ model. Finally, in the known variance case, we show that adaptive confidence sets do also exist in the 'Bernoulli' model based on a testing argument. \\ \\ Next, we consider PCA in a Gaussian observation model with complexity measured by the effective rank, the reciprocal of the percentage of variance explained by the first principal component. We investigate estimation of linear functionals of eigenvectors and prove Berry-Essen type bounds. Due to the high-dimensionality of the problem we discover a new phenomenon: The plug-in estimator based on the sample eigenvector can have non-negligible bias and hence may be not $\sqrt{n}$-consistent anymore. We show how to de-bias this estimator, achieving $\sqrt{n}$-convergence rates, and prove exact matching minimax lower bounds. \\ \\ Finally, we consider nonparametric estimation of the transition operator of a Markov chain and its transition density. We assume that the singular values of the transition operator decay exponentially. For example, this assumption is fulfilled by discrete, low frequency observations of periodised, reversible stochastic differential equations. Using penalization techniques from low rank matrix estimation we develop a new algorithm and show improved convergence rates.Financial support of ERC grant UQMSI/647812 and EPSRC grant EP/L016516/

Régression sur variable fonctionnelle: Estimation, Tests de structure et Applications

Au cours des dernières années, la branche de la statistique consacrée à l'étude de variables fonctionnelles a connu un réel essor tant en terme de développements théoriques que de diversification des domaines d'application. Nous nous intéressons plus particulièrement dans ce mémoire à des modèles de régression dans lesquels la variable réponse est réelle tandis que la variable explicative est fonctionnelle, c'est à dire à valeurs dans un espace de dimension infinie. Les résultats que nous énonçons sont liés aux propriétés asymptotiques de l'estimateur à noyau généralisé au cas d'une variable explicative fonctionnelle. Nous supposons pour commencer que l'échantillon que nous étudions est constitué de variables alpha-mélangeantes et que le modèle de régression est de nature non-paramétrique. Nous établissons la normalité asymptotique de notre estimateur et donnons l'expression explicite des termes asymptotiquement dominants du biais et de la variance. Une conséquence directe de ce résultat est la construction d'intervalles de confiance asymptotiques ponctuels dont nous étudions les propriétés aux travers de simulations et que nous appliquons sur des données liées à l'étude du courant marin El Niño. On établit également à partir du résultat de normalité asymptotique et d'un résultat d'uniforme intégrabilité l'expression explicite des termes asymptotiquement dominants des moments centrés et des erreurs Lp de notre estimateur. Nous considérons ensuite le problème des tests de structure en régression sur variable fonctionnelle et supposons maintenant que l'échantillon est composé de variables indépendantes. Nous construisons une statistique de test basée sur la comparaison de l'estimateur à noyau et d'un estimateur plus particulier dépendant de l'hypothèse nulle à tester. Nous obtenons la normalité asymptotique de notre statistique de test sous l'hypothèse nulle ainsi que sa divergence sous l'alternative. Les conditions générales sous lesquelles notre résultat est établi permettent l'utilisation de notre statistique pour construire des tests de structure innovants permettant de tester si l'opérateur de régression est de forme linéaire, à indice simple, ... Différentes procédures de rééchantillonnage sont proposées et comparées au travers de diverses simulations. Nos méthodes sont enfin appliquées dans le cadre de tests de non effet à deux jeux de données spectrométriques.Functional data analysis is a typical issue in modern statistics. During the last years, many papers have been devoted to theoretical results or applied studies on models involving functional data. In this manuscript, we focus on regression models where a real response variable depends on a functional random variable taking its values in an infinite dimensional space. We state various kinds of results linked with the asymptotic properties of the nonparametric kernel estimator generalized to the case of a functional explanatory variable. We firstly assume that the dataset under study is composed of strong-mixing variables and focus on a nonparametric regression model. A first result gives asymptotic normality of the kernel estimator with explicit expressions of the asymptotic dominant bias and variance terms. On one hand, we propose from this result a way to construct asymptotic pointwise confidence bands and study their properties on simulation studies and apply our method on a dataset dealing with El Niño phenomenon. On the other hand, we decline from both asymptotical normality and uniform integrability results the explicit expressions of the asymptotic dominant terms of centered moments and Lp errors of the kernel estimator. We now assume that the dataset is composed of independent random variables and focus on structural testing procedures in regression on functional data. We construct a test statistic based on the comparison between the nonparametric kernel estimator and a particular one that must be chosen in accordance to the null hypothesis we want to test. We then state both asymptotic normality of our test statistic under the null hypothesis and its divergence under the alternative. Moreover, we prove our result under general conditions that enable to use our approach to construct innovative structural tests allowing to check if the regression operator is linear, single index, ... Different bootstrap procedures are proposed and compared through various simulation studies. Finally, we focus on no effect tests and apply our testing procedures on two spectrometric datasets