Feature Extraction in Signal Regression: A Boosting Technique for Functional Data Regression

Main objectives of feature extraction in signal regression are the improvement of accuracy of prediction on future data and identification of relevant parts of the signal. A feature extraction procedure is proposed that uses boosting techniques to select the relevant parts of the signal. The proposed blockwise boosting procedure simultaneously selects intervals in the signal’s domain and estimates the effect on the response. The blocks that are defined explicitly use the underlying metric of the signal. It is demonstrated in simulation studies and for real-world data that the proposed approach competes well with procedures like PLS, P-spline signal regression and functional data regression. The paper is a preprint of an article published in the Journal of Computational and Graphical Statistics. Please use the journal version for citation

Implicit Copulas from Bayesian Regularized Regression Smoothers

We show how to extract the implicit copula of a response vector from a Bayesian regularized regression smoother with Gaussian disturbances. The copula can be used to compare smoothers that employ different shrinkage priors and function bases. We illustrate with three popular choices of shrinkage priors --- a pairwise prior, the horseshoe prior and a g prior augmented with a point mass as employed for Bayesian variable selection --- and both univariate and multivariate function bases. The implicit copulas are high-dimensional, have flexible dependence structures that are far from that of a Gaussian copula, and are unavailable in closed form. However, we show how they can be evaluated by first constructing a Gaussian copula conditional on the regularization parameters, and then integrating over these. Combined with non-parametric margins the regularized smoothers can be used to model the distribution of non-Gaussian univariate responses conditional on the covariates. Efficient Markov chain Monte Carlo schemes for evaluating the copula are given for this case. Using both simulated and real data, we show how such copula smoothing models can improve the quality of resulting function estimates and predictive distributions

Multivariate varying coefficient model for functional responses

Motivated by recent work studying massive imaging data in the neuroimaging literature, we propose multivariate varying coefficient models (MVCM) for modeling the relation between multiple functional responses and a set of covariates. We develop several statistical inference procedures for MVCM and systematically study their theoretical properties. We first establish the weak convergence of the local linear estimate of coefficient functions, as well as its asymptotic bias and variance, and then we derive asymptotic bias and mean integrated squared error of smoothed individual functions and their uniform convergence rate. We establish the uniform convergence rate of the estimated covariance function of the individual functions and its associated eigenvalue and eigenfunctions. We propose a global test for linear hypotheses of varying coefficient functions, and derive its asymptotic distribution under the null hypothesis. We also propose a simultaneous confidence band for each individual effect curve. We conduct Monte Carlo simulation to examine the finite-sample performance of the proposed procedures. We apply MVCM to investigate the development of white matter diffusivities along the genu tract of the corpus callosum in a clinical study of neurodevelopment.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1045 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Empirical Bayesian Smoothing Splines for Signals with Correlated Errors: Methods and Applications

Smoothing splines is a well stablished method in non-parametric statistics,  although the selection of the smoothness degree of the regression function is rarely addressed and, instead, a two times differentiable function, i.e.  cubic smoothing spline, is assumed. For a general regression function there is no known method that can identify the smoothness degree under the presence of correlated errors. This apparent disregard in the literature can be justified because the condition number of the solution increases with the smoothness degree of the function, turning the estimation unstable. In this thesis we introduce  an exact expression for the Demmler-Reinsch basis constructed as the solution of an ordinary differential equation, so that the estimation can be carried out for an arbitrary smoothness degree, and under the presence of correlated errors, without affecting the condition number of the solution. We provide asymptotic properties of the proposed estimators and conduct simulation experiments to study their finite sample properties. We expect this new approach to have a direct impact on related methods that use smoothing splines as a building block. In this direction, we present extensions of the method to signal extraction and functional principal component analysis. The empirical relevance to our findings in these areas of statistics is shown in applications for agricultural economics and biophysics. R packages of the implementation of the developed methods are also provided.