64,163 research outputs found
-Penalization in Functional Linear Regression with Subgaussian Design
We study functional regression with random subgaussian design and real-valued
response. The focus is on the problems in which the regression function can be
well approximated by a functional linear model with the slope function being
"sparse" in the sense that it can be represented as a sum of a small number of
well separated "spikes". This can be viewed as an extension of now classical
sparse estimation problems to the case of infinite dictionaries. We study an
estimator of the regression function based on penalized empirical risk
minimization with quadratic loss and the complexity penalty defined in terms of
-norm (a continuous version of LASSO). The main goal is to introduce
several important parameters characterizing sparsity in this class of problems
and to prove sharp oracle inequalities showing how the -error of the
continuous LASSO estimator depends on the underlying sparsity of the problem
Functional linear regression analysis for longitudinal data
We propose nonparametric methods for functional linear regression which are
designed for sparse longitudinal data, where both the predictor and response
are functions of a covariate such as time. Predictor and response processes
have smooth random trajectories, and the data consist of a small number of
noisy repeated measurements made at irregular times for a sample of subjects.
In longitudinal studies, the number of repeated measurements per subject is
often small and may be modeled as a discrete random number and, accordingly,
only a finite and asymptotically nonincreasing number of measurements are
available for each subject or experimental unit. We propose a functional
regression approach for this situation, using functional principal component
analysis, where we estimate the functional principal component scores through
conditional expectations. This allows the prediction of an unobserved response
trajectory from sparse measurements of a predictor trajectory. The resulting
technique is flexible and allows for different patterns regarding the timing of
the measurements obtained for predictor and response trajectories. Asymptotic
properties for a sample of subjects are investigated under mild conditions,
as , and we obtain consistent estimation for the regression
function. Besides convergence results for the components of functional linear
regression, such as the regression parameter function, we construct asymptotic
pointwise confidence bands for the predicted trajectories. A functional
coefficient of determination as a measure of the variance explained by the
functional regression model is introduced, extending the standard to the
functional case. The proposed methods are illustrated with a simulation study,
longitudinal primary biliary liver cirrhosis data and an analysis of the
longitudinal relationship between blood pressure and body mass index.Comment: Published at http://dx.doi.org/10.1214/009053605000000660 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Sparsity meets correlation in Gaussian sequence model
We study estimation of an -sparse signal in the -dimensional Gaussian
sequence model with equicorrelated observations and derive the minimax rate. A
new phenomenon emerges from correlation, namely the rate scales with respect to
and exhibits a phase transition at . Correlation
is shown to be a blessing provided it is sufficiently strong, and the critical
correlation level exhibits a delicate dependence on the sparsity level. Due to
correlation, the minimax rate is driven by two subproblems: estimation of a
linear functional (the average of the signal) and estimation of the signal's
-dimensional projection onto the orthogonal subspace. The
high-dimensional projection is estimated via sparse regression and the linear
functional is cast as a robust location estimation problem. Existing robust
estimators turn out to be suboptimal, and we show a kernel mode estimator with
a widening bandwidth exploits the Gaussian character of the data to achieve the
optimal estimation rate
Optimal linear estimation under unknown nonlinear transform
Linear regression studies the problem of estimating a model parameter
, from observations
from linear model . We consider a significant
generalization in which the relationship between and is noisy, quantized to a single bit, potentially nonlinear,
noninvertible, as well as unknown. This model is known as the single-index
model in statistics, and, among other things, it represents a significant
generalization of one-bit compressed sensing. We propose a novel spectral-based
estimation procedure and show that we can recover in settings (i.e.,
classes of link function ) where previous algorithms fail. In general, our
algorithm requires only very mild restrictions on the (unknown) functional
relationship between and . We also
consider the high dimensional setting where is sparse ,and introduce
a two-stage nonconvex framework that addresses estimation challenges in high
dimensional regimes where . For a broad class of link functions
between and , we establish minimax
lower bounds that demonstrate the optimality of our estimators in both the
classical and high dimensional regimes.Comment: 25 pages, 3 figure
Functional Regression
Functional data analysis (FDA) involves the analysis of data whose ideal
units of observation are functions defined on some continuous domain, and the
observed data consist of a sample of functions taken from some population,
sampled on a discrete grid. Ramsay and Silverman's 1997 textbook sparked the
development of this field, which has accelerated in the past 10 years to become
one of the fastest growing areas of statistics, fueled by the growing number of
applications yielding this type of data. One unique characteristic of FDA is
the need to combine information both across and within functions, which Ramsay
and Silverman called replication and regularization, respectively. This article
will focus on functional regression, the area of FDA that has received the most
attention in applications and methodological development. First will be an
introduction to basis functions, key building blocks for regularization in
functional regression methods, followed by an overview of functional regression
methods, split into three types: [1] functional predictor regression
(scalar-on-function), [2] functional response regression (function-on-scalar)
and [3] function-on-function regression. For each, the role of replication and
regularization will be discussed and the methodological development described
in a roughly chronological manner, at times deviating from the historical
timeline to group together similar methods. The primary focus is on modeling
and methodology, highlighting the modeling structures that have been developed
and the various regularization approaches employed. At the end is a brief
discussion describing potential areas of future development in this field
Varying-coefficient functional linear regression
Functional linear regression analysis aims to model regression relations
which include a functional predictor. The analog of the regression parameter
vector or matrix in conventional multivariate or multiple-response linear
regression models is a regression parameter function in one or two arguments.
If, in addition, one has scalar predictors, as is often the case in
applications to longitudinal studies, the question arises how to incorporate
these into a functional regression model. We study a varying-coefficient
approach where the scalar covariates are modeled as additional arguments of the
regression parameter function. This extension of the functional linear
regression model is analogous to the extension of conventional linear
regression models to varying-coefficient models and shares its advantages, such
as increased flexibility; however, the details of this extension are more
challenging in the functional case. Our methodology combines smoothing methods
with regularization by truncation at a finite number of functional principal
components. A practical version is developed and is shown to perform better
than functional linear regression for longitudinal data. We investigate the
asymptotic properties of varying-coefficient functional linear regression and
establish consistency properties.Comment: Published in at http://dx.doi.org/10.3150/09-BEJ231 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
- …