844 research outputs found
Kernel dimension reduction in regression
We present a new methodology for sufficient dimension reduction (SDR). Our
methodology derives directly from the formulation of SDR in terms of the
conditional independence of the covariate from the response , given the
projection of on the central subspace [cf. J. Amer. Statist. Assoc. 86
(1991) 316--342 and Regression Graphics (1998) Wiley]. We show that this
conditional independence assertion can be characterized in terms of conditional
covariance operators on reproducing kernel Hilbert spaces and we show how this
characterization leads to an -estimator for the central subspace. The
resulting estimator is shown to be consistent under weak conditions; in
particular, we do not have to impose linearity or ellipticity conditions of the
kinds that are generally invoked for SDR methods. We also present empirical
results showing that the new methodology is competitive in practice.Comment: Published in at http://dx.doi.org/10.1214/08-AOS637 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Kernel Multivariate Analysis Framework for Supervised Subspace Learning: A Tutorial on Linear and Kernel Multivariate Methods
Feature extraction and dimensionality reduction are important tasks in many
fields of science dealing with signal processing and analysis. The relevance of
these techniques is increasing as current sensory devices are developed with
ever higher resolution, and problems involving multimodal data sources become
more common. A plethora of feature extraction methods are available in the
literature collectively grouped under the field of Multivariate Analysis (MVA).
This paper provides a uniform treatment of several methods: Principal Component
Analysis (PCA), Partial Least Squares (PLS), Canonical Correlation Analysis
(CCA) and Orthonormalized PLS (OPLS), as well as their non-linear extensions
derived by means of the theory of reproducing kernel Hilbert spaces. We also
review their connections to other methods for classification and statistical
dependence estimation, and introduce some recent developments to deal with the
extreme cases of large-scale and low-sized problems. To illustrate the wide
applicability of these methods in both classification and regression problems,
we analyze their performance in a benchmark of publicly available data sets,
and pay special attention to specific real applications involving audio
processing for music genre prediction and hyperspectral satellite images for
Earth and climate monitoring
The representer theorem for Hilbert spaces: a necessary and sufficient condition
A family of regularization functionals is said to admit a linear representer
theorem if every member of the family admits minimizers that lie in a fixed
finite dimensional subspace. A recent characterization states that a general
class of regularization functionals with differentiable regularizer admits a
linear representer theorem if and only if the regularization term is a
non-decreasing function of the norm. In this report, we improve over such
result by replacing the differentiability assumption with lower semi-continuity
and deriving a proof that is independent of the dimensionality of the space
- …