15,850 research outputs found
On the Sample Complexity of Subspace Learning
A large number of algorithms in machine learning, from principal component
analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral
embedding and support estimation methods, rely on estimating a linear subspace
from samples. In this paper we introduce a general formulation of this problem
and derive novel learning error estimates. Our results rely on natural
assumptions on the spectral properties of the covariance operator associated to
the data distribu- tion, and hold for a wide class of metrics between
subspaces. As special cases, we discuss sharp error estimates for the
reconstruction properties of PCA and spectral support estimation. Key to our
analysis is an operator theoretic approach that has broad applicability to
spectral learning methods.Comment: Extendend Version of conference pape
Robust Geometry Estimation using the Generalized Voronoi Covariance Measure
The Voronoi Covariance Measure of a compact set K of R^d is a tensor-valued
measure that encodes geometric information on K and which is known to be
resilient to Hausdorff noise but sensitive to outliers. In this article, we
generalize this notion to any distance-like function delta and define the
delta-VCM. We show that the delta-VCM is resilient to Hausdorff noise and to
outliers, thus providing a tool to estimate robustly normals from a point cloud
approximation. We present experiments showing the robustness of our approach
for normal and curvature estimation and sharp feature detection
Kernel methods for detecting coherent structures in dynamical data
We illustrate relationships between classical kernel-based dimensionality
reduction techniques and eigendecompositions of empirical estimates of
reproducing kernel Hilbert space (RKHS) operators associated with dynamical
systems. In particular, we show that kernel canonical correlation analysis
(CCA) can be interpreted in terms of kernel transfer operators and that it can
be obtained by optimizing the variational approach for Markov processes (VAMP)
score. As a result, we show that coherent sets of particle trajectories can be
computed by kernel CCA. We demonstrate the efficiency of this approach with
several examples, namely the well-known Bickley jet, ocean drifter data, and a
molecular dynamics problem with a time-dependent potential. Finally, we propose
a straightforward generalization of dynamic mode decomposition (DMD) called
coherent mode decomposition (CMD). Our results provide a generic machine
learning approach to the computation of coherent sets with an objective score
that can be used for cross-validation and the comparison of different methods
Approximate Kernel PCA Using Random Features: Computational vs. Statistical Trade-off
Kernel methods are powerful learning methodologies that provide a simple way
to construct nonlinear algorithms from linear ones. Despite their popularity,
they suffer from poor scalability in big data scenarios. Various approximation
methods, including random feature approximation have been proposed to alleviate
the problem. However, the statistical consistency of most of these approximate
kernel methods is not well understood except for kernel ridge regression
wherein it has been shown that the random feature approximation is not only
computationally efficient but also statistically consistent with a minimax
optimal rate of convergence. In this paper, we investigate the efficacy of
random feature approximation in the context of kernel principal component
analysis (KPCA) by studying the trade-off between computational and statistical
behaviors of approximate KPCA. We show that the approximate KPCA is both
computationally and statistically efficient compared to KPCA in terms of the
error associated with reconstructing a kernel function based on its projection
onto the corresponding eigenspaces. Depending on the eigenvalue decay behavior
of the covariance operator, we show that only features (polynomial
decay) or features (exponential decay) are needed to match the
statistical performance of KPCA. We also investigate their statistical
behaviors in terms of the convergence of corresponding eigenspaces wherein we
show that only features are required to match the performance of
KPCA and if fewer than features are used, then approximate KPCA has
a worse statistical behavior than that of KPCA.Comment: 46 page
- …