117,650 research outputs found
Classification via Incoherent Subspaces
This article presents a new classification framework that can extract
individual features per class. The scheme is based on a model of incoherent
subspaces, each one associated to one class, and a model on how the elements in
a class are represented in this subspace. After the theoretical analysis an
alternate projection algorithm to find such a collection is developed. The
classification performance and speed of the proposed method is tested on the AR
and YaleB databases and compared to that of Fisher's LDA and a recent approach
based on on minimisation. Finally connections of the presented scheme
to already existing work are discussed and possible ways of extensions are
pointed out.Comment: 22 pages, 2 figures, 4 table
Factor Analysis of Moving Average Processes
The paper considers an extension of factor analysis to moving average
processes. The problem is formulated as a rank minimization of a suitable
spectral density. It is shown that it can be adequately approximated via a
trace norm convex relaxation
Minimax rank estimation for subspace tracking
Rank estimation is a classical model order selection problem that arises in a
variety of important statistical signal and array processing systems, yet is
addressed relatively infrequently in the extant literature. Here we present
sample covariance asymptotics stemming from random matrix theory, and bring
them to bear on the problem of optimal rank estimation in the context of the
standard array observation model with additive white Gaussian noise. The most
significant of these results demonstrates the existence of a phase transition
threshold, below which eigenvalues and associated eigenvectors of the sample
covariance fail to provide any information on population eigenvalues. We then
develop a decision-theoretic rank estimation framework that leads to a simple
ordered selection rule based on thresholding; in contrast to competing
approaches, however, it admits asymptotic minimax optimality and is free of
tuning parameters. We analyze the asymptotic performance of our rank selection
procedure and conclude with a brief simulation study demonstrating its
practical efficacy in the context of subspace tracking.Comment: 10 pages, 4 figures; final versio
OBOE: Collaborative Filtering for AutoML Model Selection
Algorithm selection and hyperparameter tuning remain two of the most
challenging tasks in machine learning. Automated machine learning (AutoML)
seeks to automate these tasks to enable widespread use of machine learning by
non-experts. This paper introduces OBOE, a collaborative filtering method for
time-constrained model selection and hyperparameter tuning. OBOE forms a matrix
of the cross-validated errors of a large number of supervised learning models
(algorithms together with hyperparameters) on a large number of datasets, and
fits a low rank model to learn the low-dimensional feature vectors for the
models and datasets that best predict the cross-validated errors. To find
promising models for a new dataset, OBOE runs a set of fast but informative
algorithms on the new dataset and uses their cross-validated errors to infer
the feature vector for the new dataset. OBOE can find good models under
constraints on the number of models fit or the total time budget. To this end,
this paper develops a new heuristic for active learning in time-constrained
matrix completion based on optimal experiment design. Our experiments
demonstrate that OBOE delivers state-of-the-art performance faster than
competing approaches on a test bed of supervised learning problems. Moreover,
the success of the bilinear model used by OBOE suggests that AutoML may be
simpler than was previously understood
Optimal Principal Component Analysis in Distributed and Streaming Models
We study the Principal Component Analysis (PCA) problem in the distributed
and streaming models of computation. Given a matrix a
rank parameter , and an accuracy parameter , we
want to output an orthonormal matrix for which where is the best rank- approximation to .
This paper provides improved algorithms for distributed PCA and streaming
PCA.Comment: STOC2016 full versio
- …