763 research outputs found
Minimax rank estimation for subspace tracking
Rank estimation is a classical model order selection problem that arises in a
variety of important statistical signal and array processing systems, yet is
addressed relatively infrequently in the extant literature. Here we present
sample covariance asymptotics stemming from random matrix theory, and bring
them to bear on the problem of optimal rank estimation in the context of the
standard array observation model with additive white Gaussian noise. The most
significant of these results demonstrates the existence of a phase transition
threshold, below which eigenvalues and associated eigenvectors of the sample
covariance fail to provide any information on population eigenvalues. We then
develop a decision-theoretic rank estimation framework that leads to a simple
ordered selection rule based on thresholding; in contrast to competing
approaches, however, it admits asymptotic minimax optimality and is free of
tuning parameters. We analyze the asymptotic performance of our rank selection
procedure and conclude with a brief simulation study demonstrating its
practical efficacy in the context of subspace tracking.Comment: 10 pages, 4 figures; final versio
Adaptive Reduced Rank Regression
We study the low rank regression problem , where and are and dimensional
vectors respectively. We consider the extreme high-dimensional setting where
the number of observations is less than . Existing algorithms
are designed for settings where is typically as large as
. This work provides an efficient algorithm which
only involves two SVD, and establishes statistical guarantees on its
performance. The algorithm decouples the problem by first estimating the
precision matrix of the features, and then solving the matrix denoising
problem. To complement the upper bound, we introduce new techniques for
establishing lower bounds on the performance of any algorithm for this problem.
Our preliminary experiments confirm that our algorithm often out-performs
existing baselines, and is always at least competitive.Comment: 40 page
On-line PCA with Optimal Regrets
We carefully investigate the on-line version of PCA, where in each trial a
learning algorithm plays a k-dimensional subspace, and suffers the compression
loss on the next instance when projected into the chosen subspace. In this
setting, we analyze two popular on-line algorithms, Gradient Descent (GD) and
Exponentiated Gradient (EG). We show that both algorithms are essentially
optimal in the worst-case. This comes as a surprise, since EG is known to
perform sub-optimally when the instances are sparse. This different behavior of
EG for PCA is mainly related to the non-negativity of the loss in this case,
which makes the PCA setting qualitatively different from other settings studied
in the literature. Furthermore, we show that when considering regret bounds as
function of a loss budget, EG remains optimal and strictly outperforms GD.
Next, we study the extension of the PCA setting, in which the Nature is allowed
to play with dense instances, which are positive matrices with bounded largest
eigenvalue. Again we can show that EG is optimal and strictly better than GD in
this setting
Robust Kalman tracking and smoothing with propagating and non-propagating outliers
A common situation in filtering where classical Kalman filtering does not
perform particularly well is tracking in the presence of propagating outliers.
This calls for robustness understood in a distributional sense, i.e.; we
enlarge the distribution assumptions made in the ideal model by suitable
neighborhoods. Based on optimality results for distributional-robust Kalman
filtering from Ruckdeschel[01,10], we propose new robust recursive filters and
smoothers designed for this purpose as well as specialized versions for
non-propagating outliers. We apply these procedures in the context of a GPS
problem arising in the car industry. To better understand these filters, we
study their behavior at stylized outlier patterns (for which they are not
designed) and compare them to other approaches for the tracking problem.
Finally, in a simulation study we discuss efficiency of our procedures in
comparison to competitors.Comment: 27 pages, 12 figures, 2 table
- …