23,764 research outputs found
Fast Robust Subspace Tracking via PCA in Sparse Data-Dependent Noise
This work studies the robust subspace tracking (ST) problem. Robust ST can be
simply understood as a (slow) time-varying subspace extension of robust PCA. It
assumes that the true data lies in a low-dimensional subspace that is either
fixed or changes slowly with time. The goal is to track the changing subspaces
over time in the presence of additive sparse outliers and to do this quickly
(with a short delay). We introduce a "fast" mini-batch robust ST solution that
is provably correct under mild assumptions. Here "fast" means two things: (i)
the subspace changes can be detected and the subspaces can be tracked with
near-optimal delay, and (ii) the time complexity of doing this is the same as
that of simple (non-robust) PCA. Our main result assumes piecewise constant
subspaces (needed for identifiability), but we also provide a corollary for the
case when there is a little change at each time.
A second contribution is a novel non-asymptotic guarantee for PCA in linearly
data-dependent noise. An important setting where this is useful is for linearly
data dependent noise that is sparse with support that changes enough over time.
The analysis of the subspace update step of our proposed robust ST solution
uses this result.Comment: To appear in IEEE Journal of Special Areas in Information Theor
Low-rank and Sparse Soft Targets to Learn Better DNN Acoustic Models
Conventional deep neural networks (DNN) for speech acoustic modeling rely on
Gaussian mixture models (GMM) and hidden Markov model (HMM) to obtain binary
class labels as the targets for DNN training. Subword classes in speech
recognition systems correspond to context-dependent tied states or senones. The
present work addresses some limitations of GMM-HMM senone alignments for DNN
training. We hypothesize that the senone probabilities obtained from a DNN
trained with binary labels can provide more accurate targets to learn better
acoustic models. However, DNN outputs bear inaccuracies which are exhibited as
high dimensional unstructured noise, whereas the informative components are
structured and low-dimensional. We exploit principle component analysis (PCA)
and sparse coding to characterize the senone subspaces. Enhanced probabilities
obtained from low-rank and sparse reconstructions are used as soft-targets for
DNN acoustic modeling, that also enables training with untranscribed data.
Experiments conducted on AMI corpus shows 4.6% relative reduction in word error
rate
Detecting and quantifying stellar magnetic fields -- Sparse Stokes profile approximation using orthogonal matching pursuit
In the recent years, we have seen a rapidly growing number of stellar
magnetic field detections for various types of stars. Many of these magnetic
fields are estimated from spectropolarimetric observations (Stokes V) by using
the so-called center-of-gravity (COG) method. Unfortunately, the accuracy of
this method rapidly deteriorates with increasing noise and thus calls for a
more robust procedure that combines signal detection and field estimation. We
introduce an estimation method that provides not only the effective or mean
longitudinal magnetic field from an observed Stokes V profile but also uses the
net absolute polarization of the profile to obtain an estimate of the apparent
(i.e., velocity resolved) absolute longitudinal magnetic field. By combining
the COG method with an orthogonal-matching-pursuit (OMP) approach, we were able
to decompose observed Stokes profiles with an overcomplete dictionary of
wavelet-basis functions to reliably reconstruct the observed Stokes profiles in
the presence of noise. The elementary wave functions of the sparse
reconstruction process were utilized to estimate the effective longitudinal
magnetic field and the apparent absolute longitudinal magnetic field. A
multiresolution analysis complements the OMP algorithm to provide a robust
detection and estimation method. An extensive Monte-Carlo simulation confirms
the reliability and accuracy of the magnetic OMP approach.Comment: A&A, in press, 15 pages, 14 figure
A linear approach for sparse coding by a two-layer neural network
Many approaches to transform classification problems from non-linear to
linear by feature transformation have been recently presented in the
literature. These notably include sparse coding methods and deep neural
networks. However, many of these approaches require the repeated application of
a learning process upon the presentation of unseen data input vectors, or else
involve the use of large numbers of parameters and hyper-parameters, which must
be chosen through cross-validation, thus increasing running time dramatically.
In this paper, we propose and experimentally investigate a new approach for the
purpose of overcoming limitations of both kinds. The proposed approach makes
use of a linear auto-associative network (called SCNN) with just one hidden
layer. The combination of this architecture with a specific error function to
be minimized enables one to learn a linear encoder computing a sparse code
which turns out to be as similar as possible to the sparse coding that one
obtains by re-training the neural network. Importantly, the linearity of SCNN
and the choice of the error function allow one to achieve reduced running time
in the learning phase. The proposed architecture is evaluated on the basis of
two standard machine learning tasks. Its performances are compared with those
of recently proposed non-linear auto-associative neural networks. The overall
results suggest that linear encoders can be profitably used to obtain sparse
data representations in the context of machine learning problems, provided that
an appropriate error function is used during the learning phase
Adaptive Image Denoising by Targeted Databases
We propose a data-dependent denoising procedure to restore noisy images.
Different from existing denoising algorithms which search for patches from
either the noisy image or a generic database, the new algorithm finds patches
from a database that contains only relevant patches. We formulate the denoising
problem as an optimal filter design problem and make two contributions. First,
we determine the basis function of the denoising filter by solving a group
sparsity minimization problem. The optimization formulation generalizes
existing denoising algorithms and offers systematic analysis of the
performance. Improvement methods are proposed to enhance the patch search
process. Second, we determine the spectral coefficients of the denoising filter
by considering a localized Bayesian prior. The localized prior leverages the
similarity of the targeted database, alleviates the intensive Bayesian
computation, and links the new method to the classical linear minimum mean
squared error estimation. We demonstrate applications of the proposed method in
a variety of scenarios, including text images, multiview images and face
images. Experimental results show the superiority of the new algorithm over
existing methods.Comment: 15 pages, 13 figures, 2 tables, journa
- …