995 research outputs found
Robust Non-Linear Matrix Factorization for Dictionary Learning, Denoising, and Clustering
Low dimensional nonlinear structure abounds in datasets across computer
vision and machine learning. Kernelized matrix factorization techniques have
recently been proposed to learn these nonlinear structures for denoising,
classification, dictionary learning, and missing data imputation, by observing
that the image of the matrix in a sufficiently large feature space is low-rank.
However, these nonlinear methods fail in the presence of sparse noise or
outliers. In this work, we propose a new robust nonlinear factorization method
called Robust Non-Linear Matrix Factorization (RNLMF). RNLMF constructs a
dictionary for the data space by factoring a kernelized feature space; a noisy
matrix can then be decomposed as the sum of a sparse noise matrix and a clean
data matrix that lies in a low dimensional nonlinear manifold. RNLMF is robust
to sparse noise and outliers and scales to matrices with thousands of rows and
columns. Empirically, RNLMF achieves noticeable improvements over baseline
methods in denoising and clustering
Low-Rank Modeling and Its Applications in Image Analysis
Low-rank modeling generally refers to a class of methods that solve problems
by representing variables of interest as low-rank matrices. It has achieved
great success in various fields including computer vision, data mining, signal
processing and bioinformatics. Recently, much progress has been made in
theories, algorithms and applications of low-rank modeling, such as exact
low-rank matrix recovery via convex programming and matrix completion applied
to collaborative filtering. These advances have brought more and more
attentions to this topic. In this paper, we review the recent advance of
low-rank modeling, the state-of-the-art algorithms, and related applications in
image analysis. We first give an overview to the concept of low-rank modeling
and challenging problems in this area. Then, we summarize the models and
algorithms for low-rank matrix recovery and illustrate their advantages and
limitations with numerical experiments. Next, we introduce a few applications
of low-rank modeling in the context of image analysis. Finally, we conclude
this paper with some discussions.Comment: To appear in ACM Computing Survey
Matrix Recovery with Implicitly Low-Rank Data
In this paper, we study the problem of matrix recovery, which aims to restore
a target matrix of authentic samples from grossly corrupted observations. Most
of the existing methods, such as the well-known Robust Principal Component
Analysis (RPCA), assume that the target matrix we wish to recover is low-rank.
However, the underlying data structure is often non-linear in practice,
therefore the low-rankness assumption could be violated. To tackle this issue,
we propose a novel method for matrix recovery in this paper, which could well
handle the case where the target matrix is low-rank in an implicit feature
space but high-rank or even full-rank in its original form. Namely, our method
pursues the low-rank structure of the target matrix in an implicit feature
space. By making use of the specifics of an accelerated proximal gradient based
optimization algorithm, the proposed method could recover the target matrix
with non-linear structures from its corrupted version. Comprehensive
experiments on both synthetic and real datasets demonstrate the superiority of
our method
Relations among Some Low Rank Subspace Recovery Models
Recovering intrinsic low dimensional subspaces from data distributed on them
is a key preprocessing step to many applications. In recent years, there has
been a lot of work that models subspace recovery as low rank minimization
problems. We find that some representative models, such as Robust Principal
Component Analysis (R-PCA), Robust Low Rank Representation (R-LRR), and Robust
Latent Low Rank Representation (R-LatLRR), are actually deeply connected. More
specifically, we discover that once a solution to one of the models is
obtained, we can obtain the solutions to other models in closed-form
formulations. Since R-PCA is the simplest, our discovery makes it the center of
low rank subspace recovery models. Our work has two important implications.
First, R-PCA has a solid theoretical foundation. Under certain conditions, we
could find better solutions to these low rank models at overwhelming
probabilities, although these models are non-convex. Second, we can obtain
significantly faster algorithms for these models by solving R-PCA first. The
computation cost can be further cut by applying low complexity randomized
algorithms, e.g., our novel filtering algorithm, to R-PCA.
Experiments verify the advantages of our algorithms over other state-of-the-art
ones that are based on the alternating direction method.Comment: Submitted to Neural Computatio
A survey of sparse representation: algorithms and applications
Sparse representation has attracted much attention from researchers in fields
of signal processing, image processing, computer vision and pattern
recognition. Sparse representation also has a good reputation in both
theoretical research and practical applications. Many different algorithms have
been proposed for sparse representation. The main purpose of this article is to
provide a comprehensive study and an updated review on sparse representation
and to supply a guidance for researchers. The taxonomy of sparse representation
methods can be studied from various viewpoints. For example, in terms of
different norm minimizations used in sparsity constraints, the methods can be
roughly categorized into five groups: sparse representation with -norm
minimization, sparse representation with -norm (0p1) minimization,
sparse representation with -norm minimization and sparse representation
with -norm minimization. In this paper, a comprehensive overview of
sparse representation is provided. The available sparse representation
algorithms can also be empirically categorized into four groups: greedy
strategy approximation, constrained optimization, proximity algorithm-based
optimization, and homotopy algorithm-based sparse representation. The
rationales of different algorithms in each category are analyzed and a wide
range of sparse representation applications are summarized, which could
sufficiently reveal the potential nature of the sparse representation theory.
Specifically, an experimentally comparative study of these sparse
representation algorithms was presented. The Matlab code used in this paper can
be available at: http://www.yongxu.org/lunwen.html.Comment: Published on IEEE Access, Vol. 3, pp. 490-530, 201
A General Model for Robust Tensor Factorization with Unknown Noise
Because of the limitations of matrix factorization, such as losing spatial
structure information, the concept of low-rank tensor factorization (LRTF) has
been applied for the recovery of a low dimensional subspace from high
dimensional visual data. The low-rank tensor recovery is generally achieved by
minimizing the loss function between the observed data and the factorization
representation. The loss function is designed in various forms under different
noise distribution assumptions, like norm for Laplacian distribution and
norm for Gaussian distribution. However, they often fail to tackle the
real data which are corrupted by the noise with unknown distribution. In this
paper, we propose a generalized weighted low-rank tensor factorization method
(GWLRTF) integrated with the idea of noise modelling. This procedure treats the
target data as high-order tensor directly and models the noise by a Mixture of
Gaussians, which is called MoG GWLRTF. The parameters in the model are
estimated under the EM framework and through a new developed algorithm of
weighted low-rank tensor factorization. We provide two versions of the
algorithm with different tensor factorization operations, i.e., CP
factorization and Tucker factorization. Extensive experiments indicate the
respective advantages of this two versions in different applications and also
demonstrate the effectiveness of MoG GWLRTF compared with other competing
methods.Comment: 13 pages, 8 figure
Learning Robust Representations for Computer Vision
Unsupervised learning techniques in computer vision often require learning
latent representations, such as low-dimensional linear and non-linear
subspaces. Noise and outliers in the data can frustrate these approaches by
obscuring the latent spaces.
Our main goal is deeper understanding and new development of robust
approaches for representation learning. We provide a new interpretation for
existing robust approaches and present two specific contributions: a new robust
PCA approach, which can separate foreground features from dynamic background,
and a novel robust spectral clustering method, that can cluster facial images
with high accuracy. Both contributions show superior performance to standard
methods on real-world test sets.Comment: 8 pages, 7 page
Self-Expressive Decompositions for Matrix Approximation and Clustering
Data-aware methods for dimensionality reduction and matrix decomposition aim
to find low-dimensional structure in a collection of data. Classical approaches
discover such structure by learning a basis that can efficiently express the
collection. Recently, "self expression", the idea of using a small subset of
data vectors to represent the full collection, has been developed as an
alternative to learning. Here, we introduce a scalable method for computing
sparse SElf-Expressive Decompositions (SEED). SEED is a greedy method that
constructs a basis by sequentially selecting incoherent vectors from the
dataset. After forming a basis from a subset of vectors in the dataset, SEED
then computes a sparse representation of the dataset with respect to this
basis. We develop sufficient conditions under which SEED exactly represents low
rank matrices and vectors sampled from a unions of independent subspaces. We
show how SEED can be used in applications ranging from matrix approximation and
denoising to clustering, and apply it to numerous real-world datasets. Our
results demonstrate that SEED is an attractive low-complexity alternative to
other sparse matrix factorization approaches such as sparse PCA and
self-expressive methods for clustering.Comment: 11 pages, 7 figure
Fast, Robust and Non-convex Subspace Recovery
This work presents a fast and non-convex algorithm for robust subspace
recovery. The data sets considered include inliers drawn around a
low-dimensional subspace of a higher dimensional ambient space, and a possibly
large portion of outliers that do not lie nearby this subspace. The proposed
algorithm, which we refer to as Fast Median Subspace (FMS), is designed to
robustly determine the underlying subspace of such data sets, while having
lower computational complexity than existing methods. We prove convergence of
the FMS iterates to a stationary point. Further, under a special model of data,
FMS converges to a point which is near to the global minimum with overwhelming
probability. Under this model, we show that the iteration complexity is
globally bounded and locally -linear. The latter theorem holds for any fixed
fraction of outliers (less than 1) and any fixed positive distance between the
limit point and the global minimum. Numerical experiments on synthetic and real
data demonstrate its competitive speed and accuracy
Low Rank Regularization: A Review
Low rank regularization, in essence, involves introducing a low rank or
approximately low rank assumption for matrix we aim to learn, which has
achieved great success in many fields including machine learning, data mining
and computer version. Over the last decade, much progress has been made in
theories and practical applications. Nevertheless, the intersection between
them is very slight. In order to construct a bridge between practical
applications and theoretical research, in this paper we provide a comprehensive
survey for low rank regularization. We first review several traditional machine
learning models using low rank regularization, and then show their (or their
variants) applications in solving practical issues, such as non-rigid structure
from motion and image denoising. Subsequently, we summarize the regularizers
and optimization methods that achieve great success in traditional machine
learning tasks but are rarely seen in solving practical issues. Finally, we
provide a discussion and comparison for some representative regularizers
including convex and non-convex relaxations. Extensive experimental results
demonstrate that non-convex regularizers can provide a large advantage over the
nuclear norm, the regularizer widely used in solving practical issues.Comment: 16 pages,4 figures,4 table
- …