1,005 research outputs found
Relations among Some Low Rank Subspace Recovery Models
Recovering intrinsic low dimensional subspaces from data distributed on them
is a key preprocessing step to many applications. In recent years, there has
been a lot of work that models subspace recovery as low rank minimization
problems. We find that some representative models, such as Robust Principal
Component Analysis (R-PCA), Robust Low Rank Representation (R-LRR), and Robust
Latent Low Rank Representation (R-LatLRR), are actually deeply connected. More
specifically, we discover that once a solution to one of the models is
obtained, we can obtain the solutions to other models in closed-form
formulations. Since R-PCA is the simplest, our discovery makes it the center of
low rank subspace recovery models. Our work has two important implications.
First, R-PCA has a solid theoretical foundation. Under certain conditions, we
could find better solutions to these low rank models at overwhelming
probabilities, although these models are non-convex. Second, we can obtain
significantly faster algorithms for these models by solving R-PCA first. The
computation cost can be further cut by applying low complexity randomized
algorithms, e.g., our novel filtering algorithm, to R-PCA.
Experiments verify the advantages of our algorithms over other state-of-the-art
ones that are based on the alternating direction method.Comment: Submitted to Neural Computatio
Kernelized Low Rank Representation on Grassmann Manifolds
Low rank representation (LRR) has recently attracted great interest due to
its pleasing efficacy in exploring low-dimensional subspace structures embedded
in data. One of its successful applications is subspace clustering which means
data are clustered according to the subspaces they belong to. In this paper, at
a higher level, we intend to cluster subspaces into classes of subspaces. This
is naturally described as a clustering problem on Grassmann manifold. The
novelty of this paper is to generalize LRR on Euclidean space onto an LRR model
on Grassmann manifold in a uniform kernelized framework. The new methods have
many applications in computer vision tasks. Several clustering experiments are
conducted on handwritten digit images, dynamic textures, human face clips and
traffic scene sequences. The experimental results show that the proposed
methods outperform a number of state-of-the-art subspace clustering methods.Comment: 13 page
Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation
This paper explores the problem of multi-view spectral clustering (MVSC)
based on tensor low-rank modeling. Unlike the existing methods that all adopt
an off-the-shelf tensor low-rank norm without considering the special
characteristics of the tensor in MVSC, we design a novel structured tensor
low-rank norm tailored to MVSC. Specifically, we explicitly impose a symmetric
low-rank constraint and a structured sparse low-rank constraint on the frontal
and horizontal slices of the tensor to characterize the intra-view and
inter-view relationships, respectively. Moreover, the two constraints could be
jointly optimized to achieve mutual refinement. On the basis of the novel
tensor low-rank norm, we formulate MVSC as a convex low-rank tensor recovery
problem, which is then efficiently solved with an augmented Lagrange multiplier
based method iteratively. Extensive experimental results on five benchmark
datasets show that the proposed method outperforms state-of-the-art methods to
a significant extent. Impressively, our method is able to produce perfect
clustering. In addition, the parameters of our method can be easily tuned, and
the proposed model is robust to different datasets, demonstrating its potential
in practice
Kernelized LRR on Grassmann Manifolds for Subspace Clustering
Low rank representation (LRR) has recently attracted great interest due to
its pleasing efficacy in exploring low-dimensional sub- space structures
embedded in data. One of its successful applications is subspace clustering, by
which data are clustered according to the subspaces they belong to. In this
paper, at a higher level, we intend to cluster subspaces into classes of
subspaces. This is naturally described as a clustering problem on Grassmann
manifold. The novelty of this paper is to generalize LRR on Euclidean space
onto an LRR model on Grassmann manifold in a uniform kernelized LRR framework.
The new method has many applications in data analysis in computer vision tasks.
The proposed models have been evaluated on a number of practical data analysis
applications. The experimental results show that the proposed models outperform
a number of state-of-the-art subspace clustering methods
Advancing Matrix Completion by Modeling Extra Structures beyond Low-Rankness
A well-known method for completing low-rank matrices based on convex
optimization has been established by Cand{\`e}s and Recht. Although
theoretically complete, the method may not entirely solve the low-rank matrix
completion problem. This is because the method captures only the low-rankness
property which gives merely a rough constraint that the data points locate on
some low-dimensional subspace, but generally ignores the extra structures which
specify in more detail how the data points locate on the subspace. Whenever the
geometric distribution of the data points is not uniform, the coherence
parameters of data might be large and, accordingly, the method might fail even
if the latent matrix we want to recover is fairly low-rank. To better handle
non-uniform data, in this paper we propose a method termed Low-Rank Factor
Decomposition (LRFD), which imposes an additional restriction that the data
points must be represented as linear combinations of the bases in a dictionary
constructed or learnt in advance. We show that LRFD can well handle non-uniform
data, provided that the dictionary is configured properly: We mathematically
prove that if the dictionary itself is low-rank then LRFD is immune to the
coherence parameters which might be large on non-uniform data. This provides an
elementary principle for learning the dictionary in LRFD and, naturally, leads
to a practical algorithm for advancing matrix completion. Extensive experiments
on randomly generated matrices and motion datasets show encouraging results.Comment: arXiv admin note: text overlap with arXiv:1404.403
Robust Subspace Discovery by Block-diagonal Adaptive Locality-constrained Representation
We propose a novel and unsupervised representation learning model, i.e.,
Robust Block-Diagonal Adaptive Locality-constrained Latent Representation
(rBDLR). rBDLR is able to recover multi-subspace structures and extract the
adaptive locality-preserving salient features jointly. Leveraging on the
Frobenius-norm based latent low-rank representation model, rBDLR jointly learns
the coding coefficients and salient features, and improves the results by
enhancing the robustness to outliers and errors in given data, preserving local
information of salient features adaptively and ensuring the block-diagonal
structures of the coefficients. To improve the robustness, we perform the
latent representation and adaptive weighting in a recovered clean data space.
To force the coefficients to be block-diagonal, we perform auto-weighting by
minimizing the reconstruction error based on salient features, constrained
using a block-diagonal regularizer. This ensures that a strict block-diagonal
weight matrix can be obtained and salient features will possess the adaptive
locality preserving ability. By minimizing the difference between the
coefficient and weights matrices, we can obtain a block-diagonal coefficients
matrix and it can also propagate and exchange useful information between
salient features and coefficients. Extensive results demonstrate the
superiority of rBDLR over other state-of-the-art methods.Comment: accepted by ACM Multimedia 201
Fast Subspace Clustering Based on the Kronecker Product
Subspace clustering is a useful technique for many computer vision
applications in which the intrinsic dimension of high-dimensional data is often
smaller than the ambient dimension. Spectral clustering, as one of the main
approaches to subspace clustering, often takes on a sparse representation or a
low-rank representation to learn a block diagonal self-representation matrix
for subspace generation. However, existing methods require solving a large
scale convex optimization problem with a large set of data, with computational
complexity reaches O(N^3) for N data points. Therefore, the efficiency and
scalability of traditional spectral clustering methods can not be guaranteed
for large scale datasets. In this paper, we propose a subspace clustering model
based on the Kronecker product. Due to the property that the Kronecker product
of a block diagonal matrix with any other matrix is still a block diagonal
matrix, we can efficiently learn the representation matrix which is formed by
the Kronecker product of k smaller matrices. By doing so, our model
significantly reduces the computational complexity to O(kN^{3/k}). Furthermore,
our model is general in nature, and can be adapted to different regularization
based subspace clustering methods. Experimental results on two public datasets
show that our model significantly improves the efficiency compared with several
state-of-the-art methods. Moreover, we have conducted experiments on synthetic
data to verify the scalability of our model for large scale datasets.Comment: 16 pages, 2 figure
Multilayer Collaborative Low-Rank Coding Network for Robust Deep Subspace Discovery
For subspace recovery, most existing low-rank representation (LRR) models
performs in the original space in single-layer mode. As such, the deep
hierarchical information cannot be learned, which may result in inaccurate
recoveries for complex real data. In this paper, we explore the deep
multi-subspace recovery problem by designing a multilayer architecture for
latent LRR. Technically, we propose a new Multilayer Collabora-tive Low-Rank
Representation Network model termed DeepLRR to discover deep features and deep
subspaces. In each layer (>2), DeepLRR bilinearly reconstructs the data matrix
by the collabo-rative representation with low-rank coefficients and projection
matrices in the previous layer. The bilinear low-rank reconstruc-tion of
previous layer is directly fed into the next layer as the input and low-rank
dictionary for representation learning, and is further decomposed into a deep
principal feature part, a deep salient feature part and a deep sparse error. As
such, the coher-ence issue can be also resolved due to the low-rank dictionary,
and the robustness against noise can also be enhanced in the feature subspace.
To recover the sparse errors in layers accurately, a dynamic growing strategy
is used, as the noise level will be-come smaller for the increase of layers.
Besides, a neighborhood reconstruction error is also included to encode the
locality of deep salient features by deep coefficients adaptively in each
layer. Extensive results on public databases show that our DeepLRR outperforms
other related models for subspace discovery and clustering.Comment: Accepted by the 24th European Conference on Artificial Intelligence
(ECAI 2020
Robust Subspace Clustering via Smoothed Rank Approximation
Matrix rank minimizing subject to affine constraints arises in many
application areas, ranging from signal processing to machine learning. Nuclear
norm is a convex relaxation for this problem which can recover the rank exactly
under some restricted and theoretically interesting conditions. However, for
many real-world applications, nuclear norm approximation to the rank function
can only produce a result far from the optimum. To seek a solution of higher
accuracy than the nuclear norm, in this paper, we propose a rank approximation
based on Logarithm-Determinant. We consider using this rank approximation for
subspace clustering application. Our framework can model different kinds of
errors and noise. Effective optimization strategy is developed with theoretical
guarantee to converge to a stationary point. The proposed method gives
promising results on face clustering and motion segmentation tasks compared to
the state-of-the-art subspace clustering algorithms.Comment: Journal, code is availabl
Learning Robust Representations for Computer Vision
Unsupervised learning techniques in computer vision often require learning
latent representations, such as low-dimensional linear and non-linear
subspaces. Noise and outliers in the data can frustrate these approaches by
obscuring the latent spaces.
Our main goal is deeper understanding and new development of robust
approaches for representation learning. We provide a new interpretation for
existing robust approaches and present two specific contributions: a new robust
PCA approach, which can separate foreground features from dynamic background,
and a novel robust spectral clustering method, that can cluster facial images
with high accuracy. Both contributions show superior performance to standard
methods on real-world test sets.Comment: 8 pages, 7 page
- …