5 research outputs found
Robust Multi-subspace Analysis Using Novel Column L0-norm Constrained Matrix Factorization
We study the underlying structure of data (approximately) generated from a
union of independent subspaces. Traditional methods learn only one subspace,
failing to discover the multi-subspace structure, while state-of-the-art
methods analyze the multi-subspace structure using data themselves as the
dictionary, which cannot offer the explicit basis to span each subspace and are
sensitive to errors via an indirect representation. Additionally, they also
suffer from a high computational complexity, being quadratic or cubic to the
sample size. To tackle all these problems, we propose a method, called Matrix
Factorization with Column L0-norm constraint (MFC0), that can simultaneously
learn the basis for each subspace, generate a direct sparse representation for
each data sample, as well as removing errors in the data in an efficient way.
Furthermore, we develop a first-order alternating direction algorithm, whose
computational complexity is linear to the sample size, to stably and
effectively solve the nonconvex objective function and non- smooth l0-norm
constraint of MFC0. Experimental results on both synthetic and real-world
datasets demonstrate that besides the superiority over traditional and
state-of-the-art methods for subspace clustering, data reconstruction, error
correction, MFC0 also shows its uniqueness for multi-subspace basis learning
and direct sparse representation.Comment: 13 pages, 8 figures, 8 table
Deep Self-representative Concept Factorization Network for Representation Learning
In this paper, we investigate the unsupervised deep representation learning
issue and technically propose a novel framework called Deep Self-representative
Concept Factorization Network (DSCF-Net), for clustering deep features. To
improve the representation and clustering abilities, DSCF-Net explicitly
considers discovering hidden deep semantic features, enhancing the robustness
proper-ties of the deep factorization to noise and preserving the local
man-ifold structures of deep features. Specifically, DSCF-Net seamlessly
integrates the robust deep concept factorization, deep self-expressive
representation and adaptive locality preserving feature learning into a unified
framework. To discover hidden deep repre-sentations, DSCF-Net designs a
hierarchical factorization architec-ture using multiple layers of linear
transformations, where the hierarchical representation is performed by
formulating the prob-lem as optimizing the basis concepts in each layer to
improve the representation indirectly. DSCF-Net also improves the robustness by
subspace recovery for sparse error correction firstly and then performs the
deep factorization in the recovered visual subspace. To obtain
locality-preserving representations, we also present an adaptive deep
self-representative weighting strategy by using the coefficient matrix as the
adaptive reconstruction weights to keep the locality of representations.
Extensive comparison results with several other related models show that
DSCF-Net delivers state-of-the-art performance on several public databases.Comment: Accepted by SDM 202
Joint Label Prediction based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation
Constrained Concept Factorization (CCF) yields the enhanced representation
ability over CF by incorporating label information as additional constraints,
but it cannot classify and group unlabeled data appropriately. Minimizing the
difference between the original data and its reconstruction directly can enable
CCF to model a small noisy perturbation, but is not robust to gross sparse
errors. Besides, CCF cannot preserve the manifold structures in new
representation space explicitly, especially in an adaptive manner. In this
paper, we propose a joint label prediction based Robust Semi-Supervised
Adaptive Concept Factorization (RS2ACF) framework. To obtain robust
representation, RS2ACF relaxes the factorization to make it simultaneously
stable to small entrywise noise and robust to sparse errors. To enrich prior
knowledge to enhance the discrimination, RS2ACF clearly uses class information
of labeled data and more importantly propagates it to unlabeled data by jointly
learning an explicit label indicator for unlabeled data. By the label
indicator, RS2ACF can ensure the unlabeled data of the same predicted label to
be mapped into the same class in feature space. Besides, RS2ACF incorporates
the joint neighborhood reconstruction error over the new representations and
predicted labels of both labeled and unlabeled data, so the manifold structures
can be preserved explicitly and adaptively in the representation space and
label space at the same time. Owing to the adaptive manner, the tricky process
of determining the neighborhood size or kernel width can be avoided. Extensive
results on public databases verify that our RS2ACF can deliver state-of-the-art
data representation, compared with other related methods.Comment: Accepted at IEEE TKD
Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior
Nonnegative matrix factorization is usually powerful for learning the
"shallow" parts-based representation, but it clearly fails to discover deep
hierarchical information within both the basis and representation spaces. In
this paper, we technically propose a new enriched prior based Dual-constrained
Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net, for
learning the hierarchical coupled representations. To ex-tract hidden deep
features, DS2CF-Net is modeled as a deep-structure and geometrical
structure-constrained neural network. Specifically, DS2CF-Net designs a deep
coupled factorization architecture using multi-layers of linear
transformations, which coupled updates the bases and new representations in
each layer. To improve the discriminating ability of learned deep
representations and deep coefficients, our network clearly considers enriching
the supervised prior by the joint deep coefficients-regularized label
prediction, and incorporates enriched prior information as additional label and
structure constraints. The label constraint can enable the samples of the same
label to have the same coordinate in the new feature space, while the structure
constraint forces the coefficient matrices in each layer to be block-diagonal
so that the enhanced prior using the self-expressive label propagation are more
accurate. Our network also integrates the adaptive dual-graph learning to
retain the local manifold structures of both the data manifold and feature
manifold by minimizing the reconstruction errors in each layer. Extensive
experiments on several real databases demonstrate that our DS2CF-Net can obtain
state-of-the-art performance for representation learning and clustering
Flexible Auto-weighted Local-coordinate Concept Factorization: A Robust Framework for Unsupervised Clustering
Concept Factorization (CF) and its variants may produce inaccurate
representation and clustering results due to the sensitivity to noise, hard
constraint on the reconstruction error and pre-obtained approximate
similarities. To improve the representation ability, a novel unsupervised
Robust Flexible Auto-weighted Local-coordinate Concept Factorization (RFA-LCF)
framework is proposed for clustering high-dimensional data. Specifically,
RFA-LCF integrates the robust flexible CF by clean data space recovery, robust
sparse local-coordinate coding and adaptive weighting into a unified model.
RFA-LCF improves the representations by enhancing the robustness of CF to noise
and errors, providing a flexible constraint on the reconstruction error and
optimizing the locality jointly. For robust learning, RFA-LCF clearly learns a
sparse projection to recover the underlying clean data space, and then the
flexible CF is performed in the projected feature space. RFA-LCF also uses a
L2,1-norm based flexible residue to encode the mismatch between the recovered
data and its reconstruction, and uses the robust sparse local-coordinate coding
to represent data using a few nearby basis concepts. For auto-weighting,
RFA-LCF jointly preserves the manifold structures in the basis concept space
and new coordinate space in an adaptive manner by minimizing the reconstruction
errors on clean data, anchor points and coordinates. By updating the
local-coordinate preserving data, basis concepts and new coordinates
alternately, the representation abilities can be potentially improved.
Extensive results on public databases show that RFA-LCF delivers the
state-of-the-art clustering results compared with other related methods.Comment: Accepted by IEEE Transactions on Knowledge and Data Engineering (IEEE
TKDE