1,094 research outputs found
Robust Unsupervised Flexible Auto-weighted Local-Coordinate Concept Factorization for Image Clustering
We investigate the high-dimensional data clustering problem by proposing a
novel and unsupervised representation learning model called Robust Flexible
Auto-weighted Local-coordinate Concept Factorization (RFA-LCF). RFA-LCF
integrates the robust flexible CF, robust sparse local-coordinate coding and
the adaptive reconstruction weighting learning into a unified model. The
adaptive weighting is driven by including the joint manifold preserving
constraints on the recovered clean data, basis concepts and new representation.
Specifically, our RFA-LCF uses a L2,1-norm based flexible residue to encode the
mismatch between clean data and its reconstruction, and also applies the robust
adaptive sparse local-coordinate coding to represent the data using a few
nearby basis concepts, which can make the factorization more accurate and
robust to noise. The robust flexible factorization is also performed in the
recovered clean data space for enhancing representations. RFA-LCF also
considers preserving the local manifold structures of clean data space, basis
concept space and the new coordinate space jointly in an adaptive manner way.
Extensive comparisons show that RFA-LCF can deliver enhanced clustering
results.Comment: Accepted at the 44th IEEE International Conference on Acoustics,
Speech, and Signal Processing(ICASSP 2019
Sparse Deep Nonnegative Matrix Factorization
Nonnegative matrix factorization is a powerful technique to realize dimension
reduction and pattern recognition through single-layer data representation
learning. Deep learning, however, with its carefully designed hierarchical
structure, is able to combine hidden features to form more representative
features for pattern recognition. In this paper, we proposed sparse deep
nonnegative matrix factorization models to analyze complex data for more
accurate classification and better feature interpretation. Such models are
designed to learn localized features or generate more discriminative
representations for samples in distinct classes by imposing -norm penalty
on the columns of certain factors. By extending one-layer model into
multi-layer one with sparsity, we provided a hierarchical way to analyze big
data and extract hidden features intuitively due to nonnegativity. We adopted
the Nesterov's accelerated gradient algorithm to accelerate the computing
process with the convergence rate of after steps iteration. We
also analyzed the computing complexity of our framework to demonstrate their
efficiency. To improve the performance of dealing with linearly inseparable
data, we also considered to incorporate popular nonlinear functions into this
framework and explored their performance. We applied our models onto two
benchmarking image datasets, demonstrating our models can achieve competitive
or better classification performance and produce intuitive interpretations
compared with the typical NMF and competing multi-layer models.Comment: 13 pages, 8 figure
Effective Spectral Unmixing via Robust Representation and Learning-based Sparsity
Hyperspectral unmixing (HU) plays a fundamental role in a wide range of
hyperspectral applications. It is still challenging due to the common presence
of outlier channels and the large solution space. To address the above two
issues, we propose a novel model by emphasizing both robust representation and
learning-based sparsity. Specifically, we apply the -norm to
measure the representation error, preventing outlier channels from dominating
our objective. In this way, the side effects of outlier channels are greatly
relieved. Besides, we observe that the mixed level of each pixel varies over
image grids. Based on this observation, we exploit a learning-based sparsity
method to simultaneously learn the HU results and a sparse guidance map. Via
this guidance map, the sparsity constraint in the -norm is adaptively imposed according to the learnt mixed
level of each pixel. Compared with state-of-the-art methods, our model is
better suited to the real situation, thus expected to achieve better HU
results. The resulted objective is highly non-convex and non-smooth, and so it
is hard to optimize. As a profound theoretical contribution, we propose an
efficient algorithm to solve it. Meanwhile, the convergence proof and the
computational complexity analysis are systematically provided. Extensive
evaluations verify that our method is highly promising for the HU task---it
achieves very accurate guidance maps and much better HU results compared with
state-of-the-art methods
Spectral Unmixing via Data-guided Sparsity
Hyperspectral unmixing, the process of estimating a common set of spectral
bases and their corresponding composite percentages at each pixel, is an
important task for hyperspectral analysis, visualization and understanding.
From an unsupervised learning perspective, this problem is very
challenging---both the spectral bases and their composite percentages are
unknown, making the solution space too large. To reduce the solution space,
many approaches have been proposed by exploiting various priors. In practice,
these priors would easily lead to some unsuitable solution. This is because
they are achieved by applying an identical strength of constraints to all the
factors, which does not hold in practice. To overcome this limitation, we
propose a novel sparsity based method by learning a data-guided map to describe
the individual mixed level of each pixel. Through this data-guided map, the
constraint is applied in an adaptive manner. Such
implementation not only meets the practical situation, but also guides the
spectral bases toward the pixels under highly sparse constraint. What's more,
an elegant optimization scheme as well as its convergence proof have been
provided in this paper. Extensive experiments on several datasets also
demonstrate that the data-guided map is feasible, and high quality unmixing
results could be obtained by our method
A Unified Joint Matrix Factorization Framework for Data Integration
Nonnegative matrix factorization (NMF) is a powerful tool in data exploratory
analysis by discovering the hidden features and part-based patterns from
high-dimensional data. NMF and its variants have been successfully applied into
diverse fields such as pattern recognition, signal processing, data mining,
bioinformatics and so on. Recently, NMF has been extended to analyze multiple
matrices simultaneously. However, a unified framework is still lacking. In this
paper, we introduce a sparse multiple relationship data regularized joint
matrix factorization (JMF) framework and two adapted prediction models for
pattern recognition and data integration. Next, we present four update
algorithms to solve this framework. The merits and demerits of these algorithms
are systematically explored. Furthermore, extensive computational experiments
using both synthetic data and real data demonstrate the effectiveness of JMF
framework and related algorithms on pattern recognition and data mining.Comment: 14 pages, 7 figure
Accelerated Parallel and Distributed Algorithm using Limited Internal Memory for Nonnegative Matrix Factorization
Nonnegative matrix factorization (NMF) is a powerful technique for dimension
reduction, extracting latent factors and learning part-based representation.
For large datasets, NMF performance depends on some major issues: fast
algorithms, fully parallel distributed feasibility and limited internal memory.
This research aims to design a fast fully parallel and distributed algorithm
using limited internal memory to reach high NMF performance for large datasets.
In particular, we propose a flexible accelerated algorithm for NMF with all its
regularized variants based on full decomposition, which is a
combination of an anti-lopsided algorithm and a fast block coordinate descent
algorithm. The proposed algorithm takes advantages of both these algorithms to
achieve a linear convergence rate of in
optimizing each factor matrix when fixing the other factor one in the sub-space
of passive variables, where is the number of latent components; where
. In addition, the algorithm can exploit the data
sparseness to run on large datasets with limited internal memory of machines.
Furthermore, our experimental results are highly competitive with 7
state-of-the-art methods about three significant aspects of convergence,
optimality and average of the iteration number. Therefore, the proposed
algorithm is superior to fast block coordinate descent methods and accelerated
methods
A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering
A recent theoretical analysis shows the equivalence between non-negative
matrix factorization (NMF) and spectral clustering based approach to subspace
clustering. As NMF and many of its variants are essentially linear, we
introduce a nonlinear NMF with explicit orthogonality and derive general
kernel-based orthogonal multiplicative update rules to solve the subspace
clustering problem. In nonlinear orthogonal NMF framework, we propose two
subspace clustering algorithms, named kernel-based non-negative subspace
clustering KNSC-Ncut and KNSC-Rcut and establish their connection with spectral
normalized cut and ratio cut clustering. We further extend the nonlinear
orthogonal NMF framework and introduce a graph regularization to obtain a
factorization that respects a local geometric structure of the data after the
nonlinear mapping. The proposed NMF-based approach to subspace clustering takes
into account the nonlinear nature of the manifold, as well as its intrinsic
local geometry, which considerably improves the clustering performance when
compared to the several recently proposed state-of-the-art methods
Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset
Recent research on problem formulations based on decomposition into low-rank
plus sparse matrices shows a suitable framework to separate moving objects from
the background. The most representative problem formulation is the Robust
Principal Component Analysis (RPCA) solved via Principal Component Pursuit
(PCP) which decomposes a data matrix in a low-rank matrix and a sparse matrix.
However, similar robust implicit or explicit decompositions can be made in the
following problem formulations: Robust Non-negative Matrix Factorization
(RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust
Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal
of these similar problem formulations is to obtain explicitly or implicitly a
decomposition into low-rank matrix plus additive matrices. In this context,
this work aims to initiate a rigorous and comprehensive review of the similar
problem formulations in robust subspace learning and tracking based on
decomposition into low-rank plus additive matrices for testing and ranking
existing algorithms for background/foreground separation. For this, we first
provide a preliminary review of the recent developments in the different
problem formulations which allows us to define a unified view that we called
Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine
carefully each method in each robust subspace learning/tracking frameworks with
their decomposition, their loss functions, their optimization problem and their
solvers. Furthermore, we investigate if incremental algorithms and real-time
implementations can be achieved for background/foreground separation. Finally,
experimental results on a large-scale dataset called Background Models
Challenge (BMC 2012) show the comparative performance of 32 different robust
subspace learning/tracking methods.Comment: 121 pages, 5 figures, submitted to Computer Science Review. arXiv
admin note: text overlap with arXiv:1312.7167, arXiv:1109.6297,
arXiv:1207.3438, arXiv:1105.2126, arXiv:1404.7592, arXiv:1210.0805,
arXiv:1403.8067 by other authors, Computer Science Review, November 201
A Survey on Multi-View Clustering
With advances in information acquisition technologies, multi-view data become
ubiquitous. Multi-view learning has thus become more and more popular in
machine learning and data mining fields. Multi-view unsupervised or
semi-supervised learning, such as co-training, co-regularization has gained
considerable attention. Although recently, multi-view clustering (MVC) methods
have been developed rapidly, there has not been a survey to summarize and
analyze the current progress. Therefore, this paper reviews the common
strategies for combining multiple views of data and based on this summary we
propose a novel taxonomy of the MVC approaches. We further discuss the
relationships between MVC and multi-view representation, ensemble clustering,
multi-task clustering, multi-view supervised and semi-supervised learning.
Several representative real-world applications are elaborated. To promote
future development of MVC, we envision several open problems that may require
further investigation and thorough examination.Comment: 17 pages, 4 figure
Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms
Matrix Factorization is a popular non-convex optimization problem, for which
alternating minimization schemes are mostly used. They usually suffer from the
major drawback that the solution is biased towards one of the optimization
variables. A remedy is non-alternating schemes. However, due to a lack of
Lipschitz continuity of the gradient in matrix factorization problems,
convergence cannot be guaranteed. A recently developed approach relies on the
concept of Bregman distances, which generalizes the standard Euclidean
distance. We exploit this theory by proposing a novel Bregman distance for
matrix factorization problems, which, at the same time, allows for
simple/closed form update steps. Therefore, for non-alternating schemes, such
as the recently introduced Bregman Proximal Gradient (BPG) method and an
inertial variant Convex--Concave Inertial BPG (CoCaIn BPG), convergence of the
whole sequence to a stationary point is proved for Matrix Factorization. In
several experiments, we observe a superior performance of our non-alternating
schemes in terms of speed and objective value at the limit point.Comment: Accepted at NeuRIPS 2019. Paper url:
http://papers.nips.cc/paper/8679-beyond-alternating-updates-for-matrix-factorization-with-inertial-bregman-proximal-gradient-algorithm
- …