503 research outputs found
A Nonlinear Orthogonal Non-Negative Matrix Factorization Approach to Subspace Clustering
A recent theoretical analysis shows the equivalence between non-negative
matrix factorization (NMF) and spectral clustering based approach to subspace
clustering. As NMF and many of its variants are essentially linear, we
introduce a nonlinear NMF with explicit orthogonality and derive general
kernel-based orthogonal multiplicative update rules to solve the subspace
clustering problem. In nonlinear orthogonal NMF framework, we propose two
subspace clustering algorithms, named kernel-based non-negative subspace
clustering KNSC-Ncut and KNSC-Rcut and establish their connection with spectral
normalized cut and ratio cut clustering. We further extend the nonlinear
orthogonal NMF framework and introduce a graph regularization to obtain a
factorization that respects a local geometric structure of the data after the
nonlinear mapping. The proposed NMF-based approach to subspace clustering takes
into account the nonlinear nature of the manifold, as well as its intrinsic
local geometry, which considerably improves the clustering performance when
compared to the several recently proposed state-of-the-art methods
Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering
Dimensionality reduction techniques play an essential role in data analytics,
signal processing and machine learning. Dimensionality reduction is usually
performed in a preprocessing stage that is separate from subsequent data
analysis, such as clustering or classification. Finding reduced-dimension
representations that are well-suited for the intended task is more appealing.
This paper proposes a joint factor analysis and latent clustering framework,
which aims at learning cluster-aware low-dimensional representations of matrix
and tensor data. The proposed approach leverages matrix and tensor
factorization models that produce essentially unique latent representations of
the data to unravel latent cluster structure -- which is otherwise obscured
because of the freedom to apply an oblique transformation in latent space. At
the same time, latent cluster structure is used as prior information to enhance
the performance of factorization. Specific contributions include several
custom-built problem formulations, corresponding algorithms, and discussion of
associated convergence properties. Besides extensive simulations, real-world
datasets such as Reuters document data and MNIST image data are also employed
to showcase the effectiveness of the proposed approaches
Global and Local Structure Preserving Sparse Subspace Learning: An Iterative Approach to Unsupervised Feature Selection
As we aim at alleviating the curse of high-dimensionality, subspace learning
is becoming more popular. Existing approaches use either information about
global or local structure of the data, and few studies simultaneously focus on
global and local structures as the both of them contain important information.
In this paper, we propose a global and local structure preserving sparse
subspace learning (GLoSS) model for unsupervised feature selection. The model
can simultaneously realize feature selection and subspace learning. In
addition, we develop a greedy algorithm to establish a generic combinatorial
model, and an iterative strategy based on an accelerated block coordinate
descent is used to solve the GLoSS problem. We also provide whole iterate
sequence convergence analysis of the proposed iterative algorithm. Extensive
experiments are conducted on real-world datasets to show the superiority of the
proposed approach over several state-of-the-art unsupervised feature selection
approaches.Comment: 32 page, 6 figures and 60 reference
Supervised Nonnegative Matrix Factorization to Predict ICU Mortality Risk
ICU mortality risk prediction is a tough yet important task. On one hand, due
to the complex temporal data collected, it is difficult to identify the
effective features and interpret them easily; on the other hand, good
prediction can help clinicians take timely actions to prevent the mortality.
These correspond to the interpretability and accuracy problems. Most existing
methods lack of the interpretability, but recently Subgraph Augmented
Nonnegative Matrix Factorization (SANMF) has been successfully applied to time
series data to provide a path to interpret the features well. Therefore, we
adopted this approach as the backbone to analyze the patient data. One
limitation of the raw SANMF method is its poor prediction ability due to its
unsupervised nature. To deal with this problem, we proposed a supervised SANMF
algorithm by integrating the logistic regression loss function into the NMF
framework and solved it with an alternating optimization procedure. We used the
simulation data to verify the effectiveness of this method, and then we applied
it to ICU mortality risk prediction and demonstrated its superiority over other
conventional supervised NMF methods.Comment: 7 Pages, 2 figure
Long-Term Identity-Aware Multi-Person Tracking for Surveillance Video Summarization
Multi-person tracking plays a critical role in the analysis of surveillance
video. However, most existing work focus on shorter-term (e.g. minute-long or
hour-long) video sequences. Therefore, we propose a multi-person tracking
algorithm for very long-term (e.g. month-long) multi-camera surveillance
scenarios. Long-term tracking is challenging because 1) the apparel/appearance
of the same person will vary greatly over multiple days and 2) a person will
leave and re-enter the scene numerous times. To tackle these challenges, we
leverage face recognition information, which is robust to apparel change, to
automatically reinitialize our tracker over multiple days of recordings.
Unfortunately, recognized faces are unavailable oftentimes. Therefore, our
tracker propagates identity information to frames without recognized faces by
uncovering the appearance and spatial manifold formed by person detections. We
tested our algorithm on a 23-day 15-camera data set (4,935 hours total), and we
were able to localize a person 53.2% of the time with 69.8% precision. We
further performed video summarization experiments based on our tracking output.
Results on 116.25 hours of video showed that we were able to generate a
reasonable visual diary (i.e. a summary of what a person did) for different
people, thus potentially opening the door to automatic summarization of the
vast amount of surveillance video generated every day
A Survey on Multi-View Clustering
With advances in information acquisition technologies, multi-view data become
ubiquitous. Multi-view learning has thus become more and more popular in
machine learning and data mining fields. Multi-view unsupervised or
semi-supervised learning, such as co-training, co-regularization has gained
considerable attention. Although recently, multi-view clustering (MVC) methods
have been developed rapidly, there has not been a survey to summarize and
analyze the current progress. Therefore, this paper reviews the common
strategies for combining multiple views of data and based on this summary we
propose a novel taxonomy of the MVC approaches. We further discuss the
relationships between MVC and multi-view representation, ensemble clustering,
multi-task clustering, multi-view supervised and semi-supervised learning.
Several representative real-world applications are elaborated. To promote
future development of MVC, we envision several open problems that may require
further investigation and thorough examination.Comment: 17 pages, 4 figure
Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction
Very often data we encounter in practice is a collection of matrices rather
than a single matrix. These multi-block data are naturally linked and hence
often share some common features and at the same time they have their own
individual features, due to the background in which they are measured and
collected. In this study we proposed a new scheme of common and individual
feature analysis (CIFA) that processes multi-block data in a linked way aiming
at discovering and separating their common and individual features. According
to whether the number of common features is given or not, two efficient
algorithms were proposed to extract the common basis which is shared by all
data. Then feature extraction is performed on the common and the individual
spaces separately by incorporating the techniques such as dimensionality
reduction and blind source separation. We also discussed how the proposed CIFA
can significantly improve the performance of classification and clustering
tasks by exploiting common and individual features of samples respectively. Our
experimental results show some encouraging features of the proposed methods in
comparison to the state-of-the-art methods on synthetic and real data.Comment: 13 pages,11 figure
Supervised Dictionary Learning and Sparse Representation-A Review
Dictionary learning and sparse representation (DLSR) is a recent and
successful mathematical model for data representation that achieves
state-of-the-art performance in various fields such as pattern recognition,
machine learning, computer vision, and medical imaging. The original
formulation for DLSR is based on the minimization of the reconstruction error
between the original signal and its sparse representation in the space of the
learned dictionary. Although this formulation is optimal for solving problems
such as denoising, inpainting, and coding, it may not lead to optimal solution
in classification tasks, where the ultimate goal is to make the learned
dictionary and corresponding sparse representation as discriminative as
possible. This motivated the emergence of a new category of techniques, which
is appropriately called supervised dictionary learning and sparse
representation (S-DLSR), leading to more optimal dictionary and sparse
representation in classification tasks. Despite many research efforts for
S-DLSR, the literature lacks a comprehensive view of these techniques, their
connections, advantages and shortcomings. In this paper, we address this gap
and provide a review of the recently proposed algorithms for S-DLSR. We first
present a taxonomy of these algorithms into six categories based on the
approach taken to include label information into the learning of the dictionary
and/or sparse representation. For each category, we draw connections between
the algorithms in this category and present a unified framework for them. We
then provide guidelines for applied researchers on how to represent and learn
the building blocks of an S-DLSR solution based on the problem at hand. This
review provides a broad, yet deep, view of the state-of-the-art methods for
S-DLSR and allows for the advancement of research and development in this
emerging area of research
Robust Multi-subspace Analysis Using Novel Column L0-norm Constrained Matrix Factorization
We study the underlying structure of data (approximately) generated from a
union of independent subspaces. Traditional methods learn only one subspace,
failing to discover the multi-subspace structure, while state-of-the-art
methods analyze the multi-subspace structure using data themselves as the
dictionary, which cannot offer the explicit basis to span each subspace and are
sensitive to errors via an indirect representation. Additionally, they also
suffer from a high computational complexity, being quadratic or cubic to the
sample size. To tackle all these problems, we propose a method, called Matrix
Factorization with Column L0-norm constraint (MFC0), that can simultaneously
learn the basis for each subspace, generate a direct sparse representation for
each data sample, as well as removing errors in the data in an efficient way.
Furthermore, we develop a first-order alternating direction algorithm, whose
computational complexity is linear to the sample size, to stably and
effectively solve the nonconvex objective function and non- smooth l0-norm
constraint of MFC0. Experimental results on both synthetic and real-world
datasets demonstrate that besides the superiority over traditional and
state-of-the-art methods for subspace clustering, data reconstruction, error
correction, MFC0 also shows its uniqueness for multi-subspace basis learning
and direct sparse representation.Comment: 13 pages, 8 figures, 8 table
Feature Weighted Non-negative Matrix Factorization
Non-negative Matrix Factorization (NMF) is one of the most popular techniques
for data representation and clustering, and has been widely used in machine
learning and data analysis. NMF concentrates the features of each sample into a
vector, and approximates it by the linear combination of basis vectors, such
that the low-dimensional representations are achieved. However, in real-world
applications, the features are usually with different importances. To exploit
the discriminative features, some methods project the samples into the subspace
with a transformation matrix, which disturbs the original feature attributes
and neglects the diversity of samples. To alleviate the above problems, we
propose the Feature weighted Non-negative Matrix Factorization (FNMF) in this
paper. The salient properties of FNMF can be summarized as threefold: 1) it
learns the weights of features adaptively according to their importances; 2) it
utilizes multiple feature weighting components to preserve the diversity; 3) it
can be solved efficiently with the suggested optimization algorithm.
Performance on synthetic and real-world datasets demonstrate that the proposed
method obtains the state-of-the-art performance
- …