12 research outputs found
Hierarchical Mutual Information Analysis: Towards Multi-view Clustering in The Wild
Multi-view clustering (MVC) can explore common semantics from unsupervised
views generated by different sources, and thus has been extensively used in
applications of practical computer vision. Due to the spatio-temporal
asynchronism, multi-view data often suffer from view missing and are unaligned
in real-world applications, which makes it difficult to learn consistent
representations. To address the above issues, this work proposes a deep MVC
framework where data recovery and alignment are fused in a hierarchically
consistent way to maximize the mutual information among different views and
ensure the consistency of their latent spaces. More specifically, we first
leverage dual prediction to fill in missing views while achieving the
instance-level alignment, and then take the contrastive reconstruction to
achieve the class-level alignment. To the best of our knowledge, this could be
the first successful attempt to handle the missing and unaligned data problem
separately with different learning paradigms. Extensive experiments on public
datasets demonstrate that our method significantly outperforms state-of-the-art
methods on multi-view clustering even in the cases of view missing and
unalignment
A Concise yet Effective model for Non-Aligned Incomplete Multi-view and Missing Multi-label Learning
In reality, learning from multi-view multi-label data inevitably confronts
three challenges: missing labels, incomplete views, and non-aligned views.
Existing methods mainly concern the first two and commonly need multiple
assumptions to attack them, making even state-of-the-arts involve at least two
explicit hyper-parameters such that model selection is quite difficult. More
roughly, they will fail in handling the third challenge, let alone addressing
the three jointly. In this paper, we aim at meeting these under the least
assumption by building a concise yet effective model with just one
hyper-parameter. To ease insufficiency of available labels, we exploit not only
the consensus of multiple views but also the global and local structures hidden
among multiple labels. Specifically, we introduce an indicator matrix to tackle
the first two challenges in a regression form while aligning the same
individual labels and all labels of different views in a common label space to
battle the third challenge. In aligning, we characterize the global and local
structures of multiple labels to be high-rank and low-rank, respectively.
Subsequently, an efficient algorithm with linear time complexity in the number
of samples is established. Finally, even without view-alignment, our method
substantially outperforms state-of-the-arts with view-alignment on five real
datasets.Comment: 15 pages, 7 figure
Information Recovery-Driven Deep Incomplete Multiview Clustering Network
Incomplete multi-view clustering is a hot and emerging topic. It is well
known that unavoidable data incompleteness greatly weakens the effective
information of multi-view data. To date, existing incomplete multi-view
clustering methods usually bypass unavailable views according to prior missing
information, which is considered as a second-best scheme based on evasion.
Other methods that attempt to recover missing information are mostly applicable
to specific two-view datasets. To handle these problems, in this paper, we
propose an information recovery-driven deep incomplete multi-view clustering
network, termed as RecFormer. Concretely, a two-stage autoencoder network with
the self-attention structure is built to synchronously extract high-level
semantic representations of multiple views and recover the missing data.
Besides, we develop a recurrent graph reconstruction mechanism that cleverly
leverages the restored views to promote the representation learning and the
further data reconstruction. Visualization of recovery results are given and
sufficient experimental results confirm that our RecFormer has obvious
advantages over other top methods.Comment: Accepted by TNNLS 2023. Please contact me if you have any questions:
[email protected]. The code is available at:
https://github.com/justsmart/RecForme
Fast Continual Multi-View Clustering with Incomplete Views
Multi-view clustering (MVC) has gained broad attention owing to its capacity
to exploit consistent and complementary information across views. This paper
focuses on a challenging issue in MVC called the incomplete continual data
problem (ICDP). In specific, most existing algorithms assume that views are
available in advance and overlook the scenarios where data observations of
views are accumulated over time. Due to privacy considerations or memory
limitations, previous views cannot be stored in these situations. Some works
are proposed to handle it, but all fail to address incomplete views. Such an
incomplete continual data problem (ICDP) in MVC is tough to solve since
incomplete information with continual data increases the difficulty of
extracting consistent and complementary knowledge among views. We propose Fast
Continual Multi-View Clustering with Incomplete Views (FCMVC-IV) to address it.
Specifically, it maintains a consensus coefficient matrix and updates knowledge
with the incoming incomplete view rather than storing and recomputing all the
data matrices. Considering that the views are incomplete, the newly collected
view might contain samples that have yet to appear; two indicator matrices and
a rotation matrix are developed to match matrices with different dimensions.
Besides, we design a three-step iterative algorithm to solve the resultant
problem in linear complexity with proven convergence. Comprehensive experiments
on various datasets show the superiority of FCMVC-IV
Joint Projection Learning and Tensor Decomposition Based Incomplete Multi-view Clustering
Incomplete multi-view clustering (IMVC) has received increasing attention
since it is often that some views of samples are incomplete in reality. Most
existing methods learn similarity subgraphs from original incomplete multi-view
data and seek complete graphs by exploring the incomplete subgraphs of each
view for spectral clustering. However, the graphs constructed on the original
high-dimensional data may be suboptimal due to feature redundancy and noise.
Besides, previous methods generally ignored the graph noise caused by the
inter-class and intra-class structure variation during the transformation of
incomplete graphs and complete graphs. To address these problems, we propose a
novel Joint Projection Learning and Tensor Decomposition Based method (JPLTD)
for IMVC. Specifically, to alleviate the influence of redundant features and
noise in high-dimensional data, JPLTD introduces an orthogonal projection
matrix to project the high-dimensional features into a lower-dimensional space
for compact feature learning.Meanwhile, based on the lower-dimensional space,
the similarity graphs corresponding to instances of different views are
learned, and JPLTD stacks these graphs into a third-order low-rank tensor to
explore the high-order correlations across different views. We further consider
the graph noise of projected data caused by missing samples and use a
tensor-decomposition based graph filter for robust clustering.JPLTD decomposes
the original tensor into an intrinsic tensor and a sparse tensor. The intrinsic
tensor models the true data similarities. An effective optimization algorithm
is adopted to solve the JPLTD model. Comprehensive experiments on several
benchmark datasets demonstrate that JPLTD outperforms the state-of-the-art
methods. The code of JPLTD is available at https://github.com/weilvNJU/JPLTD.Comment: IEEE Transactions on Neural Networks and Learning Systems, 202
Localized Sparse Incomplete Multi-view Clustering
Incomplete multi-view clustering, which aims to solve the clustering problem
on the incomplete multi-view data with partial view missing, has received more
and more attention in recent years. Although numerous methods have been
developed, most of the methods either cannot flexibly handle the incomplete
multi-view data with arbitrary missing views or do not consider the negative
factor of information imbalance among views. Moreover, some methods do not
fully explore the local structure of all incomplete views. To tackle these
problems, this paper proposes a simple but effective method, named localized
sparse incomplete multi-view clustering (LSIMVC). Different from the existing
methods, LSIMVC intends to learn a sparse and structured consensus latent
representation from the incomplete multi-view data by optimizing a sparse
regularized and novel graph embedded multi-view matrix factorization model.
Specifically, in such a novel model based on the matrix factorization, a l1
norm based sparse constraint is introduced to obtain the sparse low-dimensional
individual representations and the sparse consensus representation. Moreover, a
novel local graph embedding term is introduced to learn the structured
consensus representation. Different from the existing works, our local graph
embedding term aggregates the graph embedding task and consensus representation
learning task into a concise term. Furthermore, to reduce the imbalance factor
of incomplete multi-view learning, an adaptive weighted learning scheme is
introduced to LSIMVC. Finally, an efficient optimization strategy is given to
solve the optimization problem of our proposed model. Comprehensive
experimental results performed on six incomplete multi-view databases verify
that the performance of our LSIMVC is superior to the state-of-the-art IMC
approaches. The code is available in https://github.com/justsmart/LSIMVC.Comment: Published in IEEE Transactions on Multimedia (TMM). The code is
available at Github https://github.com/justsmart/LSIMV