9 research outputs found
Contrastive Continual Multi-view Clustering with Filtered Structural Fusion
Multi-view clustering thrives in applications where views are collected in
advance by extracting consistent and complementary information among views.
However, it overlooks scenarios where data views are collected sequentially,
i.e., real-time data. Due to privacy issues or memory burden, previous views
are not available with time in these situations. Some methods are proposed to
handle it but are trapped in a stability-plasticity dilemma. In specific, these
methods undergo a catastrophic forgetting of prior knowledge when a new view is
attained. Such a catastrophic forgetting problem (CFP) would cause the
consistent and complementary information hard to get and affect the clustering
performance. To tackle this, we propose a novel method termed Contrastive
Continual Multi-view Clustering with Filtered Structural Fusion (CCMVC-FSF).
Precisely, considering that data correlations play a vital role in clustering
and prior knowledge ought to guide the clustering process of a new view, we
develop a data buffer with fixed size to store filtered structural information
and utilize it to guide the generation of a robust partition matrix via
contrastive learning. Furthermore, we theoretically connect CCMVC-FSF with
semi-supervised learning and knowledge distillation. Extensive experiments
exhibit the excellence of the proposed method
Multiple Kernel Driven Clustering With Locally Consistent and Selfish Graph in Industrial IoT
[EN] In the cognitive computing of intelligent industrial Internet of Things, clustering is a fundamental machine learning problem to exploit the latent data relationships. To overcome the challenge of kernel choice for nonlinear clustering tasks, multiple kernel clustering (MKC) has attracted intensive attention. However, existing graph-based MKC methods mainly aim to learn a consensus kernel as well as an affinity graph from multiple candidate kernels, which cannot fully exploit the latent graph information. In this article, we propose a novel pure graph-based MKC method. Specifically, a new graph model is proposed to preserve the local manifold structure of the data in kernel space so as to learn multiple candidate graphs. Afterward, the latent consistency and selfishness of these candidate graphs are fully considered. Furthermore, a graph connectivity constraint is introduced to avoid requiring any postprocessing clustering step. Comprehensive experimental results demonstrate the superiority of our method.This work was supported in part by Sichuan Science and Technology Program under Grant 2020ZDZX0014 and Grant 2019ZDZX0119 and in part by the Key Lab of Film and TV Media Technology of Zhejiang Province under Grant 2020E10015.Ren, Z.; Mukherjee, M.; Lloret, J.; Venu, P. (2021). Multiple Kernel Driven Clustering With Locally Consistent and Selfish Graph in Industrial IoT. IEEE Transactions on Industrial Informatics. 17(4):2956-2963. https://doi.org/10.1109/TII.2020.3010357S2956296317
One-step Multi-view Clustering with Diverse Representation
Multi-view clustering has attracted broad attention due to its capacity to
utilize consistent and complementary information among views. Although
tremendous progress has been made recently, most existing methods undergo high
complexity, preventing them from being applied to large-scale tasks. Multi-view
clustering via matrix factorization is a representative to address this issue.
However, most of them map the data matrices into a fixed dimension, which
limits the expressiveness of the model. Moreover, a range of methods suffer
from a two-step process, i.e., multimodal learning and the subsequent
-means, inevitably causing a sub-optimal clustering result. In light of
this, we propose a one-step multi-view clustering with diverse representation
method, which incorporates multi-view learning and -means into a unified
framework. Specifically, we first project original data matrices into various
latent spaces to attain comprehensive information and auto-weight them in a
self-supervised manner. Then we directly use the information matrices under
diverse dimensions to obtain consensus discrete clustering labels. The unified
work of representation learning and clustering boosts the quality of the final
results. Furthermore, we develop an efficient optimization algorithm to solve
the resultant problem with proven convergence. Comprehensive experiments on
various datasets demonstrate the promising clustering performance of our
proposed method
Fast Continual Multi-View Clustering with Incomplete Views
Multi-view clustering (MVC) has gained broad attention owing to its capacity
to exploit consistent and complementary information across views. This paper
focuses on a challenging issue in MVC called the incomplete continual data
problem (ICDP). In specific, most existing algorithms assume that views are
available in advance and overlook the scenarios where data observations of
views are accumulated over time. Due to privacy considerations or memory
limitations, previous views cannot be stored in these situations. Some works
are proposed to handle it, but all fail to address incomplete views. Such an
incomplete continual data problem (ICDP) in MVC is tough to solve since
incomplete information with continual data increases the difficulty of
extracting consistent and complementary knowledge among views. We propose Fast
Continual Multi-View Clustering with Incomplete Views (FCMVC-IV) to address it.
Specifically, it maintains a consensus coefficient matrix and updates knowledge
with the incoming incomplete view rather than storing and recomputing all the
data matrices. Considering that the views are incomplete, the newly collected
view might contain samples that have yet to appear; two indicator matrices and
a rotation matrix are developed to match matrices with different dimensions.
Besides, we design a three-step iterative algorithm to solve the resultant
problem in linear complexity with proven convergence. Comprehensive experiments
on various datasets show the superiority of FCMVC-IV
Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks
© 2021 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.[EN] Next-generation wireless networks are witnessing an increasing number of clustering applications, and produce a large amount of non-linear and unlabeled data. In some degree, single kernel methods face the challenging problem of kernel choice. To overcome this problem for non-linear data clustering, multiple kernel graph-based clustering (MKGC) has attracted intense attention in recent years. However, existing MKGC methods suffer from two common problems: (1) they mainly aim to learn a consensus kernel from multiple candidate kernels, slight affinity graph learning, such that cannot fully exploit the underlying graph structure of non-linear data; (2) they disregard the high-order correlations between all base kernels, which cannot fully capture the consistent and complementary information of all kernels. In this paper, we propose a novel non-negative matrix factorization (NMF) tailored graph tensor MKGC method for non-linear data clustering, namely TMKGC. Specifically, TMKGC integrates NMF and graph learning together in kernel space so as to learn multiple candidate affinity graphs. Afterwards, the high-order structure information of all candidate graphs is captured in a 3-order tensor kernel space by introducing tensor singular value decomposition based tensor nuclear norm, such that an optimal affinity graph can be obtained subsequently. Based on the alternating direction method of multipliers, the effective local and distributed solvers are elaborated to solve the proposed objective function. Extensive experiments have demonstrated the superiority of TMKGC compared to the state-of-the-art MKGC methods.This work was supported in part by the Sichuan Science and Technology Program under Grant 2019ZDZX0043 and Grant 2020ZDZX0014, in part by the Key Laboratory of Film and TV Media Technology of Zhejiang Province under Grant 2020E10015, in part by the Natural Science Foundation of Chongqing under Grant cstc2020jcyj-msxmX0473, in part by the Scientific Research Fund of Sichuan Provincial Education Department under Grant 17ZB0441, in part by the Scientific Research Fund of Southwest University of Science and Technology under Grant 17zx7137, and in part by the Academy of Finland projects CARMA and SMARTER.Ren, Z.; Mukherjee, M.; Bennis, M.; Lloret, J. (2021). Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks. IEEE Journal on Selected Areas in Communications. 39(7):1946-1956. https://doi.org/10.1109/JSAC.2020.3041396S1946195639
Towards Generalized Multi-stage Clustering: Multi-view Self-distillation
Existing multi-stage clustering methods independently learn the salient
features from multiple views and then perform the clustering task.
Particularly, multi-view clustering (MVC) has attracted a lot of attention in
multi-view or multi-modal scenarios. MVC aims at exploring common semantics and
pseudo-labels from multiple views and clustering in a self-supervised manner.
However, limited by noisy data and inadequate feature learning, such a
clustering paradigm generates overconfident pseudo-labels that mis-guide the
model to produce inaccurate predictions. Therefore, it is desirable to have a
method that can correct this pseudo-label mistraction in multi-stage clustering
to avoid the bias accumulation. To alleviate the effect of overconfident
pseudo-labels and improve the generalization ability of the model, this paper
proposes a novel multi-stage deep MVC framework where multi-view
self-distillation (DistilMVC) is introduced to distill dark knowledge of label
distribution. Specifically, in the feature subspace at different hierarchies,
we explore the common semantics of multiple views through contrastive learning
and obtain pseudo-labels by maximizing the mutual information between views.
Additionally, a teacher network is responsible for distilling pseudo-labels
into dark knowledge, supervising the student network and improving its
predictive capabilities to enhance the robustness. Extensive experiments on
real-world multi-view datasets show that our method has better clustering
performance than state-of-the-art methods