A Clustering-guided Contrastive Fusion for Multi-view Representation Learning

The past two decades have seen increasingly rapid advances in the field of multi-view representation learning due to it extracting useful information from diverse domains to facilitate the development of multi-view applications. However, the community faces two challenges: i) how to learn robust representations from a large amount of unlabeled data to against noise or incomplete views setting, and ii) how to balance view consistency and complementary for various downstream tasks. To this end, we utilize a deep fusion network to fuse view-specific representations into the view-common representation, extracting high-level semantics for obtaining robust representation. In addition, we employ a clustering task to guide the fusion network to prevent it from leading to trivial solutions. For balancing consistency and complementary, then, we design an asymmetrical contrastive strategy that aligns the view-common representation and each view-specific representation. These modules are incorporated into a unified method known as CLustering-guided cOntrastiVE fusioN (CLOVEN). We quantitatively and qualitatively evaluate the proposed method on five datasets, demonstrating that CLOVEN outperforms 11 competitive multi-view learning methods in clustering and classification. In the incomplete view scenario, our proposed method resists noise interference better than those of our competitors. Furthermore, the visualization analysis shows that CLOVEN can preserve the intrinsic structure of view-specific representation while also improving the compactness of view-commom representation. Our source code will be available soon at https://github.com/guanzhou-ke/cloven.Comment: 13 pages, 9 figure

Learning Common Semantics via Optimal Transport for Contrastive Multi-view Clustering

Multi-view clustering aims to learn discriminativerepresentations from multi-view data. Although existing methodsshow impressive performance by leveraging contrastive learningto tackle the representation gap between every two views,they share the common limitation of not performing semanticalignment from a global perspective, resulting in the underminingof semantic patterns in multi-view data. This paper presentsCSOT, namely Common Semantics via Optimal Transport, toboost contrastive multi-view clustering via semantic learning ina common space that integrates all views. Through optimaltransport, the samples in multiple views are mapped to thejoint clusters which represent the multi-view semantic patternsin the common space. With the semantic assignment derivedfrom the optimal transport plan, we design a semantic learningmodule where the soft assignment vector works as a globalsupervision to enforce the model to learn consistent semanticsamong all views. Moreover, we propose a semantic-aware reweighting strategy to treat samples differently according to theirsemantic significance, which improves the effectiveness of crossview contrastive representation learning. Extensive experimentalresults demonstrate that CSOT achieves the state-of-the-artclustering performance