Feature learning for information-extreme classifier

The feature learning algorithm for information-extreme classifier by clustering of Fast Retina Keypoint binary descriptor, calculated for local features, and usage of spatial pyramid kernel for increasing noise immunity and informativeness of feature representation are considered. Proposed a method of parameters optimization for feature extractor and decision rules based on multi-level coarse features coding using information criterion and population-based search algorithm

Feature learning for information-extreme classifier

The feature learning algorithm for information-extreme classifier by clustering of Fast Retina Keypoint binary descriptor, calculated for local features, and usage of spatial pyramid kernel for increasing noise immunity and informativeness of feature representation are considered. Proposed a method of parameters optimization for feature extractor and decision rules based on multi-level coarse features coding using information criterion and population-based search algorithm

Feature learning for information-extreme classifier

The feature learning algorithm for information-extreme classifier by clustering of Fast Retina Keypoint binary descriptor, calculated for local features, and usage of spatial pyramid kernel for increasing noise immunity and informativeness of feature representation are considered. Proposed a method of parameters optimization for feature extractor and decision rules based on multi-level coarse features coding using information criterion and population-based search algorithm

Low-Rank and Sparse Decomposition for Hyperspectral Image Enhancement and Clustering

In this dissertation, some new algorithms are developed for hyperspectral imaging analysis enhancement. Tensor data format is applied in hyperspectral dataset sparse and low-rank decomposition, which could enhance the classification and detection performance. And multi-view learning technique is applied in hyperspectral imaging clustering. Furthermore, kernel version of multi-view learning technique has been proposed, which could improve clustering performance. Most of low-rank and sparse decomposition algorithms are based on matrix data format for HSI analysis. As HSI contains high spectral dimensions, tensor based extended low-rank and sparse decomposition (TELRSD) is proposed in this dissertation for better performance of HSI classification with low-rank tensor part, and HSI detection with sparse tensor part. With this tensor based method, HSI is processed in 3D data format, and information between spectral bands and pixels maintain integrated during decomposition process. This proposed algorithm is compared with other state-of-art methods. And the experiment results show that TELRSD has the best performance among all those comparison algorithms. HSI clustering is an unsupervised task, which aims to group pixels into different groups without labeled information. Low-rank sparse subspace clustering (LRSSC) is the most popular algorithms for this clustering task. The spatial-spectral based multi-view low-rank sparse subspace clustering (SSMLC) algorithms is proposed in this dissertation, which extended LRSSC with multi-view learning technique. In this algorithm, spectral and spatial views are created to generate multi-view dataset of HSI, where spectral partition, morphological component analysis (MCA) and principle component analysis (PCA) are applied to create others views. Furthermore, kernel version of SSMLC (k-SSMLC) also has been investigated. The performance of SSMLC and k-SSMLC are compared with sparse subspace clustering (SSC), low-rank sparse subspace clustering (LRSSC), and spectral-spatial sparse subspace clustering (S4C). It has shown that SSMLC could improve the performance of LRSSC, and k-SSMLC has the best performance. The spectral clustering has been proved that it equivalent to non-negative matrix factorization (NMF) problem. In this case, NMF could be applied to the clustering problem. In order to include local and nonlinear features in data source, orthogonal NMF (ONMF), graph-regularized NMF (GNMF) and kernel NMF (k-NMF) has been proposed for better clustering performance. The non-linear orthogonal graph NMF combine both kernel, orthogonal and graph constraints in NMF (k-OGNMF), which push up the clustering performance further. In the HSI domain, kernel multi-view based orthogonal graph NMF (k-MOGNMF) is applied for subspace clustering, where k-OGNMF is extended with multi-view algorithm, and it has better performance and computation efficiency

Multiple Kernel Driven Clustering With Locally Consistent and Selfish Graph in Industrial IoT

[EN] In the cognitive computing of intelligent industrial Internet of Things, clustering is a fundamental machine learning problem to exploit the latent data relationships. To overcome the challenge of kernel choice for nonlinear clustering tasks, multiple kernel clustering (MKC) has attracted intensive attention. However, existing graph-based MKC methods mainly aim to learn a consensus kernel as well as an affinity graph from multiple candidate kernels, which cannot fully exploit the latent graph information. In this article, we propose a novel pure graph-based MKC method. Specifically, a new graph model is proposed to preserve the local manifold structure of the data in kernel space so as to learn multiple candidate graphs. Afterward, the latent consistency and selfishness of these candidate graphs are fully considered. Furthermore, a graph connectivity constraint is introduced to avoid requiring any postprocessing clustering step. Comprehensive experimental results demonstrate the superiority of our method.This work was supported in part by Sichuan Science and Technology Program under Grant 2020ZDZX0014 and Grant 2019ZDZX0119 and in part by the Key Lab of Film and TV Media Technology of Zhejiang Province under Grant 2020E10015.Ren, Z.; Mukherjee, M.; Lloret, J.; Venu, P. (2021). Multiple Kernel Driven Clustering With Locally Consistent and Selfish Graph in Industrial IoT. IEEE Transactions on Industrial Informatics. 17(4):2956-2963. https://doi.org/10.1109/TII.2020.3010357S2956296317

A fast and effective multiple kernel clustering method on incomplete data

Multiple kernel clustering is an unsupervised data analysis method that has been used in various scenarios where data is easy to be collected but hard to be labeled. However, multiple kernel clustering for incomplete data is a critical yet challenging task. Although the existing absent multiple kernel clustering methods have achieved remarkable performance on this task, they may fail when data has a high value-missing rate, and they may easily fall into a local optimum. To address these problems, in this paper, we propose an absent multiple kernel clustering (AMKC) method on incomplete data. The AMKC method first clusters the initialized incomplete data. Then, it constructs a new multiple-kernel-based data space, referred to as K-space, from multiple sources to learn kernel combination coefficients. Finally, it seamlessly integrates an incomplete-kernel-imputation objective, a multiple-kernel-learning objective, and a kernel-clustering objective in order to achieve absent multiple kernel clustering. The three stages in this process are carried out simultaneously until the convergence condition is met. Experiments on six datasets with various characteristics demonstrate that the kernel imputation and clustering performance of the proposed method is significantly better than state-of-the-art competitors. Meanwhile, the proposed method gains fast convergence speed

Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks

© 2021 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.[EN] Next-generation wireless networks are witnessing an increasing number of clustering applications, and produce a large amount of non-linear and unlabeled data. In some degree, single kernel methods face the challenging problem of kernel choice. To overcome this problem for non-linear data clustering, multiple kernel graph-based clustering (MKGC) has attracted intense attention in recent years. However, existing MKGC methods suffer from two common problems: (1) they mainly aim to learn a consensus kernel from multiple candidate kernels, slight affinity graph learning, such that cannot fully exploit the underlying graph structure of non-linear data; (2) they disregard the high-order correlations between all base kernels, which cannot fully capture the consistent and complementary information of all kernels. In this paper, we propose a novel non-negative matrix factorization (NMF) tailored graph tensor MKGC method for non-linear data clustering, namely TMKGC. Specifically, TMKGC integrates NMF and graph learning together in kernel space so as to learn multiple candidate affinity graphs. Afterwards, the high-order structure information of all candidate graphs is captured in a 3-order tensor kernel space by introducing tensor singular value decomposition based tensor nuclear norm, such that an optimal affinity graph can be obtained subsequently. Based on the alternating direction method of multipliers, the effective local and distributed solvers are elaborated to solve the proposed objective function. Extensive experiments have demonstrated the superiority of TMKGC compared to the state-of-the-art MKGC methods.This work was supported in part by the Sichuan Science and Technology Program under Grant 2019ZDZX0043 and Grant 2020ZDZX0014, in part by the Key Laboratory of Film and TV Media Technology of Zhejiang Province under Grant 2020E10015, in part by the Natural Science Foundation of Chongqing under Grant cstc2020jcyj-msxmX0473, in part by the Scientific Research Fund of Sichuan Provincial Education Department under Grant 17ZB0441, in part by the Scientific Research Fund of Southwest University of Science and Technology under Grant 17zx7137, and in part by the Academy of Finland projects CARMA and SMARTER.Ren, Z.; Mukherjee, M.; Bennis, M.; Lloret, J. (2021). Multikernel Clustering via Non-Negative Matrix Factorization Tailored Graph Tensor Over Distributed Networks. IEEE Journal on Selected Areas in Communications. 39(7):1946-1956. https://doi.org/10.1109/JSAC.2020.3041396S1946195639

Unsupervised and semi-supervised fuzzy clustering with multiple kernels.

For real-world clustering tasks, the input data is typically not easily separable due to the highly complex data structure or when clusters vary in size, density and shape. Recently, kernel-based clustering has been proposed to perform clustering in a higher-dimensional feature space spanned by embedding maps and corresponding kernel functions. Although good results were obtained using the Gaussian kernel function, its performance depends on the selection of the scaling parameter among an extensive range of possibilities. This step is often heavily influenced by prior knowledge about the data and by the patterns we expect to discover. Unfortunately, it is often unclear which kernels are more suitable for a particular task. The problem is aggravated for many real-world clustering applications, in which the distributions of the different clusters in the feature space exhibit large variations. Thus, in the absence of a priori knowledge, a single kernel selected from a predefined group is sometimes insufficient to represent the data. One way to learn optimal scaling parameters is through an exhaustive search of one optimal scaling parameter for each cluster. However, this approach is not practical since it is computationally expensive, especially when the data includes a large number of clusters and when the dynamic range of possible values of the scaling parameters is large. Moreover, the evaluation of the resulting partition in order to select the optimal parameters is not an easy task. To overcome the above drawbacks, we introduce two novel fuzzy clustering techniques that use Multiple Kernel Learning to provide an elegant solution for parameter selection. The Fuzzy C-Means with Multiple Kernels algorithm (FCMK) simultaneously finds the optimal partition and the cluster-dependent kernel combination weights that reflect the intrinsic structure of the data. The Relational Fuzzy Clustering with Multiple Kernels (RFCMK) learns the kernel combination weights by optimizing the relational dissimilarities. Consequently, the learned kernel combination weights reflect the relative density, size, and position of each cluster with respect to the other clusters. We also extended FCMK and RFCMK to the semi-supervised paradigms. We show that the incorporation of prior knowledge in the unsupervised clustering task in the form of a small set of constraints on which instances should or should not reside in the same cluster, guides the unsupervised approaches to a better partitioning of the data and avoid local minima, especially for high dimensional real world data. All of the proposed algorithms are optimized iteratively by dynamically updating the partition and the kernel combination weights in each iteration. This makes these algorithms simple and fast. Moreover, our algorithms are formulated to work on both vector and relational data. This makes them applicable to data where objects cannot be represented by vectors or when clusters of similar objects cannot be represented efficiently by a single prototype. We also introduced two relational fuzzy clustering with multiple kernel algorithms for large data to deal with the scalability issue of RFCMK. The random sample and extend RFCMK (rseRFCMK) computes cluster prototypes from a smaller sample of randomly selected objects, and then extends the partition to the remainder of the data. The single pass RFCMK (spRFCMK) sequentially loads manageable sized chunks, clustering the chunks in a single pass, and then combining the results from each chunk. Our extensive experiments show that RFCMK and SS-RFCMK outperform existing algorithms. In particular, we show that when data include clusters with various intrinsic structures and densities, learning kernel weights that vary over clusters is crucial in obtaining a good partition