A survey of kernel and spectral methods for clustering

Clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines. The focus of this paper is the partitioning clustering problem with a special interest in two recent approaches: kernel and spectral methods. The aim of this paper is to present a survey of kernel and spectral clustering methods, two approaches able to produce nonlinear separating hypersurfaces between clusters. The presented kernel clustering methods are the kernel version of many classical clustering algorithms, e.g., K-means, SOM and neural gas. Spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized. An explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation. Besides, fuzzy kernel clustering methods are presented as extensions of kernel K-means clustering algorithm. (C) 2007 Pattem Recognition Society. Published by Elsevier Ltd. All rights reserved

Spectral clustering and fuzzy similarity measure for images segmentation

In image segmentation algorithms using spectral clustering, due to the size of the images, the computational load for the construction of the similarity matrix and the solution to the eigenvalue problem for the Laplacian matrix is high. Furthermore, the Gaussian kernel similarity measure is the most used, but it presents problems with irregular data distributions. This work proposes to perform a pre-segmentation or decimation by superpixels with the Simple Linear Iterative Clustering algorithm to reduce the computational cost, and to build the similarity matrix with a fuzzy measure based on the Fuzzy C-Means classifier, providing the algorithm a greater robustness against images with complex distributions and by spectral clustering the final segmentation is determined. Experimentally, it was found that the proposed approach obtains adequate segmentations, good clustering results and a comparable precision with respect to five algorithms; measuring performance under four determined validation metrics.En los algoritmos de segmentación de imágenes mediante agrupamiento espectral, debido al tamaño de las imágenes, la carga computacional para la construcción de la matriz de similitud y la solución al problema de valores propios para la matriz laplaciana son altos. Además, la medida de similitud más utilizada es el kernel gaussiano, el cual presenta problemas con distribuciones de datos irregulares. Este trabajo propone realizar una presegmentación o diezmado mediante superpíxeles con el algoritmo Simple Linear Iterative Clustering, para disminuir el costo computacional y construir la matriz de similaridad con una medida difusa basada en el clasificador Fuzzy C-Means, que proporciona al algoritmo una mayor robustez frente a imágenes con distribuciones complejas; mediante agrupamiento espectral se determina la segmentación final. Experimentalmente, se comprobó que el enfoque propuesto obtiene segmentaciones adecuadas, buenos resultados de agrupamiento y una precisión comparable respecto a cinco algoritmos, midiendo el desempeño bajo cuatro métricas de validación

SPATIAL-SPECTRAL FUZZY K-MEANS CLUSTERING FOR REMOTE SENSING IMAGE SEGMENTATION

Spectral clustering is a clustering method based on algebraic graph theory. The clustering effect by using spectral method depends heavily on the description of similarity between instances of the datasets. Althought, spectral clustering has been significant interest in recent times, but the raw spectral clustering is often based on Euclidean distance, but it is impossible to accurately reflect the complexity of the data. Despite having a well-defined mathematical framework, good performance and simplicity, it suffers from several drawbacks, such as it is unable to determine a reasonable cluster number, sensitive to initial condition and not robust to outliers. In this paper, we present a new approach named spatial-spectral fuzzy clustering which combines spectral clustering and fuzzy clustering with spatial information into a unified framework to solve these problems, the paper consists of three main steps: Step 1, calculate the spatial information value of the pixels, step 2 applies the spectral clustering algorithm to change the data space from the color space to the new space and step 3 clusters the data in new data space by fuzzy clustering algorithm. Experimental results on the remote sensing image were evaluated based on a number of indicators, such as IQI, MSE, DI and CSI, show that it can improve the clustering accuracy and avoid falling into local optimum.

A new kernel method for hyperspectral image feature extraction

Hyperspectral image provides abundant spectral information for remote discrimination of subtle differences in ground covers. However, the increasing spectral dimensions, as well as the information redundancy, make the analysis and interpretation of hyperspectral images a challenge. Feature extraction is a very important step for hyperspectral image processing. Feature extraction methods aim at reducing the dimension of data, while preserving as much information as possible. Particularly, nonlinear feature extraction methods (e.g. kernel minimum noise fraction (KMNF) transformation) have been reported to benefit many applications of hyperspectral remote sensing, due to their good preservation of high-order structures of the original data. However, conventional KMNF or its extensions have some limitations on noise fraction estimation during the feature extraction, and this leads to poor performances for post-applications. This paper proposes a novel nonlinear feature extraction method for hyperspectral images. Instead of estimating noise fraction by the nearest neighborhood information (within a sliding window), the proposed method explores the use of image segmentation. The approach benefits both noise fraction estimation and information preservation, and enables a significant improvement for classification. Experimental results on two real hyperspectral images demonstrate the efficiency of the proposed method. Compared to conventional KMNF, the improvements of the method on two hyperspectral image classification are 8 and 11%. This nonlinear feature extraction method can be also applied to other disciplines where high-dimensional data analysis is required

Efficient Semidefinite Spectral Clustering via Lagrange Duality

We propose an efficient approach to semidefinite spectral clustering (SSC), which addresses the Frobenius normalization with the positive semidefinite (p.s.d.) constraint for spectral clustering. Compared with the original Frobenius norm approximation based algorithm, the proposed algorithm can more accurately find the closest doubly stochastic approximation to the affinity matrix by considering the p.s.d. constraint. In this paper, SSC is formulated as a semidefinite programming (SDP) problem. In order to solve the high computational complexity of SDP, we present a dual algorithm based on the Lagrange dual formalization. Two versions of the proposed algorithm are proffered: one with less memory usage and the other with faster convergence rate. The proposed algorithm has much lower time complexity than that of the standard interior-point based SDP solvers. Experimental results on both UCI data sets and real-world image data sets demonstrate that 1) compared with the state-of-the-art spectral clustering methods, the proposed algorithm achieves better clustering performance; and 2) our algorithm is much more efficient and can solve larger-scale SSC problems than those standard interior-point SDP solvers.Comment: 13 page