Clustering Of Complex Shaped Data Sets Via Kohonen Maps And Mathematical Morphology

Clustering is the process of discovering groups within the data, based on similarities, with a minimal, if any, knowledge of their structure. The self-organizing (or Kohonen) map (SOM) is one of the best known neural network algorithms. It has been widely studied as a software tool for visualization of high-dimensional data. Important features include information compression while preserving topological and metric relationship of the primary data items. Although Kohonen maps had been applied for clustering data, usually the researcher sets the number of neurons equal to the expected number of clusters, or manually segments a two-dimensional map using some a priori knowledge of the data. This paper proposes techniques for automatic partitioning and labeling SOM networks in clusters of neurons that may be used to represent the data clusters. Mathematical morphology operations, such as watershed, are performed on the U-matrix, which is a neuron-distance image. The direct application of watershed leads to an oversegmented image. It is used markers to identify significant clusters and homotopy modification to suppress the others. Markers are automatically found by performing a multi-level scan of connected regions of the U-matrix. Each cluster of neurons is a sub-graph that defines, in the input space, complex and nonparametric geometries which approximately describes the shape of the clusters. The process of map partitioning is extended recursively. Each cluster of neurons gives rise to a new map, which are trained with the subset of data that were classified to it. The algorithm produces dynamically a hierarchical tree of maps, which explains the cluster's structure in levels of granularity. The distributed and multiple prototypes cluster representation enables the discoveries of clusters even in the case when we have two or more non-separable pattern classes.43841627Vinod, V.V., Chaudhury, S., Mukherjee, J., Ghose, S., A connectionist approach for clustering with applications in image analysis (1994) IEEE Trans. Systems, Man & Cybernetics, 24 (3), pp. 356-384Costa, J.A.F., (1999) Automatic classification and data analysis by self-organizing neural networks, , Ph.D. Thesis. State University of Campinas, SP, BrazilEveritt, B.S., (1993) Cluster Analysis, , Wiley: New YorkKaufman, L., Rousseeuw, P., (1990) Finding Groups in Data: An Introduction to Cluster Analysis, , Wiley: New YorkSu, M.-C., Declaris, N., Liu, T.-K., Application of neural networks in cluster analysis (1997) Proc. of the 1997 IEEE Intl. Conf. on Systems, Man, and Cybernetics, pp. 1-6Kothari, R., Pitts, D., On finding the number of clusters (1999) Pattern Recognition Letters, 20, pp. 405-416Hardy, A., (1996) On the number of clusters. Computational Statistics and Data Analysis, 23, pp. 83-96Jain, A.K., Murty, M.N., Flynn, P.J., Data clustering: A review (1999) ACM Computing Surveys, 31 (3), pp. 264-323Ball, G., Hall, D., A clustering technique for summarizing multivariate data (1967) Behavioral Science, 12, pp. 153-155Bezdek, J.C., Pal, N.R., Some new indexes of cluster validity (1998) IEEE Transactions on Systems, Man, and Cybernetics (Part B), 28, pp. 301-315Haykin, S., (1999) Neural Networks: A Comprehensive Foundation, , 2nd edition, Prentice-Hall: New YorkKamgar-Parsi, B., Gualtieri, J.A., Devaney, J.E., Kamgar-Parsi, B., Clustering with neural networks (1990) Biological Cybernetics, 63, pp. 201-208Frank, T., Kraiss, K.-F., Kuhlen, T., Comparative analysis of fuzzy ART and ART-2A network clustering performance (1998) IEEE Trans. on Neural Networks, 9, pp. 544-559Kohonen, T., (1997) Self-Organizing Maps, , 2nd Ed., Springer-Verlag: BerlinUltsch, A., Self-Organizing Neural Networks for Visualization and Classification (1993) Information and Classification, pp. 301-306. , O. Opitz et al. (Eds)., Springer: BerlinGirardin, L., (1995) Cyberspace geography visualization, , heiwww.unige.ch/girardin/cgvGonzales, R.C., Woods, R.E., (1992) Digital Image Processing. Reading, , MA: Addison-WesleyBarrera, J., Banon, J., Lotufo, R., Mathematical Morphology Toolbox for the Khoros System (1994) Image Algebra and Morphological Image Processing V, 2300, pp. 241-252. , E. Dougherty et al. Eds. Proc. SPIESerra, J., (1982) Image Analysis and Mathematical Morphology, , Academic Press: LondonNajman, L., Schmitt, M., Geodesic Saliency of Watershed Contours and Hierarchical Segmentation (1996) IEEE Trans. on Pattern Analysis and Machine Intelligence, 18, pp. 1163-1173Bleau, A., Leon, L.J., Watershed-based segmentation and region merging Comp. Vis. Image Underst., 77, pp. 317-370Costa, J.A.F., Mascarenhas, N., Netto, M.L.A., Cell nuclei segmentation in noisy images using morphological watersheds (1997) Applications of Digital Image Processing XX., 3164, pp. 314-324. , A. Tescher Ed. Proc. of the SPIECosta, J.A.F., Netto, M.L.A., Estimating the Number of Clusters in Multivariate Data by Self-Organizing Maps (1999) International Journal of Neural Systems, 9 (3), pp. 195-202Costa, J.A.F., Netto, M.L.A., Cluster analysis using self-organizing maps and image processing techniques Proc. of the 1999 IEEE Intl. Conf. on Systems, Man, and Cybernetics, , Tokyo, JapanNakamura, E., Kehtarnavaz, N., Determining the number of clusters and prototype locations via multi-scale clustering (1998) Pattern Recognition Letters, 19, pp. 1265-1283Li, T., Tang, Y., Suen, S., Fang, L., Hierarchical classification and vector quantisation with neural trees (1993) Neurocomputing, 5, pp. 119-139Racz, J., Klotz, T., Knowledge representation by dynamic competitive learning techniques Proc. SPIE, 1469, pp. 778-783Adams, R., Butchart, K., Davey, N., Hierarchical classification with a competitive evolutionary neural tree (1999) Neural Networks, 12, pp. 541-551Costa, J.A.F., Netto, M.L.A., Automatic Data Classification by a Hierarchy of Self-Organizing Maps Proc. 1999 IEEE Intl. Conf. on Systems, Man, and Cybernetics, , Tokyo, JapanKoikkalainen, P., Progress with the Tree-Structured Self-Organizing Map (1994) Proc. of the 11 th European Conference on Artificial Intelligence, pp. 211-215Miikkulainen, R., Script Recognition with Hierarchical Feature Maps (1990) Connection Science, pp. 83-10

Segmentação de mapas auto-organizáveis com espaço de saída 3-D

The self-organizing map (SOM) has been widely used as a software tool for visualization of high-dimensional data. Important SOM features include information compression while trying to preserve topological and metric relationship of the primary data items. Similar data in the input space would be mapped to the same neuron or in a nearby unit. The clustering properties of a trained SOM 2-D can be visualized by the U-matrix, which is a neuron's neighborhood distance based image. This assumption of topological preservation is not true for many SOM mappings involving dimension reduction. With the automation of cluster detection in SOM network higher output dimensions can be used in problems involving discovery of classes in multidimensional data. Results of topological errors are shown in a simple 2-D clustering in a 1-D output grid SOM. This paper presents the U-array as an extension of the U-matrix for 3-D output grids. The advantage of the method relies in working with higher dimensions in the output space, which can lead to a better topological preservation in data analysis. Examples of automatic class discovery using U-arrays are also presented.O mapa de Kohonen (SOM) tem sido utilizado como ferramenta para visualização de dados de elevada dimensionalidade. Características importantes da rede SOM incluem a compressão de informação e a tentativa de manutenção da topologia dos dados. Dados similares no espaço de entrada deveriam ser mapeados no mesmo neurônio, ou em neurônios vizinhos. Uma das ferramentas de visualização de um mapa 2-D treinado é U-matrix, que apresenta as relações de distância de pesos entre neurônios vizinhos do espaço de saída. A suposição de preservação topológica não é verdade em muitos problemas envolvendo redução de dimensionalidade. Com a automação da detecção de agrupamentos na rede SOM espaços de saída maiores podem ser utilizados em problemas envolvendo a descoberta de classes em dados multidimensionais. Mostra-se a ocorrência de erros topológicos em um exemplo simples de agrupamentos de dados 2-D em um mapa com saída 1-D. Este artigo apresenta uma extensão da U-matrix, o U-array, para espaços de saída maior que 2-D e sua aplicação em conjunto com o algoritmo SL-SOM, que possibilita a detecção do número e o geometria das classes em mapas treinados. Todo o processo é não-supervisionado. A vantagem de trabalhar com dimensões mais elevadas no espaço de saída é a melhor preservação da topologia em problemas de análise automática de dados. Apresenta-se um exemplo de uso de descoberta de classes de dados não linearmente separáveis.15016

Unsupervised methods of classifying remotely sensed imges using Kohonen self-organizing maps

Orientadores: Marcio Luiz de Andrade Netto, Jose Alfredo Ferreira CostaAcompanha Anexo A: Midia com informações adicionais em CD-RTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de ComputaçãoResumo: Esta tese propõe novas metodologias de classificação não-supervisionada de imagens de sensoriamento remoto que particularmente exploram as características e propriedades do Mapa Auto-organizável de Kohonen (SOM - Self-Organizing Map). O ponto chave dos métodos de classificação propostos é realizar a análise de agrupamentos das imagens através do mapeamento produzido pelo SOM, ao invés de trabalhar diretamente com os padrões originais das cenas. Tal estratégia reduz significativamente a complexidade da análise dos dados, tornando possível a utilização de técnicas normalmente consideradas computacionalmente inviáveis para o processamento de imagens de sensoriamento remoto, como métodos de agrupamentos hierárquicos e índices de validação de agrupamentos. Diferentemente de outras abordagens, nas quais o SOM é utilizado como ferramenta de auxílio visual para a detecção de agrupamentos, nos métodos de classificação propostos, mecanismos para analisar de maneira automática o arranjo de neurônios de um SOM treinado são aplicados e aprimorados com o objetivo de encontrar as melhores partições para os conjuntos de dados das imagens. Baseando-se nas propriedades estatísticas do SOM, modificações nos cálculos de índices de validação agrupamentos são propostas com o objetivo de reduzir o custo computacional do processo de classificação das imagens. Técnicas de análise de textura em imagens são aplicadas para avaliar e filtrar amostras de treinamento e/ou protótipos do SOM que correspondem a regiões de transição entre classes de cobertura terrestre. Informações espaciais a respeito dos protótipos do SOM, além das informações de distância multiespectral, também são aplicadas em critérios de fusão de agrupamentos procurando facilitar a discriminação de classes de cobertura terrestre que apresentam alto grau de similaridade espectral. Resultados experimentais mostram que os métodos de classificação propostos apresentam vantagens significativas em relação às técnicas de classificação não-supervisionada mais freqüentemente utilizadas na área de sensoriamento remoto.Abstract: This thesis proposes new methods of unsupervised classification for remotely sensed images which particularly exploit the characteristics and properties of the Kohonen Self-Organizing Map (SOM). The key point is to execute the clustering process through a set of prototypes of SOM instead of analyzing directly the original patterns of the image. This strategy significantly reduces the complexity of data analysis, making it possible to use techniques that have not usually been considered computationally viable for processing remotely sensed images, such as hierarchical clustering methods and cluster validation indices. Unlike other approaches in which SOM is used as a visual tool for detection of clusters, the proposed classification methods automatically analyze the neurons grid of a trained SOM in order to find better partitions for data sets of images. Based on the statistical properties of the SOM, clustering validation indices calculated in a modified manner are proposed with the aim of reducing the computational cost of the classification process of images. Image texture analysis techniques are applied to evaluate and filter training samples and/or prototypes of the SOM that correspond to transition regions between land cover classes. Spatial information about the prototypes of the SOM, in addition to multiespectral distance information, are also incorporated in criteria for merging clusters with aim to facilitate the discrimination of land cover classes which have high spectral similarity. Experimental results show that the proposed classification methods present significant advantages when compared to unsupervised classification techniques frequently used in remote sensing.DoutoradoEngenharia de ComputaçãoDoutor em Engenharia Elétric

Neuroengineering of Clustering Algorithms

Cluster analysis can be broadly divided into multivariate data visualization, clustering algorithms, and cluster validation. This dissertation contributes neural network-based techniques to perform all three unsupervised learning tasks. Particularly, the first paper provides a comprehensive review on adaptive resonance theory (ART) models for engineering applications and provides context for the four subsequent papers. These papers are devoted to enhancements of ART-based clustering algorithms from (a) a practical perspective by exploiting the visual assessment of cluster tendency (VAT) sorting algorithm as a preprocessor for ART offline training, thus mitigating ordering effects; and (b) an engineering perspective by designing a family of multi-criteria ART models: dual vigilance fuzzy ART and distributed dual vigilance fuzzy ART (both of which are capable of detecting complex cluster structures), merge ART (aggregates partitions and lessens ordering effects in online learning), and cluster validity index vigilance in fuzzy ART (features a robust vigilance parameter selection and alleviates ordering effects in offline learning). The sixth paper consists of enhancements to data visualization using self-organizing maps (SOMs) by depicting in the reduced dimension and topology-preserving SOM grid information-theoretic similarity measures between neighboring neurons. This visualization\u27s parameters are estimated using samples selected via a single-linkage procedure, thereby generating heatmaps that portray more homogeneous within-cluster similarities and crisper between-cluster boundaries. The seventh paper presents incremental cluster validity indices (iCVIs) realized by (a) incorporating existing formulations of online computations for clusters\u27 descriptors, or (b) modifying an existing ART-based model and incrementally updating local density counts between prototypes. Moreover, this last paper provides the first comprehensive comparison of iCVIs in the computational intelligence literature --Abstract, page iv

A New Tree-structured Self-organizing Map For Data Analysis

Self-organizing map has been applied to a variety of tasks including data visualization and clustering. Once the point density of the neurons approximates the density of data, it is possible to miner clustering information from the data set after its unsupervised learning by using the neuron's relations. This paper presents a new algorithm for dynamical generation of a hierarchical structure of self-organizing maps with applications to data analysis. Differently from other tree-structured SOM approaches, which nodes are neurons, in this case the tree nodes are actually maps. From top to down, maps are automatically segmented by using the U-matrix information, which presents relations between neighboring neurons. The automatic map partitioning algorithm is based on mathematical morphology segmentation and it is applied to each map in each level of the hierarchy. Clusters of neurons are automatically identified and labeled and generate new sub-maps. Data are partitioned accordingly the label of its best match unit in each level of the tree. The algorithm may be seen as a recursive partition clustering method with multiple prototypes cluster representation, which enables the discoveries of clusters in a variety of geometrical shapes.319311936Murthy, S.K., Automatic construction of decision trees from data: A multidisciplinary survey Azevedo, H., Vellasco, M., Passos, E., Mineração de dados aplicada a CRM em uma base de clientes de telefonia de longa distância (2001) Proc. of the Brazilian Conf. on Neural Networks, pp. 397-402. , Rio de Janeiro, BrazilSerrano-Cinca, C., Self-organizing neural networks for financial diagnosis Decision Support Systems, 17, pp. 227-238Reibnegger, G., Wachter, H., Self-organizing neural networks - An alternative way of cluster analysis in clinical chemistry (1996) Clinica Chimica Acta, 248, pp. 91-98Jain, A.K., Data clustering: A review ACM Computing Surveys, 31, pp. 264-323Su, M.-C., DeClaris, N., Liu, T.-K., Application of neural networks in cluster analysis (1997) Proc. of the 1997 IEEE Intl. Conf. on Systems, Man, and Cybernetics, Orlando, FL, pp. 1-6Costa, J.A.F., Automatic classification and data analysis by self-organizing neural networks (in portuguese) (1999), Ph.D. Thesis. UNICAMP, Campinas, SP, BrazilKohonen, T., (1997) Self-Organizing Maps, 2nd Edition, , Berlin: Springer-VerlagHaykin, S., (1999) Neural Networks a Comprehensive Foundation, 2nd Ed., , Prentice-Hall, New JerseyKaski, S., Nikkilä, J., Kohonen, T., Methods for exploratory cluster analysis Proc. of SSGRR 2000, Intl. Conf. on Advances in Infrastructure for Electronic Business, Science, and Education on the Internet, L'Aquila, 2000Costa, J.A.F., Netto, M.L.A., Estimating the number of clusters in multivariate data by self-organizing maps Intl. Journal of Neural Systems, 9, pp. 195-202Costa, J.A.F., Netto, M.L.A., Clustering of complex shaped data sets via Kohonen maps and mathematical morphology (2001) Proc. of the SPIE, Data Mining and Knowledge Discovery, 4384, pp. 16-27. , B. Dasarathy (Ed.)Murtagh, F., Interpreting the Kohonen self-organizing feature map using contiguity-constrained clustering (1995) Pattern Recognition Letters, 16, pp. 399-408Vesanto, J., Alhoniemi, E., Clustering of the self-organizing map IEEE Trans. on Neural Netw., 11, pp. 586-600Ultsch, A., Self-organizing neural networks for visualization and classification (1993) Information and Classification, pp. 307-313. , In: O. Opitz et al. (Eds)Springer, BerlinBleau, A., Leon, L.J., Watershed-based segmentation and region merging Comp. Vis. Image Underst., 77, pp. 317-370Costa, J.A.F., Mascarenhas, N., Netto, M.L.A., Cell nuclei segmentation in noisy images using morphological watersheds (1997) Proc. of the SPIE, 3164, pp. 314-324Bezdek, J., Pal, N.R., An index of topological preservation for feature extraction (1995) Pattern Recognition, 28, pp. 381-39

Data Clustering Using Self-organizing Maps Segmented By Mathematic Morphology And Simplified Cluster Validity Indexes: An Application In Remotely Sensed Images

This paper presents a cluster analysis method which automatically finds the number of clusters as well as the partitioning of a data set without any type of interaction with the user. The data clustering is made using the Self-Organizing (or Kobonen) Map (SOM). Different partitions of the trained SOM are obtained from different segmentations of the U-matrix (a neuron-distance image) that are generated by means of mathematical morphology techniques. The different partitions of the trained SOM produce different partitions for the data set which are evaluated by cluster validity indexes. To reduce the computational cost of the cluster analysis process this work also proposes the simplification of cluster validity indexes using the statistical properties of the SOM. The proposed methodology is applied in the cluster analysis of remotely sensed images. © 2006 IEEE.44214428Richards, J.A., Analysis of remotely sensed data: The formative decades and the future (2005) IEEE Transactions on Geoscience and Remote Sensing, 43, pp. 422-432. , MarchSun, Z., Huang, D., Cheung, Y., Liu, J., Huang, G., Using FCMC, FVS, and PCA techniques for feature extraction of Multispectral images (2005) IEEE Geosc. and Rem. Sens. Lett, 2 (2), pp. 108-112. , AprilXu, R., Wunsch, D., Survey of clustering algorithms (2005) IEEE Trans. on Neural Networks, 16 (3), pp. 645-678. , MayTran, T.N., Wehrens, R., Buydes, L.M.C., Clustering multispectral images: A tutorial (2005) Chemometrics and Intelligent Laboratory Systems, 77, pp. 3-17Ball, G.H., Hall, D.J., (1965) ISODATA, a novel method of data analysis and pattern classification, , Stanford Research Institute, Menlo Park, CA, NTIS Report AD 699616Haykin, S., (1999) Neural Network: A Comprehensive Foundation, , New York: Prentice-Hall, 2nd editionCosta, J.A.F., Netto, M.L.A., Estimating the number of clusters in multivariate data by self-organizing maps (1999) International Journal of Neural Systems, 9, pp. 195-202J. A. F. Costa and M. L. A. Netto, Clustering of complex shaped data sets via kohonen maps and mathematical morphology, in Proceedings of the SPIE, Data Mining and Knowledge Discovery. B. Dasarathy (Ed.), 2001, 4384, pp. 16-27Gonçalves, M.L., Netto, M.L.A., Zullo, J., A neural architecture for the classification of remote sensing imagery with advanced learning algorithms (1998) Proc. of the IEEE Signal Processing Society Workshop, pp. 577-585M. L. Gonçalves, M. L. A. Netto, J. A. F. Costa and J. Zullo, Automatic remotely sensed data Clustering by tree-structured self-organizing, in Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, IGARSS'05, Korea, 2005Kohonen, T., (1997) Self-Organizing Maps, , 2nd Edition, Berlim: Springer VerlagUltsch, A., Self-organizing neural networks for visualization and classification (1993) Information and Classification, pp. 307-313. , O. Opitz et al, Eds, Berlin, Springer-Verlag, ppLampinen, J., Oja, E., Clustering properties of hierarchical self-organizing maps (1992) J. of Math. Im. and Vis, 2, pp. 261-272Murtagh, F., Interpreting the kohonen self-organizing feature map using contiguity-constrained clustering (1995) Pattern Recognition Letters, 16, pp. 399-408Kiang, M.Y., Extending the kohonen self-organizing map networks for clustering analysis (2001) Computational Statistics & Data Analysis, 38, pp. 161-180Vesanto, J., Alhoniemi, E., Clustering of the self-organizing map (2000) IEEE Trans. on Neural Networks, 11, pp. 586-600. , MayWu, S., Chow, T.W.S., Clustering of the self-organizing map using a clustering validity index based on inter-cluster and intracluster density (2004) Pattern Recognition, 37, pp. 175-188Bleau, A., Leon, L.J., Watershed-based segmentation and region merging (2000) Comp. Vis. Image Underst, 77, pp. 317-370Bezdek, J.C., Pal, N.R., Some new indexes of cluster validity (1998) IEEE Trans. on Syst, Man and Cybern, 28, pp. 301-315Davies, D., Bouldin, D., A cluster separation measure (1979) IEEE Trans. Patt. Rec. and Mach. Intell, PAM1-1, pp. 224-227Pakhira, M.K., Bandyopedhyay, S., Maulik, U., Validity index for crisp and fuzzy clusters (2003) Pattern Recog, 37, pp. 487-501Halkidi, M., Vazirgiannis, M., Clustering validity assessment using multi representatives (2002) Proceedings of SETN Conference, , Thessaloniki, GreeceJI, M., Using fuzy sets to improve cluster labeling in unsupervised classification (2003) Int. J. of Remote Sensing, 24, pp. 657-67