One of the problems with existing clustering methods is that the interpretation of clusters may be difficult. Two different approaches have been used to solve this problem: conceptual clustering in machine learning and clustering visualization in statistics and graphics. The purpose of this paper is to investigate the benefits of combining clustering visualization and conceptual clustering to obtain better cluster interpretations. In our research we have combined concept trees for conceptual clustering with shaded similarity matrices for visualization. Experimentation shows that the two interpretation approaches can complement each other to help us understand data better

Gasser, Les

Wang, Jun

Yu, Bei

Syracuse University Research Facility and Collaborative Environment

Syracuse University SURFACE School of Information Studies - Faculty Scholarship School of Information Studies (iSchool) 12-2002 Concept Tree Based Clustering Visualization with Shaded Similarity Matrices Bei Yu University of Illinois at Urbana-Champaign Jun Wang University of Illinois at Urbana-Champaign Les Gasser University of Illinois at Urbana-Champaign Follow this and additional works at: https://surface.syr.edu/istpub  Part of the Library and Information Science Commons Recommended Citation Wang, J., Yu, B., and Gasser, L. (2002) Concept tree based ordering for shaded similarity matrix. Proceedings of the 2nd IEEE International Conference on Data Mining, Maebashi City, Japan, December 2002, pp. 697-700. This Conference Document is brought to you for free and open access by the School of Information Studies (iSchool) at SURFACE. It has been accepted for inclusion in School of Information Studies - Faculty Scholarship by an authorized administrator of SURFACE. For more information, please contact surface@syr.edu. Concept Tree Based Clustering Visualization with Shaded Similarity MatricesJun Wang Bei Yu Les GasserGraduate School of Library and Information ScienceUniversity of Illinois at Urbana-Champaign501 E. Daniel St., Champaign, IL 61820, USA{junwang4, beiyu, gasser}@uiuc.eduAbstractOne of the problems with existing clustering methods isthat the interpretation of clusters may be difficult. Two dif-ferent approaches have been used to solve this problem:conceptual clustering in machine learning and clusteringvisualization in statistics and graphics. The purpose of thispaper is to investigate the benefits of combining cluster-ing visualization and conceptual clustering to obtain bettercluster interpretations. In our research we have combinedconcept trees for conceptual clustering with shaded similar-ity matrices for visualization. Experimentation shows thatthe two interpretation approaches can complement eachother to help us understand data better.Keywords: Clustering Visualization, Conceptual Clus-tering, Shaded Similarity Matrix, Concept Tree1. IntroductionOne of the problems with existing clustering methods isthat the interpretation of clusters produced may be difficult.To address this interpretation problem, on the one hand,people from statistics and graphics have focused on visual-ization approaches[4, 5]. On the other hand, researchers inmachine learning (or artificial intelligence) have developedconceptual clustering[1, 10]. Clustering visualization canhelp users visually perceive the clusterings, and sometimeseven hidden patterns in data. Conceptual clustering aimsat representing the clusterings using symbolic knowledge.Clustering visualization utilizes people’s perceptual ability(low-level information processing), while conceptual clus-tering exploits human inference ability (high-level informa-tion processing). However, these two different approacheshave not previously been combined. The purpose of this pa-per is to combine conceptual clustering with visualization inorder to obtain better intrepretations of clusterings.Our approach is to use shaded similarity matrices[3] forvisualization and concept trees[7] for conceptual clustering.Since there are exponentially many ways to order a set ofobjects, the key problem for the shaded similarity matrix ap-proach is how to order the data or objects in a matrix so thatsimilar objects are adjacent. Heuristic strategies are neededfor generating a near-optimal ordering. Happily, concepttrees provide not only an approach to conceptual clustering,but also a potential approach to solve the ordering prob-lem, because the more specific the concept shared by twoobjects, the more similar the two objects. Our experiments(presented later) do show that concept trees are effective.2. Shaded Similarity MatricesOver the past forty years, shaded similarity matrices havebeen used in visual cluster anaylsis[8, 3, 11]. In a shadedsimilarity matrix1, similarity in each cell is represented us-ing a shade to indicate the similarity value: greater simi-larity is represented by dark shading, and lesser similarityby light shading. The dark and light cells may initially bescattered over the matrix. To reveal the potential clusteringsvisually, the rows and columns need to be re-organized sothat similar objects are put in adjacent positions. If “real”clusters exist in the data, they should appear as symmetricaldark squares along the diagonal.Here we will briefly show how shaded similarity matri-ces are constructed and how one looks through an example.The data used in the example is part of the Iris data from theUCI repository[9]. The Iris data set contains 150 instances,evenly distributed in 3 classes. We fetch 5 instances fromeach class, and thus obtain 15 instances (Table 1). The sim-ilarity matrix was computed based on Euclidean distance(Table 2).The shaded similarity matrix is illustrated in Fig. 1. Theright figure in Fig. 1 is generated from the original simi-larity matrix using the seriation algorithm which was pro-1Some researchers use the term shaded distance matrix, shaded prox-imity matrix, or trellis diagram.Table 1. Data matrix extracted from the Irisdata set. Abbreviations: sl: sepal-length, sw:sepal-width, pl: petal-length, pw: petal-width.Instance sl sw pl pwe1 5.1 3.5 1.4 0.2e2 6.3 2.9 5.6 1.8e3 7.0 3.2 4.7 1.4. . . . . . . . . . . . . . .e15 6.5 2.8 4.6 1.5Table 2. Similarity matrix corresponding to Ta-ble 1.1.0 0.1 0.2 0.8 0.2 0.1 . . . 0.20.1 1.0 0.3 0.2 0.3 0.8 . . . 0.30.2 0.3 1.0 0.2 0.9 0.3 . . . 0.8. . . . . .0.2 0.3 0.8 0.2 0.8 0.3 . . . 1.0posed in ClustanGraphics [11]. It works by weighting eachsimilarity using the distance of the similarity cell from thediagonal. The algorithm tries to minimize the sum of theweighted similarities in the similarity matrix by reorderingthe pre-computed clusters in an agglomerative hierarchicalclustering such as a dendrogram.Figure 1. LEFT: Randomly ordered shaded simi-larity matrix; RIGHT: Reordered shaded similar-ity matrix using a seriation algorithm.To display a similarity matrix of n objects, we need n2cells or n22cells (in the case of half matrix). In practice,usually it is not necessary to display all cells in a matrix. Inthis paper, only those cells are displayed whose similarityvalues are over a pre-specified threshold.Petal lengthPetal width{e1, e4, e5, e11, e13} Petal length≤2.6≤2.6 >2.6≤1.6 >1.6{e2, e6, e7, e8, e14} Petal length>2.6∧ Petal width≤1.6{e3, e9, e11, e12, e15} Petal length>2.6∧ Petal width>1.6Figure 2. A concept tree on the Iris data set.3. Concept Tree Based Clustering3.1. Concept treeA concept tree (also known as concept hierarchy) iscomposed of nodes and links with each node representinga concept[7]. The links connecting a node to its childrenspecify an ‘IS-A’ or ‘subset’ relation. Fig.2 is an illustra-tion of a concept tree generated by using the algorithm inTable 3 on the 15 Iris instances mentioned above.3.2. Concept tree construction algorithmTable 3 is a general algorithm for constructing a con-cept tree. The key part of the algorithm is how to selecta best attribute using some split measurement. In this pa-per, we take the within-group average similarity[2] as themeasurement. The best attribute is the one with the maxi-mum within-group average similarity. Suppose an attributea with k possible values splits a data set S into k subsets:{S1, · · · , Sk}. Let σ(e1, e2) be the similarity of two in-stances e1 and e2. Then the within-group average similarityis defined as:WAS(a) =k∑i=1|Si||S|∑e1,e2∈Siσ(e1, e2)|Si|2(1)3.3. Concept tree based ordering for shaded simi-larity matricesGiven that objects sharing the same concept are prob-ably similar, if we put together all objects belonging to aconcept, the ordering problem for shaded similarity matri-ces will be heuristically solved. Concretely speaking, givena tree with k leaves from left to right: < L1, L2, · · · , Lk >.Each leaf represents a concept which covers a set of objects:Table 3. Concept tree construction algorithm.Inputs: The current node N of the concept tree,an instance set S,and an attribute set A .Output: A concept tree.Procedure: CTree(N,S, A)If both S and A are not empty,Then:Select a best attribute a ∈ A using some metric,For each possible value vi of a:Form a node C, corresponding to the test a = vi, ∗Let Svi be the subset of S that have value vi for a,Make node C a child of Node N ,CTree(C,Svi , A − {a}).∗ For numberical attributes, the test is represented as a ≤ vi or a > vi.Li = {ei1 , · · · , ein}. Then the ordering of all objects willbe:< {e11 , · · · , e1n}, · · · · · · , {ek1 , · · · , ekn} >We don’t care about the internal object ordering within aleaf. We assume that the objects within a leaf are similarenough, and if not the leaf node can be partitioned intosmaller leaves until the objects within a leaf do becomesimilar enough. Note that this ordering can only producea partial order of the objects due to limitations of the treestructure.4. ExperimentationTo demonstrate the effectiveness of our approach, wehave tested it on some UCI datasets. Here we use the fullIris dataset (150 instances) to generate a concept tree. Ta-ble 4 shows 3 concepts which are visualized as three squareblocks along the diagonal from the left-top to right-bottomin the Fig. 3. Note that the concept tree here is a little bit dif-ferent from the one in Fig. 2 because of the different datasetsize.Table 4. Concepts shown in Fig. 3.Concept name Square block Concept descriptionconcept-1 left-top pl ≤ 2.5concept-2 center pl > 2.5 ∧ pw ≤ 1.8concept-3 right-bottom pl > 2.5 ∧ pw > 1.8Among these 3 concepts, the concept-1 is the mostclearly separated from other concepts. The concept-2 andpetal length <= 2.5  [50] petal length > 2.5  [100]petal width <= 1.8  [54] petal width > 1.8  [46]Figure 3. Concept tree based clustering visu-alization on the Iris dataset.concept-3 are not perfectly separated. There are some in-stances covered by concept-2 which have high similarity2with the instances in concept-3 and vice versa.Overall, the visualization result in the Fig. 3, allows usto discover two properties of the Iris dataset: 1) the datasetis naturally divided into three groups or clusters (there arethree self-similar blocks); 2) each group can be described bya simple concept. In other words, we can say that both thevisualization and the acquired concepts complement eachother to help us better understand the data.5. DiscussionFirst, it is apparent that the effectiveness of our approachdepends on the definition of similarity, which is also a gen-eral problem for clustering methods. Second, our approachhas an assumption that the data can be described conceptu-ally, which does not always hold. However, if the data can-not be described conceptually, it is hard to say that the dataare fully understood. Therefore, the problem becomes howto choose the right concept representation. Concept treesare only one kind of representation, and are not appropriatefor all data.Third, visualization based on shaded similarity matrices2In the Fig. 3, if the similarity between two instances is over a thresh-old, a spot will be displayed for these two instances. In this situation wesay that these two instances have high similarity.has a scalability limitation. One solution is to use samplingand ensemble approaches. Using small sample sizes suchas 100 or 200, we have tested the sampling approach onsome Statlog datasets, including the Shuttle dataset whichcontains 43, 500 instances[6]. The results are promising.6. SummaryThis paper proposes a new approach for getting betterinterpretations for clustering results by combining visual-ization and conceptual clustering. This is achieved by usingconcept trees as a heuristic ordering strategy for shaded sim-ilarity matrices. Our experiment shows that the two cluster-ing interpretation approaches can complement each other tohelp us better understand the data.AcknowledgmentsThis work was partially supported by the InformationSystems Research Lab (ISRL) of the Graduate School ofLibrary and Information Science at University of Illinoisat Urbana-Champaign. We are thankful to Dr. David Du-bin, Professor Jiawei Han, and reviewers for valuable com-ments.References[1] F. Douglas. Knowledge acquisition via incemental concep-tual clustering. Machine Learning, 2:139–172, 1987.[2] B. S. Everitt. Cluster Analysis. John Wiley & Sons, Inc.,New York, 1974.[3] N. Gale, W. Halperin, and C. Costanzo. Unclassed matrixshading and optimal ordering in hierarchical cluster analy-sis. Journal of Classification, 1:75–92, 1984.[4] P. E. Hoffman and G. G. Grinstein. A survey of visual-izations for high-dimensional data mining. In U. Fayyad,G. Grinstein, and A. Wierse, editors, Information Visual-ization in Data Mining and Knowledge Discovery. MorganKaufmann, 2001.[5] D. A. Keim. Information visualization and visual data min-ing. IEEE Transactions on Visualization and ComputerGraphics (Special Section on Information Visualization andVisual Data Mining), 8(1):1–8, 2002.[6] R. King, C. Feng, and A. Shutherland. Statlog: compar-ison of classification algorithms on large real-world prob-lems. Applied Artificial Intelligence, 9(3):259–287, 1995.[7] P. Langley. Elements of Machine Learning. Morgan Kauf-mann Publishers, 1996.[8] R. L. Ling. A computer generated aid for cluster analysis.Communications of the ACM, 16(6):355–361, 1973.[9] C. J. Merz, P. M. Murphy, and D. W. Aha.UCI repository of machine learning databases,http://www.ics.uci.edu/˜mlearn/mlrepository.html, 1997.Dept. of Information and Computer Science, University ofCalifornia at Irvine.[10] L. Talavera and J. Bejar. Generality-based conceptual clus-tering with probabilistic concepts. IEEE Transactions onPattern Analysis and Machine Intelligence, 23(2):196–206,2001.[11] D. Wishart. ClustanGraphics3: Interactive graphics for clus-ter analysis. In W. Gaul and H. Locarek-Junge, editors, Clas-sification in the Information Age, pages 268–275. Springer-Verlag, 1999.

Concept Tree Based Clustering Visualization with Shaded Similarity Matrices

https://surface.syr.edu/cgi/viewcontent.cgi?article=1030&amp;context=istpub

Concept Tree Based Clustering Visualization with Shaded Similarity Matrices

Abstract

Similar works

Full text

Available Versions

Syracuse University Research Facility and Collaborative Environment