Semi-supervised classification using tree-based self-organizing maps

This paper presents a classifier which uses a tree-based Neural Network (NN), and uses both, unlabeled and labeled instances. First, we learn the structure of the data distribution in an unsupervised manner. After convergence, and once labeled data become available, our strategy tags each of the clusters according to the evidence provided by the instances. Unlike other neighborhood-based schemes, our classifier uses only a small set of representatives whose cardinality can be much smaller than that of the input set. Our experiments show that, on average, the accuracy of such classifier is reasonably comparable to those obtained by some of the state-of-the-art classification schemes that only use labeled instances during the training phase. The experiments also show that improved levels of accuracy can be obtained by imposing trees with a larger number of nodes

On using adaptive Binary Search Trees to enhance self organizing maps

We present a strategy by which a Self-OrganizingMap (SOM) with an underlying Binary Search Tree (BST) structure can be adaptively re-structured using conditional rotations. These rotations on the nodes of the tree are local and are performed in constant time, guaranteeing a decrease in the Weighted Path Length (WPL) of the entire tree. As a result, the algorithm, referred to as the Tree-based Topology-Oriented SOM with Conditional Rotations (TTO-CONROT), converges in such a manner that the neurons are ultimately placed in the input space so as to represent its stochastic distribution, and additionally, the neighborhood properties of the neurons suit the best BST that represents the data

Concept drift detection using online histogram-based bayesian classifiers

In this paper, we present a novel algorithm that performs online histogram-based classification, i.e., specifically designed for the case when the data is dynamic and its distribution is non-stationary. Our method, called the Online Histogram-based Naïve Bayes Classifier (OHNBC) involves a statistical classifier based on the well-established Bayesian theory, but which makes some assumptions with respect to the independence of the attributes. Moreover, this classifier generates a prediction model using uni-dimensional histograms, whose segments or buckets are fixed in terms of their cardinalities but dynamic in terms of their widths. Additionally, our algorithm invokes the principles of information theory to automatically identify changes in the performance of the classifier, and consequently, forces the reconstruction of the classification model in run-time as and when it is needed. These properties have been confirmed experimentally over numerous data sets (In the interest of space and brevity, we present here only a subset of the available results. More detailed results are found in [2].) from different domains. As far as we know, our histogram-based Naïve Bayes classification paradigm for time-varying datasets is both novel and of a pioneering sort

On achieving semi-supervised pattern recognition by utilizing tree-based SOMs

Years of research in the field of Pattern Recognition (PR) has led to scores of algorithms which can achieve supervised pattern classification. Such algorithms assume the knowledge of well-defined training sets with a clear specification of the identity of all the training samples. However, more recently, a new stream has emerged, namely, the so-called semi-supervised paradigm, i.e., one that uses a combination of labeled and unlabeled samples to perform classification [41]. Classifiers based on the latter, do not demand the specification of the class labels of every sample. Rather, a clustering-like mechanism processes the manifold, and attempts to distinguish the training samples into the separate classes, subsequent to which a supervised classifier is derived using a small subset of the training samples whose class identities are known. In this paper we will venture to utilize the Tree-based Topology Oriented SOM (TTOSOM) [3] for semi-supervised pattern classification. We first train a TTOSOM in which the neurons collectively obey the stochastic, topological and structural distribution of all the classes. Subsequently, we make use of the information provided in the labeled dataset. By using this information, we assign a class label to every single node in the Neural Network (NN), which, in turn, partitions the space into its Voronoi regions. On receiving the testing data, the task at hand is rather straightforward. One nearly determines the closest neuron to the testing sample and assigns the sample to the corresponding class. The complexity of the testing is linear, not in cardinality of the training set, but rather in the size of the TTOSOM tree! Our experimental results show that on average, the classification capabilities of our proposed strategy, even with a small number of neurons, are reasonably comparable to those obtained by some of the state-of-the-art classification schemes that only use labeled instances during the training phase. The experiments also show that improved levels of accuracy can be obtained by imposing trees with a larger number of nodes

A novel self organizing map which utilizes imposed tree-based topologies

In this paper we propose a strategy, the Tree-based Topology-Oriented SOM (TTO-SOM) by which we can impose an arbitrary, user-defined, tree-like topology onto the codebooks. Such an imposition enforces a neighborhood phenomenon which is based on the user-defined tree, and consequently renders the so-called bubble of activity to be drastically different from the ones defined in the prior literature. The map learnt as a consequence of training with the TTO-SOM is able to infer both the distribution of the data and its structured topology interpreted via the perspective of the user-defined tree. The TTO-SOM also reveals multi-resolution capabilities, which are helpful for representing the original data set with different numbers of points, whithout the necessity of recomputing the whole tree. The ability to extract an skeleton, which is a “stick-like” representation of the image in a lower dimensional space, is discussed as well. These properties have been confirmed by our experimental results on a variety of data sets

Topology-oriented self-organizing maps: A survey

The self-organizing map (SOM) is a prominent neural network model that has found wide application in a spectrum of domains. Accordingly, it has received widespread attention both from the communities of researchers and practitioners. As a result, several variations of the basic architecture have been devised, specifically in the early years of the SOM's evolution, which were introduced so as to address various architectural shortcomings or to explore other structures of the basic model. The overall goal of this survey is to present a comprehensive comparison of these networks, in terms of their primitive components and properties. We dichotomize these schemes as being either tree based or non-tree based. We have embarked on this venture with the hope that since the survey is comprehensive and the bibliography extensive, it will be an asset and resource for future researchers

Self-organizing maps whose topologies can be learned with adaptive binary search trees using conditional rotations

Numerous variants of Self-Organizing Maps (SOMs) have been proposed in the literature, including those which also possess an underlying structure, and in some cases, this structure itself can be defined by the user. Although the concepts of growing the SOM and updating it have been studied, the whole issue of using a self-organizing Adaptive Data Structure (ADS) to further enhance the properties of the underlying SOM, has been unexplored. In an earlier work, we impose an arbitrary, user-defined, tree-like topology onto the codebooks, which consequently enforced a neighborhood phenomenon and the so-called tree-based Bubble of Activity (BoA). In this paper, we consider how the underlying tree itself can be rendered dynamic and adaptively transformed. To do this, we present methods by which a SOM with an underlying Binary Search Tree (BST) structure can be adaptively re-structured using Conditional Rotations (CONROT). These rotations on the nodes of the tree are local, can be done in constant time, and performed so as to decrease the Weighted Path Length (WPL) of the entire tree. In doing this, we introduce the pioneering concept referred to as Neural Promotion, where neurons gain prominence in the Neural Network (NN) as their significance increases. We are not aware of any research which deals with the issue of Neural Promotion. The advantage of such a scheme is that the user need not be aware of any of the topological peculiarities of the stochastic data distribution. Rather, the algorithm, referred to as the TTOSOM with Conditional Rotations (TTOCONROT), converges in such a manner that the neurons are ultimately placed in the input space so as to represent its stochastic distribution, and additionally, the neighborhood properties of the neurons suit the best BST that represents the data. These properties have been confirmed by our experimental results on a variety of data sets. We submit that all these concepts are both novel and of a pioneering sort