Improving the k-nearest neighbour rule: using geometrical neighbourhoods and manifold-based metrics

Sample weighting and variations in neighbourhood or data-dependent distance metric definitions are three principal directions considered for improving the k-NN classification technique. Recently, manifold-based distance metrics attracted considerable interest and computationally less demanding approximations have been developed. However, a careful comparison of these alternative approaches is missing. In this study, an extensive comparison is firstly performed for three alternative neighbourhood definitions and four manifold-based distance measures. Then, a novel computationally less demanding feature line-based method is proposed, which exploits geometrical neighbourhoods of test samples for feature line construction. Experimental results have shown that the improvements achieved by the majority of the existing schemes are not considerable. It is also verified that the proposed scheme surpasses other computationally less demanding manifold-based schemes

An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification

In this paper, we describe a relation between classification systems and information transmission systems. By looking at the classification systems from this perspective, we propose a method of classifier weight estimation for the linear (LIN-OP) and logarithmic opinion pool (LOG-OP) type classifier combination schemes for which some tools from information theory are used. These weights provide contextual information about the classifiers such as class dependent classifier reliability and global classifier reliability. A measure for decision consensus among the classifiers is also proposed which is formulated as a multiplicative part of the classifier weights. A method of selecting the classifiers which provide complementary information for the combination operation is given. Using the proposed method, two classifiers are selected to be used in the combination operation. Simulation experiments in closed set speaker identification have shown that the method of weight estimation described in this paper improved the identification rates of both linear and logarithmic opinion type combination schemes. A comparison between the proposed method and some other methods of weight selection is also given at the end of the paper

Plurality voting-based multiple classifier systems: statistically independent with respect to dependent classifier sets

The simultaneous use of multiple classifiers has been shown to provide performance improvement in classification problems. The selection of an optimal set of classifiers is an important part of multiple classifier systems and the independence of classifier outputs is generally considered to be an advantage for obtaining better multiple classifier systems. In this paper, the need for the classifier independence is interrogated from classification performance point of view. The performance achieved with the use of classifiers having independent joint distributions is compared to some other classifiers which are defined to have best and worst joint distributions. These distributions are obtained by formulating the combination operation as an optimization problem. The analysis revealed several important observations about classifier selection which are then used to analyze the problem of selecting an additional classifier to be used with the available multiple classifier system. (C) 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved

Post-processing of classifier outputs in multiple classifier systems

Incomparability in classifier outputs due to the variability in their scales is a major problem in the combination of different classification systems. In order to compensate this, output normalization is generally performed where the main aim is to transform the outputs onto the same scale. In this paper, it is proposed that in selecting the transformation function, the scale similarity goal should be accomplished with two more requirements. The first one is the separability of the pattern classes in the transformed output space and the second is the compatibility of the outputs with the combination rule. A method of transformation that provides improved satisfaction of the additional requirements is proposed which is shown to improve the classification performance of both linear and Bayesian combination systems based on the use of confusion matrix based a posteriori probabilities...

A novel rank-based classifier combination scheme for speaker identification

In this paper, we propose a novel rank-based classifier combination scheme under uncertainty for speaker identification (SI). The combination is based on a heuristic method that uses Dempster-Shafer theory of evidence under some conditions. The method is based on the extraction of first and R-th level ranking statistics. Using these statistics, the pat tern classes are clustered into model sets where the classes in these sets share set specific properties. Some of these model sets are used to reflect the strengths and weaknesses where some others carr class dependent ranking statistics of the corresponding classifier. The experiments conducted on the Polycost database have shown that the proposed approach is more effective compared to some other rank-based combination schemes

Undesirable effects of output normalization in multiple classifier systems

Incomparability of the classifier output scores is a major problem in the combination of different classification systems. In order to deal with this problem, the measurement level classifier outputs are generally normalized. However, empirical results have shown that output normalization may lead to some undesirable effects. This paper presents analyses for some most frequently used normalization methods and it is shown that the main reason for these undesirable effects of output normalization is the dimensionality reduction in the output space. An artificial classifier combination example and a real-data experiment are provided where these effects are further clarified

Use of model confusion learning for speaker identification a rule-based approach

This paper presents a multiple classifier system for text-independent speaker identification (SI). For the speaker identification problem, several different classifiers can be developed, each having strengths and weaknesses compared to the others. When the strengths and weaknesses of the individual classifiers do not overlap, i.e. a speaker which is misclassified by one classifier is correctly classified by some others, robust classification systems can be developed with the use of multiple classifiers. The studies in multiple classifier systems mainly concentrate on reliable methods of extracting, contextual information (i.e. strengths and weaknesses) about the classifiers and the methods of combining these classifiers. In this paper, a method is proposed for the extraction of contextual information about the classifiers and a rule based approach is developed for the combination of the information from different classifiers

Speaker identification by combining multiple classifiers using Dempster-Shafer theory of evidence

This paper presents a multiple classifier approach as an alternative solution to the closed-set text-independent speaker identification problem. The proposed algorithm which is based on Dempster-Shafer theory of evidence computes the first and Rth level ranking statistics. Rth level confusion matrices extracted from these ranking statistics are used to cluster the speakers into model sets where they share set specific properties. Some of these model sets are used to reflect the strengths and weaknesses of the classifiers while some others carry speaker dependent ranking statistics of the corresponding classifier. These information sets from multiple classifiers are combined to arrive at a joint decision. For the combination task, a rule-based algorithm is developed where Dempster's rule of combination is applied in the final step. Experimental results have shown that the proposed method performed much better compared to some other rank-based combination methods

Speaker Identification in Multi-Coder Environment

No abstract available