Dimension Reduction by Mutual Information Discriminant Analysis

In the past few decades, researchers have proposed many discriminant analysis (DA) algorithms for the study of high-dimensional data in a variety of problems. Most DA algorithms for feature extraction are based on transformations that simultaneously maximize the between-class scatter and minimize the withinclass scatter matrices. This paper presents a novel DA algorithm for feature extraction using mutual information (MI). However, it is not always easy to obtain an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction that is based on one-dimensional MI estimations. We will refer to this algorithm as mutual information discriminant analysis (MIDA). The performance of this proposed method was evaluated using UCI databases. The results indicate that MIDA provides robust performance over different data sets with different characteristics and that MIDA always performs better than, or at least comparable to, the best performing algorithms.Comment: 13pages, 3 tables, International Journal of Artificial Intelligence & Application

Linear classifier design under heteroscedasticity in Linear Discriminant Analysis

Under normality and homoscedasticity assumptions, Linear Discriminant Analysis (LDA) is known to be optimal in terms of minimising the Bayes error for binary classification. In the heteroscedastic case, LDA is not guaranteed to minimise this error. Assuming heteroscedasticity, we derive a linear classifier, the Gaussian Linear Discriminant (GLD), that directly minimises the Bayes error for binary classification. In addition, we also propose a local neighbourhood search (LNS) algorithm to obtain a more robust classifier if the data is known to have a non-normal distribution. We evaluate the proposed classifiers on two artificial and ten real-world datasets that cut across a wide range of application areas including handwriting recognition, medical diagnosis and remote sensing, and then compare our algorithm against existing LDA approaches and other linear classifiers. The GLD is shown to outperform the original LDA procedure in terms of the classification accuracy under heteroscedasticity. While it compares favourably with other existing heteroscedastic LDA approaches, the GLD requires as much as 60 times lower training time on some datasets. Our comparison with the support vector machine (SVM) also shows that, the GLD, together with the LNS, requires as much as 150 times lower training time to achieve an equivalent classification accuracy on some of the datasets. Thus, our algorithms can provide a cheap and reliable option for classification in a lot of expert systems

A Simple Iterative Algorithm for Parsimonious Binary Kernel Fisher Discrimination

By applying recent results in optimization theory variously known as optimization transfer or majorize/minimize algorithms, an algorithm for binary, kernel, Fisher discriminant analysis is introduced that makes use of a non-smooth penalty on the coefficients to provide a parsimonious solution. The problem is converted into a smooth optimization that can be solved iteratively with no greater overhead than iteratively re-weighted least-squares. The result is simple, easily programmed and is shown to perform, in terms of both accuracy and parsimony, as well as or better than a number of leading machine learning algorithms on two well-studied and substantial benchmarks

Target differentiation with simple infrared sensors using statistical pattern recognition techniques

Cataloged from PDF version of article.This study compares the performances of various statistical pattern recognition techniques for the differentiation of commonly encountered features in indoor environments, possibly with different surface properties, using simple infrared (IR) sensors. The intensity measurements obtained from such sensors are highly dependent on the location, geometry, and surface properties of the reflecting feature in a way that cannot be represented by a simple analytical relationship, therefore complicating the differentiation process. We construct feature vectors based on the parameters of angular IR intensity scans from different targets to determine their geometry and/or surface type. Mixture of normals classifier with three components correctly differentiates three types of geometries with different surface properties, resulting in the best performance (100%) in geometry differentiation. Parametric differentiation correctly identifies six different surface types of the same planar geometry, resulting in the best surface differentiation rate (100%). However, this rate is not maintained with the inclusion of more surfaces. The results indicate that the geometrical properties of the targets are more distinctive than their surface properties, and surface recognition is the limiting factor in differentiation. The results demonstrate that simple IR sensors, when coupled with appropriate processing and recognition techniques, can be used to extract substantially more information than such devices are commonly employed for. (C) 2007 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserve

Robust linear discriminant analysis using MOM-Qn and WMOM-Qn estimators: Coordinate-wise approach

Robust linear discriminant analysis (RLDA) methods are becoming the better choice for classification problems as compared to the classical linear discriminant analysis (LDA) due to their ability in circumventing outliers issue. Classical LDA relies on the usual location and scale estimators which are the sample mean and covariance matrix. The sensitivity of these estimators towards outliers will jeopardize the classification process. To alleviate the issue, robust estimators of location and covariance are proposed. Thus, in this study, two RLDA for two groups classification were modified using two highly robust location estimators namely Modified One-Step M-estimator (MOM) and Winsorized Modified One-Step M-estimator (WMOM). Integrated with a highly robust scale estimator, Qn, in the trimming criteria of MOM and WMOM, two new RLDA were developed known as RLDAMQ and RLDAWMQ respectively. In the computation of the new RLDA, the usual mean is replaced by MOM-Qn and WMOM-Qn accordingly. The performance of the new RLDA were tested on simulated as well as real data and then compared against the classical LDA. For simulated data, several variables were manipulated to create various conditions that always occur in real life. The variables were homogeneity of covariance (equal and unequal), samples (balanced and unbalanced), dimension of variables, and the percentage of contamination. In general, the results show that the performance of the new RLDA are more favorable than the classical LDA in terms of average misclassification error for contaminated data, although the new RLDA have the shortcoming of requiring more computational time. RLDAMQ works best under balanced sample sizes while RLDAWMQ surpasses the others under unbalanced sample sizes. When real financial data were considered, RLDAMQ shows capability in handling outliers with lowest misclassification error. As a conclusion, this research has achieved its primary objective which is to develop new RLDA for two groups classification of multivariate data in the presence of outliers

Revisiting probabilistic neural networks: a comparative study with support vector machines and the microhabitat suitability for the Eastern Iberian chub (Squalius valentinus)

[EN] Probabilistic Neural Networks (PNNs) and Support Vector Machines (SVMs) are flexible classification techniques suited to render trustworthy species distribution and habitat suitability models. Although several alternatives to improve PNNs¿ reliability and performance and/or to reduce computational costs exist, PNNs are currently not well recognised as SVMs because the SVMs were compared with standard PNNs. To rule out this idea, the microhabitat suitability for the Eastern Iberian chub (Squalius valentinus Doadrio & Carmona, 2006) was modelled with SVMs and four types of PNNs (homoscedastic, heteroscedastic, cluster and enhanced PNNs); all of them optimised with differential evolution. The fitness function and several performance criteria (correctly classified instances, true skill statistic, specificity and sensitivity) and partial dependence plots were used to assess respectively the performance and reliability of each habitat suitability model. Heteroscedastic and enhanced PNNs achieved the highest performance in every index but specificity. However, these two PNNs rendered ecologically unreliable partial dependence plots. Conversely, homoscedastic and cluster PNNs rendered ecologically reliable partial dependence plots. Thus, Eastern Iberian chub proved to be a eurytopic species, presenting the highest suitability in microhabitats with cover present, low flow velocity (approx. 0.3 m/s), intermediate depth (approx. 0.6 m) and fine gravel (64¿256 mm). PNNs outperformed SVMs; thus, based on the results of the cluster PNN, which also showed high values of the performance criteria, we would advocate a combination of approaches (e.g., cluster & heteroscedastic or cluster & enhanced PNNs) to balance the trade-off between accuracy and reliability of habitat suitability models.The study has been partially funded by the national Research project IMPADAPT (CGL2013-48424-C2-1-R) with MINECO (Spanish Ministry of Economy) and Feder funds and by the Confederacion Hidrografica del Near (Spanish Ministry of Agriculture and Fisheries, Food and Environment). This study was also supported in part by the University Research Administration Center of the Tokyo University of Agriculture and Technology. Thanks to Maria Jose Felipe for reviewing the mathematical notation and to the two anonymous reviewers who helped to improve the manuscript.Muñoz Mas, R.; Fukuda, S.; Portolés, J.; Martinez-Capel, F. (2018). Revisiting probabilistic neural networks: a comparative study with support vector machines and the microhabitat suitability for the Eastern Iberian chub (Squalius valentinus). Ecological Informatics. 43:24-37. https://doi.org/10.1016/J.ECOINF.2017.10.008S24374

New Modelling of Modified Two Dimensional Fisherface Based Feature Extraction

Biometric researches have been interesting field for many researches included facial recognition. Crucial process of facial recognition is feature extraction. One Dimensional Linear Discriminant Analysis is one of feature extraction method is development of Principal Component Analysis mostly used by researches. But, it has limitation, it can efficiently work when number of training sets greater or equal than number of dimensions of image training set. This limitation has been overcome by using Two Dimensional Linear Discriminant Analysis. However, search value of matrix identity R and L by using Two Dimensional Linear Discriminant Analysis takes high cost, which is O(n3). In this research, the seeking of “Scatter between Class” and “Scatter within Class” by using Discriminant Analysis without having to find the value of R and L advance are proposed. Time complexity of proposed method is O(n2). Proposed method has been tested by using AT&T face image database. The experimental results show that maximum recognition rate of proposed method is 100%

画像情報を利用した複数識別統合による性別と年齢層の識別

制度:新 ; 文部省報告番号:甲2483号 ; 学位の種類:博士(工学) ; 授与年月日:2007/7/26 ; 早大学位記番号:新459