IMPROVED STATISTICS ESTIMATION AND FEATURE EXTRACTION FOR HYPERSPECTRAL DATA CLASSIFICATION

For hyperspectral data classification, the avoidance of singularity of covariance estimates or excessive near singularity estimation error due to limited training data is a key problem. This study is intended to solve problem via regularized covariance estimators and feature extraction algorithms. A second purpose is to build a robust classification procedure with the advantages of the algorithms proposed in this study but robust in the sense of not requiring extensive analyst operator skill. A pair of covariance estimators called Mixed-LOOCs is proposed for avoiding excessive covariance estimator error. Mixed-LOOC2 has advantages over LOOC and BLOOC and needs less computation than those two. Based on Mixed-LOOC2, new DAFE and mixture classifier algorithms are proposed. Current feature extraction algorithms, while effective in some circumstances, have significant limitations. Discriminate analysis feature extraction (DAFE) is fast but does not perform well with classes whose mean values are similar, and it produces only N-1 reliable features where N is the number of classes. Decision Boundary Feature Extraction does not have these limitations but does not perform well when training sets are small, A new nonparametric feature extraction method (NWFE) is developed to solve the problems of DAFE and DBFE. NWFE takes advantage of the desirable characteristics of DAFE and DBFE, while avoiding their shortcomings. Finally, experimental results show that using NWFE features applied to a mixture classifier based on the Mixed-LOOC2 covariance estimator has the best performance and is a robust procedure for classifying hyperspectral data

Dimension Reduction by Mutual Information Discriminant Analysis

In the past few decades, researchers have proposed many discriminant analysis (DA) algorithms for the study of high-dimensional data in a variety of problems. Most DA algorithms for feature extraction are based on transformations that simultaneously maximize the between-class scatter and minimize the withinclass scatter matrices. This paper presents a novel DA algorithm for feature extraction using mutual information (MI). However, it is not always easy to obtain an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction that is based on one-dimensional MI estimations. We will refer to this algorithm as mutual information discriminant analysis (MIDA). The performance of this proposed method was evaluated using UCI databases. The results indicate that MIDA provides robust performance over different data sets with different characteristics and that MIDA always performs better than, or at least comparable to, the best performing algorithms.Comment: 13pages, 3 tables, International Journal of Artificial Intelligence & Application

Pattern recognition characterizations of micromechanical and morphological materials states via analytical quantitative ultrasonics

One potential approach to the quantitative acquisition of discriminatory information that can isolate a single structural state is pattern recognition. The pattern recognition characterizations of micromechanical and morphological materials states via analytical quantiative ultrasonics are outlined. The concepts, terminology, and techniques of statistical pattern recognition are reviewed. Feature extraction and classification and states of the structure can be determined via a program of ultrasonic data generation

Nonparametric Transient Classification using Adaptive Wavelets

Classifying transients based on multi band light curves is a challenging but crucial problem in the era of GAIA and LSST since the sheer volume of transients will make spectroscopic classification unfeasible. Here we present a nonparametric classifier that uses the transient's light curve measurements to predict its class given training data. It implements two novel components: the first is the use of the BAGIDIS wavelet methodology - a characterization of functional data using hierarchical wavelet coefficients. The second novelty is the introduction of a ranked probability classifier on the wavelet coefficients that handles both the heteroscedasticity of the data in addition to the potential non-representativity of the training set. The ranked classifier is simple and quick to implement while a major advantage of the BAGIDIS wavelets is that they are translation invariant, hence they do not need the light curves to be aligned to extract features. Further, BAGIDIS is nonparametric so it can be used for blind searches for new objects. We demonstrate the effectiveness of our ranked wavelet classifier against the well-tested Supernova Photometric Classification Challenge dataset in which the challenge is to correctly classify light curves as Type Ia or non-Ia supernovae. We train our ranked probability classifier on the spectroscopically-confirmed subsample (which is not representative) and show that it gives good results for all supernova with observed light curve timespans greater than 100 days (roughly 55% of the dataset). For such data, we obtain a Ia efficiency of 80.5% and a purity of 82.4% yielding a highly competitive score of 0.49 whilst implementing a truly "model-blind" approach to supernova classification. Consequently this approach may be particularly suitable for the classification of astronomical transients in the era of large synoptic sky surveys.Comment: 14 pages, 8 figures. Published in MNRA

Grid base classifier in comparison to nonparametric methods in multiclass classification

In this paper, a new method known as Grid Base Classifier was proposed. This method carries the advantages of the two previous methods in order to improve the classification tasks. The problem with the current lazy algorithms is that they learn quickly, but classify very slowly. On the other hand, the eager algorithms classify quickly, but they learn very slowly. The two algorithms were compared, and the proposed algorithm was found to be able to both learn and classify quickly. The method was developed based on the grid structure which was done to create a powerful method for classification. In the current research, the new algorithm was tested and applied to the multiclass classification of two or more categories, which are important for handling problems related to practical classification. The new method was also compared with the Levenberg-Marquardt back-propagation neural network in the learning stage and the Condensed nearest neighbour in the generalization stage to examine the performance of the model. The results from the artificial and real-world data sets (from UCI Repository) showed that the new method could improve both the efficiency and accuracy of pattern classification

Supervised Classification: Quite a Brief Overview

The original problem of supervised classification considers the task of automatically assigning objects to their respective classes on the basis of numerical measurements derived from these objects. Classifiers are the tools that implement the actual functional mapping from these measurements---also called features or inputs---to the so-called class label---or output. The fields of pattern recognition and machine learning study ways of constructing such classifiers. The main idea behind supervised methods is that of learning from examples: given a number of example input-output relations, to what extent can the general mapping be learned that takes any new and unseen feature vector to its correct class? This chapter provides a basic introduction to the underlying ideas of how to come to a supervised classification problem. In addition, it provides an overview of some specific classification techniques, delves into the issues of object representation and classifier evaluation, and (very) briefly covers some variations on the basic supervised classification task that may also be of interest to the practitioner

Classification software technique assessment

A catalog of software options is presented for the use of local user communities to obtain software for analyzing remotely sensed multispectral imagery. The resources required to utilize a particular software program are described. Descriptions of how a particular program analyzes data and the performance of that program for an application and data set provided by the user are shown. An effort is made to establish a statistical performance base for various software programs with regard to different data sets and analysis applications, to determine the status of the state-of-the-art

Supervised Machine Learning Under Test-Time Resource Constraints: A Trade-off Between Accuracy and Cost

The past decade has witnessed how the field of machine learning has established itself as a necessary component in several multi-billion-dollar industries. The real-world industrial setting introduces an interesting new problem to machine learning research: computational resources must be budgeted and cost must be strictly accounted for during test-time. A typical problem is that if an application consumes x additional units of cost during test-time, but will improve accuracy by y percent, should the additional x resources be allocated? The core of this problem is a trade-off between accuracy and cost. In this thesis, we examine components of test-time cost, and develop different strategies to manage this trade-off. We first investigate test-time cost and discover that it typically consists of two parts: feature extraction cost and classifier evaluation cost. The former reflects the computational efforts of transforming data instances to feature vectors, and could be highly variable when features are heterogeneous. The latter reflects the effort of evaluating a classifier, which could be substantial, in particular nonparametric algorithms. We then propose three strategies to explicitly trade-off accuracy and the two components of test-time cost during classifier training. To budget the feature extraction cost, we first introduce two algorithms: GreedyMiser and Anytime Representation Learning (AFR). GreedyMiser employs a strategy that incorporates the extraction cost information during classifier training to explicitly minimize the test-time cost. AFR extends GreedyMiser to learn a cost-sensitive feature representation rather than a classifier, and turns traditional Support Vector Machines (SVM) into test- time cost-sensitive anytime classifiers. GreedyMiser and AFR are evaluated on two real-world data sets from two different application domains, and both achieve record performance. We then introduce Cost Sensitive Tree of Classifiers (CSTC) and Cost Sensitive Cascade of Classifiers (CSCC), which share a common strategy that trades-off the accuracy and the amortized test-time cost. CSTC introduces a tree structure and directs test inputs along different tree traversal paths, each is optimized for a specific sub-partition of the input space, extracting different, specialized subsets of features. CSCC extends CSTC and builds a linear cascade, instead of a tree, to cope with class-imbalanced binary classification tasks. Since both CSTC and CSCC extract different features for different inputs, the amortized test-time cost is greatly reduced while maintaining high accuracy. Both approaches out-perform the current state-of-the-art on real-world data sets. To trade-off accuracy and high classifier evaluation cost of nonparametric classifiers, we propose a model compression strategy and develop Compressed Vector Machines (CVM). CVM focuses on the nonparametric kernel Support Vector Machines (SVM), whose test-time evaluation cost is typically substantial when learned from large training sets. CVM is a post-processing algorithm which compresses the learned SVM model by reducing and optimizing support vectors. On several benchmark data sets, CVM maintains high test accuracy while reducing the test-time evaluation cost by several orders of magnitude