A Comparison of pattern classification techniques for orienting chest X-rays

The problem of orienting digital images of chest x-rays, which were captured at some multiple of 90 degrees from the true orientation, is a typical pattern classification problem. In this case, the solution to the problem must assign an instance of a digital image to one of four classes, where each class corresponds to one of the four possible orientations. A large number of techniques are available for developing a pattern classifier. Some of these techniques are characterized by independent variables whose values are difficult to relate back to the problem being solved. If a technique is highly sensitive to the values of these variables, the lack of a rigorous way of defining them can be a significant disadvantage to the inexperienced researcher. This thesis presents experiments by the author to solve the chest x-ray orientation problem using four different pattern classification techniques: genetic programming, an artificial neural network trained with back propagation, a probabilistic neural network, and a simple linear classifier. In addition, the author will demonstrate that an understanding of the design of a feature set may allow a programmer to develop a traditional program which does an adequate job of solving the classification problem. Comparisons of the different techniques will be based not only on their success at solving the problem, but also on the time required to find an acceptable solution and the degree to which each technique is sensitive to the values of the variables which characterize it. The thesis demonstrates that all of the techniques can be used to derive very accurate chest x-ray orientation classifiers. While it is dangerous to generalize the results of these experiments to pattern classification problems in general, the author will argue that the magnitude of the differences in performance between the different techniques minimizes this danger. In particular, the experiments suggest that the linear classifier is so computationally inexpensive that it is always worth trying, unless there is a priori knowledge that it will fail. The experiments also suggest that genetic programming is much more computationally expensive than are the linear classifier, artificial neural network, and probabilistic neural network techniques. Of the four conventional pattern classification techniques which were examined, it will be shown that the artificial neural network produced the most accurate classifiers for the x-ray orientation problem. In addition, the results of a number of trials suggest that the final accuracy of the classifier is relatively insensitive to the values of the parameters which characterize this technique, making it an appropriate choice for the inexperienced researcher. With respect to the ability of the resulting classifier to accurately orient sample x-rays which were not included in the training set, the artificial neural network performed well, when compared to the other techniques. Although the classifiers produced by the genetic programming technique were significantly more expensive to construct and were slightly less accurate than the best artificial neural networks, the results of genetic programming experiments can provide insights into the problem being studied, which would be difficult to discern from the classifiers produced by the other techniques. For example, one of the classifiers which was produced by genetic programming uses only eight of the twenty feature values extracted from the sample x-ray. Not only does this reduce the cost of extracting the feature values from an unknown sample, but the classifier itself would be much more efficient to evaluate than the classifiers produced by any of the other techniques

Voxel selection in fMRI data analysis based on sparse representation

Multivariate pattern analysis approaches toward detection of brain regions from fMRI data have been gaining attention recently. In this study, we introduce an iterative sparse-representation-based algorithm for detection of voxels in functional MRI (fMRI) data with task relevant information. In each iteration of the algorithm, a linear programming problem is solved and a sparse weight vector is subsequently obtained. The final weight vector is the mean of those obtained in all iterations. The characteristics of our algorithm are as follows: 1) the weight vector (output) is sparse; 2) the magnitude of each entry of the weight vector represents the significance of its corresponding variable or feature in a classification or regression problem; and 3) due to the convergence of this algorithm, a stable weight vector is obtained. To demonstrate the validity of our algorithm and illustrate its application, we apply the algorithm to the Pittsburgh Brain Activity Interpretation Competition 2007 functional fMRI dataset for selecting the voxels, which are the most relevant to the tasks of the subjects. Based on this dataset, the aforementioned characteristics of our algorithm are analyzed, and a comparison between our method with the univariate general-linear-model-based statistical parametric mapping is performed. Using our method, a combination of voxels are selected based on the principle of effective/sparse representation of a task. Data analysis results in this paper show that this combination of voxels is suitable for decoding tasks and demonstrate the effectiveness of our method

A generic optimising feature extraction method using multiobjective genetic programming

In this paper, we present a generic, optimising feature extraction method using multiobjective genetic programming. We re-examine the feature extraction problem and show that effective feature extraction can significantly enhance the performance of pattern recognition systems with simple classifiers. A framework is presented to evolve optimised feature extractors that transform an input pattern space into a decision space in which maximal class separability is obtained. We have applied this method to real world datasets from the UCI Machine Learning and StatLog databases to verify our approach and compare our proposed method with other reported results. We conclude that our algorithm is able to produce classifiers of superior (or equivalent) performance to the conventional classifiers examined, suggesting removal of the need to exhaustively evaluate a large family of conventional classifiers on any new problem. (C) 2010 Elsevier B.V. All rights reserved

Learning the LMP-Load Coupling From Data: A Support Vector Machine Based Approach

This paper investigates the fundamental coupling between loads and locational marginal prices (LMPs) in security-constrained economic dispatch (SCED). Theoretical analysis based on multi-parametric programming theory points out the unique one-to-one mapping between load and LMP vectors. Such one-to-one mapping is depicted by the concept of system pattern region (SPR) and identifying SPRs is the key to understanding the LMP-load coupling. Built upon the characteristics of SPRs, the SPR identification problem is modeled as a classification problem from a market participant's viewpoint, and a Support Vector Machine based data-driven approach is proposed. It is shown that even without the knowledge of system topology and parameters, the SPRs can be estimated by learning from historical load and price data. Visualization and illustration of the proposed data-driven approach are performed on a 3-bus system as well as the IEEE 118-bus system

Positive Semidefinite Metric Learning Using Boosting-like Algorithms

The success of many machine learning and pattern recognition methods relies heavily upon the identification of an appropriate distance metric on the input data. It is often beneficial to learn such a metric from the input training data, instead of using a default one such as the Euclidean distance. In this work, we propose a boosting-based technique, termed BoostMetric, for learning a quadratic Mahalanobis distance metric. Learning a valid Mahalanobis distance metric requires enforcing the constraint that the matrix parameter to the metric remains positive definite. Semidefinite programming is often used to enforce this constraint, but does not scale well and easy to implement. BoostMetric is instead based on the observation that any positive semidefinite matrix can be decomposed into a linear combination of trace-one rank-one matrices. BoostMetric thus uses rank-one positive semidefinite matrices as weak learners within an efficient and scalable boosting-based learning process. The resulting methods are easy to implement, efficient, and can accommodate various types of constraints. We extend traditional boosting algorithms in that its weak learner is a positive semidefinite matrix with trace and rank being one rather than a classifier or regressor. Experiments on various datasets demonstrate that the proposed algorithms compare favorably to those state-of-the-art methods in terms of classification accuracy and running time.Comment: 30 pages, appearing in Journal of Machine Learning Researc