Advances in Feature Selection with Mutual Information

The selection of features that are relevant for a prediction or classification problem is an important problem in many domains involving high-dimensional data. Selecting features helps fighting the curse of dimensionality, improving the performances of prediction or classification methods, and interpreting the application. In a nonlinear context, the mutual information is widely used as relevance criterion for features and sets of features. Nevertheless, it suffers from at least three major limitations: mutual information estimators depend on smoothing parameters, there is no theoretically justified stopping criterion in the feature selection greedy procedure, and the estimation itself suffers from the curse of dimensionality. This chapter shows how to deal with these problems. The two first ones are addressed by using resampling techniques that provide a statistical basis to select the estimator parameters and to stop the search procedure. The third one is addressed by modifying the mutual information criterion into a measure of how features are complementary (and not only informative) for the problem at hand

Swarm Intelligence In Myoelectric Control: Particle Swarm Based Dimensionality Reduction

The myoelectric signals (MES) from human muscles have been utilized in many applications such as prosthesis control. The identification of various MES temporal structures is used to control the movement of prosthetic devices by utilizing a pattern recognition approach. Recent advances in this field have shown that there are a number of factors limiting the clinical availability of such systems. Several control strategies have been proposed but deficiencies still exist with most of those strategies especially with the Dimensionality Reduction (DR) part. This paper proposes using Particle Swarm Optimization (PSO) algorithm with the concept of Mutual Information (MI) to produce a novel hybrid feature selection algorithm. The new algorithm, called PSOMIFS, is utilized as a DR tool in myoelectric control problems. The PSOMIFS will be compared with other techniques to prove the effectiveness of the proposed method. Accurate results are acquired using only a small subset of the original feature set producing a classification accuracy of 99% across a problem of ten classes based on tests done on six subjects MES datasets

High-Dimensional Feature Selection by Feature-Wise Kernelized Lasso

The goal of supervised feature selection is to find a subset of input features that are responsible for predicting output values. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on linear dependency between input features and output values. In this paper, we consider a feature-wise kernelized Lasso for capturing non-linear input-output dependency. We first show that, with particular choices of kernel functions, non-redundant features with strong statistical dependence on output values can be found in terms of kernel-based independence measures. We then show that the globally optimal solution can be efficiently computed; this makes the approach scalable to high-dimensional problems. The effectiveness of the proposed method is demonstrated through feature selection experiments with thousands of features.Comment: 18 page