Performing Feature Selection with ACO

Summary. The main aim of feature selection is to determine a minimal feature subset from a problem domain while retaining a suitably high accuracy in representing the original features. In real world problems FS is a must due to the abundance of noisy, irrelevant or misleading features. However, current methods are inadequate at finding optimal reductions. This chapter presents a feature selection mechanism based on Ant Colony Optimization in an attempt to combat this. The method is then applied to the problem of finding optimal feature subsets in the fuzzy-rough data reduction process. The present work is applied to two very different challenging tasks, namely web classification and complex systems monitoring.

Application of biosignal-driven intelligent systems for multifunction prosthesis control

University of Technology, Sydney. Faculty of Engineering and Information Technology.Prosthetic devices aim to provide an artificial alternative to missing limbs. The controller for such devices is usually driven by the biosignals generated by the human body, particularly Electromyogram (EMG) or Electroencephalogram (EEG) signals. Such a controller utilizes a pattern recognition approach to classify the EMG signal recorded from the human muscles or the EEG signal from the brain. The aim of this thesis is to improve the EMG and EEG pattern classification accuracy. Due to the fact that the success of pattern recognition based biosignal driven systems highly depends on the quality of extracted features, a number of novel, robust, hybrid and innovative methods are proposed to achieve better performance. These methods are developed to effectively tackle many of the limitations of existing systems, in particular feature representation and dimensionality reduction. A set of knowledge extraction methods that can accurately and rapidly identify the most important attributes for classifying the arm movements are formulated. This is accomplished through the following: 1. Developing a new feature extraction technique that can identify the most important features from the high-dimensional time-frequency representation of the multichannel EMG and EEG signals. For this task, an information content estimation method using fuzzy entropies and fuzzy mutual information is proposed to identify the optimal wravelet packet transform decomposition for classification. 2. Developing a powerful variable (feature or channel) selection paradigm to improve the performance of multi-channel EMG and EEG driven systems. This will eventually lead to the development of a combined channel and feature selection technique as one possible scheme for dimensionality reduction. Two novel feature selection methods are developed under this scheme utilizing the ant colony arid differential evolution optimization techniques. The differential evolution optimization technique is further modified in a novel attempt in employing a float optimizer for the combinatorial task of feature selection, proving powerful performance by both methods. 3. Developing two feature projection techniques that extract a small subset of highly informative discriminant features, thus acting as an alternative scheme for dimensionality reduction. The two methods represent novel variations to fuzzy discriminant analysis based projection techniques. In addition, an extension to the non-linear discriminant analysis is proposed based on a mixture of differential evolution and fuzzy discriminant analysis. The testing and verification process of the proposed methods on different EMG and EEG datasets provides very encouraging results

Fuzzy-Rough Data Reduction with Ant Colony Optimization

Feature selection refers to the problem of selecting those input features that are most predictive of a given outcome; a problem encountered in many areas such as machine learning, pattern recognition and signal processing. In particular, solution to this has found successful application in tasks that involve datasets containing huge numbers of features (in the order of tens of thousands), which would be impossible to process further. Recent examples include text processing and web content classification. Rough set theory has been used as such a dataset pre-processor with much success, but current methods are inadequate at finding minimal reductions, the smallest sets of features possible. To alleviate this difficulty, a feature selection technique that employs a hybrid variant of rough sets, fuzzy-rough sets, has been developed recently and has been shown to be effective. However, this method is still not able to find the optimal subsets regularly. This paper proposes a new feature selection mechanism based on Ant Colony Optimization in an attempt to combat this. The method is then applied to the problem of finding optimal feature subsets in the fuzzy-rough data reduction process. The present work is applied to complex systems monitoring and experimentally compared with the original fuzzy-rough method, an entropy-based feature selector, and a transformation-based reduction method, PCA. Comparisons with the use of a support vector classifier are also included

TSE-IDS: A Two-Stage Classifier Ensemble for Intelligent Anomaly-based Intrusion Detection System

Intrusion detection systems (IDS) play a pivotal role in computer security by discovering and repealing malicious activities in computer networks. Anomaly-based IDS, in particular, rely on classification models trained using historical data to discover such malicious activities. In this paper, an improved IDS based on hybrid feature selection and two-level classifier ensembles is proposed. An hybrid feature selection technique comprising three methods, i.e. particle swarm optimization, ant colony algorithm, and genetic algorithm, is utilized to reduce the feature size of the training datasets (NSL-KDD and UNSW-NB15 are considered in this paper). Features are selected based on the classification performance of a reduced error pruning tree (REPT) classifier. Then, a two-level classifier ensembles based on two meta learners, i.e., rotation forest and bagging, is proposed. On the NSL-KDD dataset, the proposed classifier shows 85.8% accuracy, 86.8% sensitivity, and 88.0% detection rate, which remarkably outperform other classification techniques recently proposed in the literature. Results regarding the UNSW-NB15 dataset also improve the ones achieved by several state of the art techniques. Finally, to verify the results, a two-step statistical significance test is conducted. This is not usually considered by IDS research thus far and, therefore, adds value to the experimental results achieved by the proposed classifier

FEATURE SELECTION APPLIED TO THE TIME-FREQUENCY REPRESENTATION OF MUSCLE NEAR-INFRARED SPECTROSCOPY (NIRS) SIGNALS: CHARACTERIZATION OF DIABETIC OXYGENATION PATTERNS

Diabetic patients might present peripheral microcirculation impairment and might benefit from physical training. Thirty-nine diabetic patients underwent the monitoring of the tibialis anterior muscle oxygenation during a series of voluntary ankle flexo-extensions by near-infrared spectroscopy (NIRS). NIRS signals were acquired before and after training protocols. Sixteen control subjects were tested with the same protocol. Time-frequency distributions of the Cohen's class were used to process the NIRS signals relative to the concentration changes of oxygenated and reduced hemoglobin. A total of 24 variables were measured for each subject and the most discriminative were selected by using four feature selection algorithms: QuickReduct, Genetic Rough-Set Attribute Reduction, Ant Rough-Set Attribute Reduction, and traditional ANOVA. Artificial neural networks were used to validate the discriminative power of the selected features. Results showed that different algorithms extracted different sets of variables, but all the combinations were discriminative. The best classification accuracy was about 70%. The oxygenation variables were selected when comparing controls to diabetic patients or diabetic patients before and after training. This preliminary study showed the importance of feature selection techniques in NIRS assessment of diabetic peripheral vascular impairmen