Feature clustering for pso-based feature construction on high-dimensional data

Feature construction (FC) refers to a process that uses the original features to construct new features with better discrimination ability. Particle Swarm Optimisation (PSO) is an effective search technique that has been successfully utilised in FC. However, the application of PSO for feature construction using high dimensional data has been a challenge due to its large search space and high computational cost. Moreover, unnecessary features that were irrelevant, redundant and contained noise were constructed when PSO was applied to the whole feature. Therefore, the main purpose of this paper is to select the most informative features and construct new features from the selected features for a better classification performance. The feature clustering methods were used to aggregate similar features into clusters, whereby the dimensionality of the data was lowered by choosing representative features from every cluster to form the final feature subset. The clustering of each features are proven to be accurate in feature selection (FS), however, only one study investigated its application in FC for classification. The study identified some limitations, such as the implementation of only two binary classes and the decreasing accuracy of the data. This paper proposes a cluster based PSO feature construction approach called ClusPSOFC. The Redundancy-Based Feature Clustering (RFC) algorithm was applied to choose the most informative features from the original data, while PSO was used to construct new features from those selected by RFC. Experimental results were obtained by using six UCI data sets and six high-dimensional data to demonstrate the efficiency of the proposed method when compared to the original full features, other PSO based FC methods, and standard genetic programming based feature construction (GPFC). Hence, the ClusPSOFC method is effective for feature construction in the classification of high dimensional data

Semantic Feature Construction

An effective set of features is integral to the success of machine learning algorithms. Semantic feature construction is the knowledge-driven manipulation of the propositional descriptor space of a set of examples for use in a learning algorithm. Two important sources of semanticsfor feature construction are the semantic type (and associated semantic properties) and the semantic class of features. These semantics canbe captured in a knowledge base and utilized to constrain search through the space of constructed features. This dissertation presents a systemthat captures semantic feature construction knowledge and implements a search algorithm that respects that knowledge. Results are presentedfor different combinations of features generated from different successor functions used in search. These results are compiled over many learning problems and several learning algorithms. Other results are also presentedfor different levels of detail in semantic knowledge. Generally, semantics are an effective guide in the space of constructed features

Fragmentation problem and automated feature construction

Proceedings of the International Conference on Tools with Artificial Intelligence208-215PCTI