Feature Selection with Evolving, Fast and Slow Using Two Parallel Genetic Algorithms

Feature selection is one of the most challenging issues in machine learning, especially while working with high dimensional data. In this paper, we address the problem of feature selection and propose a new approach called Evolving Fast and Slow. This new approach is based on using two parallel genetic algorithms having high and low mutation rates, respectively. Evolving Fast and Slow requires a new parallel architecture combining an automatic system that evolves fast and an effortful system that evolves slow. With this architecture, exploration and exploitation can be done simultaneously and in unison. Evolving fast, with high mutation rate, can be useful to explore new unknown places in the search space with long jumps; and Evolving Slow, with low mutation rate, can be useful to exploit previously known places in the search space with short movements. Our experiments show that Evolving Fast and Slow achieves very good results in terms of both accuracy and feature elimination

Evolving Ensemble Fuzzy Classifier

The concept of ensemble learning offers a promising avenue in learning from data streams under complex environments because it addresses the bias and variance dilemma better than its single model counterpart and features a reconfigurable structure, which is well suited to the given context. While various extensions of ensemble learning for mining non-stationary data streams can be found in the literature, most of them are crafted under a static base classifier and revisits preceding samples in the sliding window for a retraining step. This feature causes computationally prohibitive complexity and is not flexible enough to cope with rapidly changing environments. Their complexities are often demanding because it involves a large collection of offline classifiers due to the absence of structural complexities reduction mechanisms and lack of an online feature selection mechanism. A novel evolving ensemble classifier, namely Parsimonious Ensemble pENsemble, is proposed in this paper. pENsemble differs from existing architectures in the fact that it is built upon an evolving classifier from data streams, termed Parsimonious Classifier pClass. pENsemble is equipped by an ensemble pruning mechanism, which estimates a localized generalization error of a base classifier. A dynamic online feature selection scenario is integrated into the pENsemble. This method allows for dynamic selection and deselection of input features on the fly. pENsemble adopts a dynamic ensemble structure to output a final classification decision where it features a novel drift detection scenario to grow the ensemble structure. The efficacy of the pENsemble has been numerically demonstrated through rigorous numerical studies with dynamic and evolving data streams where it delivers the most encouraging performance in attaining a tradeoff between accuracy and complexity.Comment: this paper has been published by IEEE Transactions on Fuzzy System

Self-Configuring and Evolving Fuzzy Image Thresholding

Every segmentation algorithm has parameters that need to be adjusted in order to achieve good results. Evolving fuzzy systems for adjustment of segmentation parameters have been proposed recently (Evolving fuzzy image segmentation -- EFIS [1]. However, similar to any other algorithm, EFIS too suffers from a few limitations when used in practice. As a major drawback, EFIS depends on detection of the object of interest for feature calculation, a task that is highly application-dependent. In this paper, a new version of EFIS is proposed to overcome these limitations. The new EFIS, called self-configuring EFIS (SC-EFIS), uses available training data to auto-configure the parameters that are fixed in EFIS. As well, the proposed SC-EFIS relies on a feature selection process that does not require the detection of a region of interest (ROI).Comment: To appear in proceedings of The 14th International Conference on Machine Learning and Applications (IEEE ICMLA 2015), Miami, Florida, USA, 201

Accuracy Increase on Evolving Product Unit Neural Networks via Feature Subset Selection

A framework that combines feature selection with evolution ary artificial neural networks is presented. This paper copes with neural networks that are applied in classification tasks. In machine learning area, feature selection is one of the most common techniques for pre processing the data. A set of filters have been taken into consideration to assess the proposal. The experimentation has been conducted on nine data sets from the UCI repository that report test error rates about fif teen percent or above with reference classifiers such as C4.5 or 1-NN. The new proposal significantly improves the baseline framework, both approaches based on evolutionary product unit neural networks. Also several classifiers have been tried in order to illustrate the performance of the different methods considered.Comisión Interministerial de ciencia y Tecnología TIN2011-28956-C02- 02Comisión Interministerial de Ciencia y Tecnología TIN2014-55894-C2-RJunta de Andalucía P11-TIC-752

Heterogeneous probabilistic models for optimisation and modelling of evolving spiking neural networks

This thesis proposes a novel feature selection and classification method employing evolving spiking neural networks (eSNN) and evolutionary algorithms (EA). The method is named the Quantum-inspired Spiking Neural Network (QiSNN) framework. QiSNN represents an integrated wrapper approach. An evolutionary process evolves appropriate feature subsets for a given classification task and simultaneously optimises the neural and learning-related parameters of the network. Unlike other methods, the connection weights of this network are determined by a fast one-pass learning algorithm which dramatically reduces the training time. In its core, QiSNN employs the Thorpe neural model that allows the efficient simulation of even large networks. In QiSNN, the presence or absence of features is represented by a string of concatenated bits, while the parameters of the neural network are continuous. For the exploration of these two entirely different search spaces, a novel Estimation of Distribution Algorithm (EDA) is developed. The method maintains a population of probabilistic models specialised for the optimisation of either binary, continuous or heterogeneous search spaces while utilising a small and intuitive set of parameters. The EDA extends the Quantum-inspired Evolutionary Algorithm (QEA) proposed by Han and Kim (2002) and was named the Heterogeneous Hierarchical Model EDA (hHM-EDA). The algorithm is compared to numerous contemporary optimisation methods and studied in terms of convergence speed, solution quality and robustness in noisy search spaces. The thesis investigates the functioning and the characteristics of QiSNN using both synthetic feature selection benchmarks and a real-world case study on ecological modelling. By evolving suitable feature subsets, QiSNN significantly enhances the classification accuracy of eSNN. Compared to numerous other feature selection techniques, like the wrapper-based Multilayer Perceptron (MLP) and the Naive Bayesian Classifier (NBC), QiSNN demonstrates a competitive classification and feature selection performance while requiring comparatively low computational costs

Evolution of field early-type galaxies: The view from GOODS CDFS

We explore the evolution of field early-type galaxies in a sample extracted from the ACS images of the southern GOODS field. The galaxies are selected by means of a nonparametric analysis, followed by visual inspection of the candidates with a concentrated surface brightness distribution. We furthermore exclude from the final sample those galaxies that are not consistent with an evolution into the Kormendy relation between surface brightness and size that is observed for z = 0 ellipticals. The final set, which comprises 249 galaxies with a median redshift z(m) = 0.71, represents a sample of early-type systems not selected with respect to color, with similar scaling relations as those of bona fide elliptical galaxies. The distribution of number counts versus apparent magnitude rejects a constant number density with cosmic time and suggests a substantial decrease with redshift: n proportional to (1 + z)(-2.5). The majority of the galaxies (78%) feature passively evolving old stellar populations. One-third of those in the upper half of the redshift distribution have blue colors, in contrast to only 10% in the lower redshift subsample. An adaptive binning of the color maps using an optimal Voronoi tessellation is performed to explore the internal color distribution. We find that the red and blue early-type galaxies in our sample have distinct behavior with respect to the color gradients, so that most blue galaxies feature blue cores whereas most of the red early-types are passively evolving stellar populations with red cores, i.e., similar systems to local early-type galaxies. Furthermore, the color gradients and scatter do not evolve with redshift and are compatible with the observations at z 0, assuming a radial dependence of the metallicity within each galaxy. Significant gradients in the stellar age are readily ruled out. This work emphasizes the need for a careful sample selection, as we found that most of those galaxies that were visually classified as candidate early types-but then rejected based on the Kormendy relation-feature blue colors characteristic of recent star formation

Passively Evolving Early-type Galaxies at 1.4<z<2.5 in the Hubble Ultra Deep Field

We report on a complete sample of 7 luminous early-type galaxies in the Hubble Ultra Deep Field (UDF) with spectroscopic redshifts between 1.39 and 2.47 and to K<23 AB. Using the BzK selection criterion we have pre-selected a set of objects over the UDF which fulfill the photometric conditions for being passively evolving galaxies at z>1.4. Low-resolution spectra of these objects have been extracted from the HST+ACS grism data taken over the UDF by the GRAPES project. Redshift for the 7 galaxies have been identified based on the UV feature at rest frame 2640<lambda<2850 AA. This feature is mainly due to a combination of FeII, MgI and MgII absorptions which are characteristic of stellar populations dominated by stars older than about 0.5 Gyr. The redshift identification and the passively evolving nature of these galaxies is further supported by the photometric redshifts and by the overall spectral energy distribution (SED), with the ultradeep HST+ACS/NICMOS imaging revealing compact morphologies typical of elliptical/early-type galaxies. From the SED we derive stellar masses of 10^{11}Msun or larger and ages of about 1 Gyr. Their space density at =1.7 appears to be roughly a factor of 2--3 smaller than that of their local counterparts, further supporting the notion that such massive and old galaxies are already ubiquitous at early cosmic times. Much smaller effective radii are derived for some of the objects compared to local massive ellipticals, which may be due to morphological K corrections, evolution, or the presence of a central point-like source. Nuclear activity is indeed present in a subset of the galaxies, as revealed by them being hard X-ray sources, hinting to AGN activity having played a role in discontinuing star formation.Comment: 18 pages, 15 figures, ApJ in pres

Incremental learning in autonomous systems: evolving connectionist systems for on-line image and speech recognition

The paper presents an integrated approach to incremental learning in autonomous systems, that includes both pattern recognition and feature selection. The approach utilizes evolving connectionist systems (ECoS) and is applied on on-line image and speech pattern learning and recognition tasks. The experiments show that ECoS are a suitable paradigm for building autonomous systems for learning and navigation in a new environment using both image and speech modalities. © 2005 IEEE