1 research outputs found
Genetic Programming is Naturally Suited to Evolve Bagging Ensembles
Learning ensembles by bagging can substantially improve the generalization
performance of low-bias, high-variance estimators, including those evolved by
Genetic Programming (GP). To be efficient, modern GP algorithms for evolving
(bagging) ensembles typically rely on several (often inter-connected)
mechanisms and respective hyper-parameters, ultimately compromising ease of
use. In this paper, we provide experimental evidence that such complexity might
not be warranted. We show that minor changes to fitness evaluation and
selection are sufficient to make a simple and otherwise-traditional GP
algorithm evolve ensembles efficiently. The key to our proposal is to exploit
the way bagging works to compute, for each individual in the population,
multiple fitness values (instead of one) at a cost that is only marginally
higher than the one of a normal fitness evaluation. Experimental comparisons on
classification and regression tasks taken and reproduced from prior studies
show that our algorithm fares very well against state-of-the-art ensemble and
non-ensemble GP algorithms. We further provide insights into the proposed
approach by (i) scaling the ensemble size, (ii) ablating the changes to
selection, (iii) observing the evolvability induced by traditional subtree
variation. Code: https://github.com/marcovirgolin/2SEGP.Comment: Added interquartile range in tables 1, 2, and 3; improved Fig. 3 and
its analysis, improved experiment design of section 7.