Neural network ensembles: Evaluation of aggregation algorithms

Ensembles of artificial neural networks show improved generalization capabilities that outperform those of single networks. However, for aggregation to be effective, the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We present here an extensive evaluation of several algorithms for ensemble construction, including new proposals and comparing them with standard methods in the literature. We also discuss a potential problem with sequential aggregation algorithms: the non-frequent but damaging selection through their heuristics of particularly bad ensemble members. We introduce modified algorithms that cope with this problem by allowing individual weighting of aggregate members. Our algorithms and their weighted modifications are favorably tested against other methods in the literature, producing a sensible improvement in performance on most of the standard statistical databases used as benchmarks.Comment: 35 pages, 2 figures, In press AI Journa

SECA: a stepwise algorithm for construction of neural networks ensembles

Ensembles of artificial neural networks (ANN) have been used in the last years as classification/regression machines, showing improved generalization capabilities that outperform those of single networks. However, it has been recognized that for aggregation to be effective the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. Recently, we proposed a new method for constructing ANN ensembles —termed here Stepwise Ensemble Construction Algorithm (SECA)— which leads to overtrained aggregate members with an adequate balance between accuracy and diversity. We present here a more extensive evaluation of SECA and discuss a potential problem with this algorithm: the unfrequent but damaging selection through its heuristic of particularly bad ensemble members. We introduce a modified version of SECA that can cope with this problem by allowing individual weighing of aggregate members. The original algorithm and its weighed modification are favorably tested against other methods, producing an improvement in performance on the standard statistical databases used as benchmarks.Eje: Sistemas inteligentesRed de Universidades con Carreras en Informática (RedUNCI

Aggregation algorithms for regression : A comparison with boosting and SVM techniques

Classi cation and regression ensembles sho w generalization capabilities that outperform those of single predictors. We present here a further ev aluation of tw o algorithms for ensemble construction recently proposed by us. In particular, we compare them with Boosting and Support Vector Machine tec hniques, which are the newest and most sophisticated methods to treat classi cation and regression problems. We sho w that our comparatively simpler algorithms are very competitive with these tec hniques, showing even a sensible improvement in performance in some of the standard statistical databases used as benchmarks.Eje: Agentes y Sistemas Inteligentes (ASI)Red de Universidades con Carreras en Informática (RedUNCI

Aggregation algorithms for regression : A comparison with boosting and SVM techniques

Classi cation and regression ensembles sho w generalization capabilities that outperform those of single predictors. We present here a further ev aluation of tw o algorithms for ensemble construction recently proposed by us. In particular, we compare them with Boosting and Support Vector Machine tec hniques, which are the newest and most sophisticated methods to treat classi cation and regression problems. We sho w that our comparatively simpler algorithms are very competitive with these tec hniques, showing even a sensible improvement in performance in some of the standard statistical databases used as benchmarks.Eje: Agentes y Sistemas Inteligentes (ASI)Red de Universidades con Carreras en Informática (RedUNCI