3 research outputs found
Clustering and combinatorial optimization in recursive supervised learning
The use of combinations of weak learners to learn a dataset has been shown to be
better than the use of a single strong learner. In fact, the idea is so successful that boosting, an
algorithm combining several weak learners for supervised learning, has been considered to
be the best off the shelf classifier. However, some problems still exist, including determining
the optimal number of weak learners and the over fitting of data. In an earlier work, we developed
the RPHP algorithm which solves both these problems by using a combination of global
search, weak learning and pattern distribution. In this chapter,werevise the global search component
by replacing it with a cluster based combinatorial optimization. Patterns are clustered
according to the output space of the problem, i.e., natural clusters are formed based on patterns
belonging to each class. A combinatorial optimization problem is therefore created, which is
solved using evolutionary algorithms. The evolutionary algorithms identify the “easy” and
the “difficult” clusters in the system. The removal of the easy patterns then givesway to the focused
learning of the more complicated patterns. The problem therefore becomes recursively
simpler. Over fitting is overcome by using a set of validation patterns along with a pattern distributor.
An algorithm is also proposed to use the pattern distributor to determine the optimal
number of recursions and hence the optimal number of weak learners for the problem. Empirical
studies showgenerally good performance when compared to other state of the art methods