Metaheuristics for the unit commitment problem : The Constraint Oriented Neighbourhoods search strategy

Tese de mestrado. Faculdade de Engenharia. Universidade do Porto. 199

Developments on a Multi-objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules

In this paper, we experiment with a combination of innovative approaches to rule induction to encourage the production of interesting sets of classification rules. These include multi-objective metaheuristics to induce the rules; measures of rule dissimilarity to encourage the production of dissimilar rules; and rule clustering algorithms to evaluate the results obtained. Our previous implementation of NSGA-II for rule induction produces a set of cc-optimal rules (coverage-confidence optimal rules). Among the set of rules produced there may be rules that are very similar. We explore the concept of rule similarity and experiment with a number of modifications of the crowding distance to increasing the diversity of the partial classification rules produced by the multi-objective algorithm

Aprendizado de regras de classificaçăo com otimizaçăo por nuvem de particulas multiobjetivo /

Orientadora : ProfŞ DrŞ Aurora PozoDissertaçăo (mestrado) - Universidade Federal do Paraná, Setor de Ciencias Exatas, Programa de Pós-Graduaçăo em Informática. Defesa: Curitiba, 2008Inclui bibliografi

Genetic Programming for Classification with Unbalanced Data

In classification,machine learning algorithms can suffer a performance bias when data sets are unbalanced. Binary data sets are unbalanced when one class is represented by only a small number of training examples (called the minority class), while the other class makes up the rest (majority class). In this scenario, the induced classifiers typically have high accuracy on the majority class but poor accuracy on the minority class. As the minority class typically represents the main class-of-interest in many real-world problems, accurately classifying examples from this class can be at least as important as, and in some cases more important than, accurately classifying examples from the majority class. Genetic Programming (GP) is a promising machine learning technique based on the principles of Darwinian evolution to automatically evolve computer programs to solve problems. While GP has shown much success in evolving reliable and accurate classifiers for typical classification tasks with balanced data, GP, like many other learning algorithms, can evolve biased classifiers when data is unbalanced. This is because traditional training criteria such as the overall success rate in the fitness function in GP, can be influenced by the larger number of examples from the majority class. This thesis proposes a GP approach to classification with unbalanced data. The goal is to develop new internal cost-adjustment techniques in GP to improve classification performances on both the minority class and the majority class. By focusing on internal cost-adjustment within GP rather than the traditional databalancing techniques, the unbalanced data can be used directly or "as is" in the learning process. This removes any dependence on a sampling algorithm to first artificially re-balance the input data prior to the learning process. This thesis shows that by developing a number of new methods in GP, genetic program classifiers with good classification ability on the minority and the majority classes can be evolved. This thesis evaluates these methods on a range of binary benchmark classification tasks with unbalanced data. This thesis demonstrates that unlike tasks with multiple balanced classes where some dynamic (non-static) classification strategies perform significantly better than the simple static classification strategy, either a static or dynamic strategy shows no significant difference in the performance of evolved GP classifiers on these binary tasks. For this reason, the rest of the thesis uses this static classification strategy. This thesis proposes several new fitness functions in GP to perform cost adjustment between the minority and the majority classes, allowing the unbalanced data sets to be used directly in the learning process without sampling. Using the Area under the Receiver Operating Characteristics (ROC) curve (also known as the AUC) to measure how well a classifier performs on the minority and majority classes, these new fitness functions find genetic program classifiers with high AUC on the tasks on both classes, and with fast GP training times. These GP methods outperform two popular learning algorithms, namely, Naive Bayes and Support Vector Machines on the tasks, particularly when the level of class imbalance is large, where both algorithms show biased classification performances. This thesis also proposes a multi-objective GP (MOGP) approach which treats the accuracies of the minority and majority classes separately in the learning process. The MOGP approach evolves a good set of trade-off solutions (a Pareto front) in a single run that perform as well as, and in some cases better than, multiple runs of canonical single-objective GP (SGP). In SGP, individual genetic program solutions capture the performance trade-off between the two objectives (minority and majority class accuracy) using an ROC curve; whereas in MOGP, this requirement is delegated to multiple genetic program solutions along the Pareto front. This thesis also shows how multiple Pareto front classifiers can be combined into an ensemble where individual members vote on the class label. Two ensemble diversity measures are developed in the fitness functions which treat the diversity on both the minority and the majority classes as equally important; otherwise, these measures risk being biased toward the majority class. The evolved ensembles outperform their individual members on the tasks due to good cooperation between members. This thesis further improves the ensemble performances by developing a GP approach to ensemble selection, to quickly find small groups of individuals that cooperate very well together in the ensemble. The pruned ensembles use much fewer individuals to achieve performances that are as good as larger (unpruned) ensembles, particularly on tasks with high levels of class imbalance, thereby reducing the total time to evaluate the ensemble

Classification systems optimization with multi-objective evolutionary algorithms

L'optimisation des systèmes de classification est une tâche complexe qui requiert l'intervention d'un spécialiste (expérimentateur). Cette tâche exige une bonne connaissance du domaine d'application afin de réaliser l'extraction de l'information pertinente pour la mise en oeuvre du système de classification ou de reconnaissance. L'extraction de caractéristiques est un processus itératif basé sur l'expérience. Normalement plusieurs évaluations de la performance en généralisation du système de reconnaissance, sur une base de données représentative du problème réel, sont requises pour trouver l'espace de représentation adéquat. Le processus d'extraction de caractéristiques est normalement suivi par une étape de sélection des caractéristiques pertinentes (FSS). L'objectif poursuivi est de réduire la complexité du système de reconnaissance tout en maintenant la performance en généralisation du système. Enfin, si le processus d'extraction de caractéristiques permet la génération de plusieurs représentations du problème, alors il est possible d'obtenir un gain en performance en combinant plusieurs classificateurs basés sur des représentations complémentaires. L'ensemble de classificateurs (EoC) permet éventuellement une meilleure performance en généralisation pour le système de reconnaissance. Nous proposons dans cette thèse une approche globale pour l'automatisation des tâches d'extraction, de sélection de caractéristiques et de sélection des ensembles de classificateurs basés sur l'optimisation multicritère. L'approche proposée est modulaire et celle-ci permet l'intégration de l'expertise de l'expérimentateur dans le processus d'optimisation. Deux algorithmes génétiques pour l'optimisation multicritère ont été évalués, le Fast Elitist Non-Dominated sorting Algorithm (NSGA-II) et le Multi-Objective Memetic Algorithm (MOMA). Les algorithmes d'optimisation ont été validés sur un problème difficile, soit la reconnaissance de chiffres manuscrits isolés tirés de la base NIST SD19. Ensuite, notre méthode a été utilisée une seule fois sur un problème de reconnaissance de lettres manuscrites, un problème de reconnaissance provenant du même domaine, pour lequel nous n'avons pas développé une grande expertise. Les résultats expérimentaux sont concluants et ceux-ci ont permis de démontrer que la performance obtenue dépasse celle de l'expérimentateur. Finalement, une contribution très importante de cette thèse réside dans la mise au point d'une méthode qui permet de visualiser et de contrôler le sur-apprentissage relié aux algorithmes génétiques utilisés pour l'optimisation des systèmes de reconnaissance. Les résultats expérimentaux révèlent que tous les problèmes d'optimisation étudiés (extraction et sélection de caractéristiques de même que la sélection de classificateurs) souffrent éventuellement du problème de sur-apprentissage. À ce jour, cet aspect n'a pas été traité de façon satisfaisante dans la littérature et nous avons proposé une solution efficace pour contribuer à la solution de ce problème d'apprentissage