Incremental construction of classifier and discriminant ensembles

We discuss approaches to incrementally construct an ensemble. The first constructs an ensemble of classifiers choosing a subset from a larger set, and the second constructs an ensemble of discriminants, where a classifier is used for some classes only. We investigate criteria including accuracy, significant improvement, diversity, correlation, and the role of search direction. For discriminant ensembles, we test subset selection and trees. Fusion is by voting or by a linear model. Using 14 classifiers on 38 data sets. incremental search finds small, accurate ensembles in polynomial time. The discriminant ensemble uses a subset of discriminants and is simpler, interpretable, and accurate. We see that an incremental ensemble has higher accuracy than bagging and random subspace method; and it has a comparable accuracy to AdaBoost. but fewer classifiers.We would like to thank the three anonymous referees and the editor for their constructive comments, pointers to related literature, and pertinent questions which allowed us to better situate our work as well as organize the ms and improve the presentation. This work has been supported by the Turkish Academy of Sciences in the framework of the Young Scientist Award Program (EA-TUBA-GEBIP/2001-1-1), Bogazici University Scientific Research Project 05HA101 and Turkish Scientific Technical Research Council TUBITAK EEEAG 104EO79Publisher's VersionAuthor Pre-Prin

Optimized classification predictions with a new index combining machine learning algorithms

Voting is a commonly used ensemble method aiming to optimize classification predictions by combining results from individual base classifiers. However, the selection of appropriate classifiers to participate in voting algorithm is currently an open issue. In this study we developed a novel Dissimilarity-Performance (DP) index which incorporates two important criteria for the selection of base classifiers to participate in voting: their differential response in classification (dissimilarity) when combined in triads and their individual performance. To develop this empirical index we firstly used a range of different datasets to evaluate the relationship between voting results and measures of dissimilarity among classifiers of different types (rules, trees, lazy classifiers, functions and Bayes). Secondly, we computed the combined effect on voting performance of classifiers with different individual performance and/or diverse results in the voting performance. Our DP index was able to rank the classifier combinations according to their voting performance and thus to suggest the optimal combination. The proposed index is recommended for individual machine learning users as a preliminary tool to identify which classifiers to combine in order to achieve more accurate classification predictions avoiding computer intensive and time-consuming search

Agnostic Bayes

Tableau d'honneur de la Faculté des études supérieures et postdorales, 2014-2015L’apprentissage automatique correspond à la science de l’apprentissage à partir d’exemples. Des algorithmes basés sur cette approche sont aujourd’hui omniprésents. Bien qu’il y ait eu un progrès significatif, ce domaine présente des défis importants. Par exemple, simplement sélectionner la fonction qui correspond le mieux aux données observées n’offre aucune garantie statistiques sur les exemples qui n’ont pas encore été observées. Quelques théories sur l’apprentissage automatique offrent des façons d’aborder ce problème. Parmi ceux-ci, nous présentons la modélisation bayésienne de l’apprentissage automatique et l’approche PACbayésienne pour l’apprentissage automatique dans une vue unifiée pour mettre en évidence d’importantes similarités. Le résultat de cette analyse suggère que de considérer les réponses de l’ensemble des modèles plutôt qu’un seul correspond à un des éléments-clés pour obtenir une bonne performance de généralisation. Malheureusement, cette approche vient avec un coût de calcul élevé, et trouver de bonnes approximations est un sujet de recherche actif. Dans cette thèse, nous présentons une approche novatrice qui peut être appliquée avec un faible coût de calcul sur un large éventail de configurations d’apprentissage automatique. Pour atteindre cet objectif, nous appliquons la théorie de Bayes d’une manière différente de ce qui est conventionnellement fait pour l’apprentissage automatique. Spécifiquement, au lieu de chercher le vrai modèle à l’origine des données observées, nous cherchons le meilleur modèle selon une métrique donnée. Même si cette différence semble subtile, dans cette approche, nous ne faisons pas la supposition que le vrai modèle appartient à l’ensemble de modèles explorés. Par conséquent, nous disons que nous sommes agnostiques. Plusieurs expérimentations montrent un gain de généralisation significatif en utilisant cette approche d’ensemble de modèles durant la phase de validation croisée. De plus, cet algorithme est simple à programmer et n’ajoute pas un coût de calcul significatif à la recherche d’hyperparamètres conventionnels. Finalement, cet outil probabiliste peut également être utilisé comme un test statistique pour évaluer la qualité des algorithmes sur plusieurs ensembles de données d’apprentissage.Machine learning is the science of learning from examples. Algorithms based on this approach are now ubiquitous. While there has been significant progress, this field presents important challenges. Namely, simply selecting the function that best fits the observed data was shown to have no statistical guarantee on the examples that have not yet been observed. There are a few learning theories that suggest how to address this problem. Among these, we present the Bayesian modeling of machine learning and the PAC-Bayesian approach to machine learning in a unified view to highlight important similarities. The outcome of this analysis suggests that model averaging is one of the key elements to obtain a good generalization performance. Specifically, one should perform predictions based on the outcome of every model instead of simply the one that best fits the observed data. Unfortunately, this approach comes with a high computational cost problem, and finding good approximations is the subject of active research. In this thesis, we present an innovative approach that can be applied with a low computational cost on a wide range of machine learning setups. In order to achieve this, we apply the Bayes’ theory in a different way than what is conventionally done for machine learning. Specifically, instead of searching for the true model at the origin of the observed data, we search for the best model according to a given metric. While the difference seems subtle, in this approach, we do not assume that the true model belongs to the set of explored model. Hence, we say that we are agnostic. An extensive experimental setup shows a significant generalization performance gain when using this model averaging approach during the cross-validation phase. Moreover, this simple algorithm does not add a significant computational cost to the conventional search of hyperparameters. Finally, this probabilistic tool can also be used as a statistical significance test to evaluate the quality of learning algorithms on multiple datasets

A forecasting solution to the oil spill problem based on a hybrid intelligent system

Oil spills represent one of the most destructive environmental disasters. Predicting the possibility of finding oil slicks in a certain area after an oil spill can be critical in reducing environmental risks. The system presented here uses the Case-Based Reasoning (CBR) methodology to forecast the presence or absence of oil slicks in certain open sea areas after an oil spill. CBR is a computational methodology designed to generate solutions to certain problems by analysing previous solutions given to previously solved problems. The proposed CBR system includes a novel network for data classification and retrieval. This type of network, which is constructed by using an algorithm to summarize the results of an ensemble of Self-Organizing Maps, is explained and analysed in the present study. The Weighted Voting Superposition (WeVoS) algorithm mainly aims to achieve the best topographically ordered representation of a dataset in the map. This study shows how the proposed system, called WeVoS-CBR, uses information such as salinity, temperature, pressure, number and area of the slicks, obtained from various satellites to accurately predict the presence of oil slicks in the north-west of the Galician coast, using historical data

Text categorization and ensemble pruning in Turkish news portals

Ankara : The Department of Computer Engineering and the Graduate School of Engineering and Science of Bilkent University, 2011.Thesis (Master's) -- Bilkent University, 2011.Includes bibliographical references leaves 53-60.In news portals, text category information is needed for news presentation. However, for many news stories the category information is unavailable, incorrectly assigned or too generic. This makes the text categorization a necessary tool for news portals. Automated text categorization (ATC) is a multifaceted diffi- cult process that involves decisions regarding tuning of several parameters, term weighting, word stemming, word stopping, and feature selection. It is important to find a categorization setup that will provide highly accurate results in ATC for Turkish news portals. Two Turkish test collections with different characteristics are created using Bilkent News Portal. Experiments are conducted with four classification methods: C4.5, KNN, Naive Bayes, and SVM (using polynomial and rbf kernels). Results recommend a text categorization template for Turkish news portals. Regarding recommended text categorization template, ensemble learning methods are applied to increase effectiveness. Since they require many computational workload, ensemble pruning strategies are developed. Data partitioning ensembles are constructed and ranked-based ensemble pruning is applied with several machine learning categorization algorithms. The aim is to answer the following questions: (1) How much data can we prune using data partitioning on the text categorization domain? (2) Which partitioning and categorization methods are more suitable for ensemble pruning? (3) How do English and Turkish differ in ensemble pruning? (4) Can we increase effectiveness with ensemble pruning in the text categorization? Experiments are conducted on two text collections: Reuters-21578 and BilCat-TRT. 90% of ensemble members can be pruned with almost no decreasing in accuracy.Toraman, ÇağrıM.S

Redes neurais especializadas para inferência de regime permanente em testes de performance de compressores de refrigeração

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Mecânica, Florianópolis, 2016.Ensaios de desempenho energético de compressores de refrigeração têm como objetivo a obtenção de parâmetros de desempenho, dentre eles a capacidade de refrigeração, sob condições preestabelecidas de operação. É uma etapa necessária no desenvolvimento e produção de compressores e que, devido às suas características de execução, torna-se um gargalo no processo de controle de qualidade. Trabalhos anteriores apresentam soluções associadas à inteligência artificial para reduzir o tempo de execução desses ensaios. Para isso, tais ferramentas analisam o comportamento de certas variáveis durante o período transiente do ensaio para inferir o momento no qual ocorre a transição para o regime permanente. Entretanto, a grande variedade de dinâmicas relacionadas a capacidade, temperatura de corpo e pressão de sucção afeta o desempenho das redes neurais. Para contornar esse problema, ensaios provenientes de uma empresa fabricante de compressores foram agrupados manualmente de acordo com regras que levaram em consideração a capacidade inicial e a região da faixa de aceitação pela qual os ensaios adentravam o regime permanente. Isso forneceu 4 grupos, os quais foram utilizados para treinamento de redes especializadas para inferência de regime permanente. Os melhores resultados vieram de redes especializadas treinadas com dois dos quatro grupos, que quando comparadas com aquelas treinadas com ensaios não agrupados dessas mesmas dinâmicas, apresentaram desempenho superior tanto nas taxas de acerto (aumento de até 6%) quanto nas taxas de erro de falso positivo (redução de até 17%). Paralelamente, foram realizadas análises estatísticas entre grupos na busca de comportamentos consideravelmente predominantes. Como destaque, um desses grupos apresentou duração de regime transitório consideravelmente inferior a dos demais. Além disso, como o agrupamento manual é lento e demanda um operador especialista, foi criado um método automático para esse fim que apresentou acerto de 92% em uma primeira análise. De forma geral, os resultados forneceram maior confiabilidade às redes de inferência e motivam a continuação dos estudos sobre as características relacionadas ao processo de treinamento.Abstract : Energetic performance tests of refrigerating compressors are used to obtain performance parameters, among them refrigerating capacity under pre-established operating condititons. It is a necessary step on development and compressor manufacturing, and due to its execution characteristics, become a bottleneck on quality control process. Previous works presented solutions based on artificial intelligence, which were used to reduce test time. For this, such tools analyse the behaviours of certain variables during the test transient state to infer on which moment occurs the transition to steady state. However, the considerable variety of dynamics related to such used parameters affects the neural network performance. To avoid such issue, tests from a compressor manufacturer were manually grouped accordingly to rules that took into account initial capacity value and acceptance band region by which the test entered steady state. This provided four groups, which were used to train specialized neural nwtworks. The best results came from specialized neural networks trained with two of those groups, which when compared to those neural netowkrs trained with non-grouped tests of these same dynamics, presents superior performance in both of true positive rate (up to 6% increase) and false positive rate (up to 17% decrease). Parallel to it, statistical analysis were conducted between groups in the search for predominant behaviours. As a highlight, one of these groups had both overall duration and transient state duration considerably lower than the other groups. In addition, since manual grouping is slow and requires a specialized operator and automatic method developed for this purpose, which showed a 92% accuracy in a first analysis. In general, results provided greater reliability to the inference neural networks and motivate the continuity of studies about training process related characteristics

Organization based multiagent architecture for distributed environments

[EN]Distributed environments represent a complex field in which applied solutions should be flexible and include significant adaptation capabilities. These environments are related to problems where multiple users and devices may interact, and where simple and local solutions could possibly generate good results, but may not be effective with regards to use and interaction. There are many techniques that can be employed to face this kind of problems, from CORBA to multi-agent systems, passing by web-services and SOA, among others. All those methodologies have their advantages and disadvantages that are properly analyzed in this documents, to finally explain the new architecture presented as a solution for distributed environment problems. The new architecture for solving complex solutions in distributed environments presented here is called OBaMADE: Organization Based Multiagent Architecture for Distributed Environments. It is a multiagent architecture based on the organizations of agents paradigm, where the agents in the architecture are structured into organizations to improve their organizational capabilities. The reasoning power of the architecture is based on the Case-Based Reasoning methology, being implemented in a internal organization that uses agents to create services to solve the external request made by the users. The OBaMADE architecture has been successfully applied to two different case studies where its prediction capabilities have been properly checked. Those case studies have showed optimistic results and, being complex systems, have demonstrated the abstraction and generalizations capabilities of the architecture. Nevertheless OBaMADE is intended to be able to solve much other kind of problems in distributed environments scenarios. It should be applied to other varieties of situations and to other knowledge fields to fully develop its potencial.[ES]Los entornos distribuidos representan un campo de conocimiento complejo en el que las soluciones a aplicar deben ser flexibles y deben contar con gran capacidad de adaptación. Este tipo de entornos está normalmente relacionado con problemas donde varios usuarios y dispositivos entran en juego. Para solucionar dichos problemas, pueden utilizarse sistemas locales que, aunque ofrezcan buenos resultados en términos de calidad de los mismos, no son tan efectivos en cuanto a la interacción y posibilidades de uso. Existen múltiples técnicas que pueden ser empleadas para resolver este tipo de problemas, desde CORBA a sistemas multiagente, pasando por servicios web y SOA, entre otros. Todas estas mitologías tienen sus ventajas e inconvenientes, que se analizan en este documento, para explicar, finalmente, la nueva arquitectura presentada como una solución para los problemas generados en entornos distribuidos. La nueva arquitectura aquí se llama OBaMADE, que es el acrónimo del inglés Organization Based Multiagent Architecture for Distributed Environments (Arquitectura Multiagente Basada en Organizaciones para Entornos Distribuidos). Se trata de una arquitectura multiagente basasa en el paradigma de las organizaciones de agente, donde los agentes que forman parte de la arquitectura se estructuran en organizaciones para mejorar sus capacidades organizativas. La capacidad de razonamiento de la arquitectura está basada en la metodología de razonamiento basado en casos, que se ha implementado en una de las organizaciones internas de la arquitectura por medio de agentes que crean servicios que responden a las solicitudes externas de los usuarios. La arquitectura OBaMADE se ha aplicado de forma exitosa a dos casos de estudio diferentes, en los que se han demostrado sus capacidades predictivas. Aplicando OBaMADE a estos casos de estudio se han obtenido resultados esperanzadores y, al ser sistemas complejos, se han demostrado las capacidades tanto de abstracción como de generalización de la arquitectura presentada. Sin embargo, esta arquitectura está diseñada para poder ser aplicada a más tipo de problemas de entornos distribuidos. Debe ser aplicada a más variadas situaciones y a otros campos de conocimiento para desarrollar completamente el potencial de esta arquitectura