Otimização de hiperparâmetros em algoritmos de arvore de decisão utilizando computação evolutiva

Some algorithms in machine learning are parameterizable, they allow the configuration of parameters in order to increase the performance in some tasks. In most cases, these parameters are empirically found by the developer. Another approach is to use some optimization technique to find an optimized set of parameters. The aim of this project is the application of evolutionary algorithms, Genetic Algorithm (GA), Fluid Genetic Algorithm (FGA) and Genetic Algorithm using Theory of Chaos (GATC) to optimize the search for hyperparameters in decision tree algorithms. This work presents some satisfactory results within the data set tested, where the Classification and Regression Trees (CART) algorithm was used as a classifier algorithm for the tests. In these, the decision trees generated from the default values of the hyperparameters are compared with those optimized by the proposed approach. We has tried to optimize the accuracy and final size of the generated tree, which were successfully optimized by the proposed algorithms.Alguns algoritmos em aprendizado de máquina são parametrizáveis, ou seja, permitem a configuração de parâmetros de maneira a aumentar o desempenho na tarefa utilizada. Na maioria dos casos, estes parâmetros são encontrados empiricamente pelo desenvolvedor. Outra abordagem é utilizar alguma técnica de otimização para encontrar um conjunto otimizado de parâmetros. Este projeto tem por objetivo a aplicação dos algoritmos evolutivos, Algoritmo Genético (AG), Fluid Genetic Algorithm (FGA) e Genetic Algorithm using Theory of Chaos (GATC) para otimizar a busca de hiperparâmetros em algoritmos de ´arvores de decisão. Este trabalho apresenta alguns resultados satisfatórios dentro do conjunto de dados testados, onde o algoritmo Classification and. Regressivo Trees (CART) foi utilizado como algoritmo classificador para os testes. Nestes, as arvores de decisão geradas a partir dos valores padrão dos hiperparâmetros são comparados com os otimizados pela abordagem proposta. Buscou-se otimizar a acurácia e o tamanho final da ´arvore gerada, o que foram otimizadas com sucesso pelos algoritmos propostos

A population-based optimization method using Newton fractal

Department of Mathematical SciencesMetaheuristic is a general procedure to draw an agreement in a group based on the decision making of each individual beyond heuristic. For last decade, there have been many attempts to develop metaheuristic methods based on swarm intelligence to solve global optimization such as particle swarm optimizer, ant colony optimizer, firefly optimizer. These methods are mostly stochastic and independent on specific problems. Since metaheuristic methods based on swarm intelligence require no central coordination (or minimal, if any), they are especially well-applicable to those problems which have distributed or parallel structures. Each individual follows few simple rules, keeping the searching cost at a decent level. Despite its simplicity, the methods often yield a fast approximation in good precision, compared to conventional methods. Exploration and exploitation are two important features that we need to consider to find a global optimum in a high dimensional domain, especially when prior information is not given. Exploration is to investigate the unknown space without using the information from history to find undiscovered optimum. Exploitation is to trace the neighborhood of the current best to improve it using the information from history. Because these two concepts are at opposite ends of spectrum, the tradeoff significantly affects the performance at the limited cost of search. In this work, we develop a chaos-based metaheuristic method, ???Newton Particle Optimization(NPO)???, to solve global optimization problems. The method is based on the Newton method which is a well-established mathematical root-finding procedure. It actively utilizes the chaotic nature of the Newton method to place a proper balance between exploration and exploitation. While most current population-based methods adopt stochastic effects to maximize exploration, they often suffer from weak exploitation. In addition, stochastic methods generally show poor reproducing ability and premature convergence. It has been argued that an alternative approach using chaos may mitigate such disadvantages. The unpredictability of chaos is correspondent with the randomness of stochastic methods. Chaos-based methods are deterministic and therefore easy to reproduce the results with less memory. It has been shown that chaos avoids local optimum better than stochastic methods and buffers the premature convergence issue. Newton method is deterministic but shows chaotic movements near the roots. It is such complexity that enables the particles to search the space for global optimization. We initialize the particle???s position randomly at first and choose the ???leading particles??? to attract other particles near them. We can make a polynomial function whose roots are those leading particles, called ???a guiding function???. Then we update the positions of particles using the guiding function by Newton method. Since the roots are not updated by Newton method, the leading particles survive after update. For diverse movements of particles, we use modified newton method, which has a coefficient $m$ in the variation of movements for each particle. Efficiency in local search is closely related to the value of m which determines the convergence rate of the Newton method. We can control the balance between exploration and exploitation by choice of leading particles. It is interesting that selection of excellent particles as leading particles not always results in the best result. Including mediocre particles in the roots of guiding function maintains the diversity of particles in position. Though diversity seems to be inefficient at first, those particles contribute to the exploration for global search finally. We study the conditions for the convergence of NPO. NPO enjoys the well-established analysis of the Newton method. This contrasts with other ???nature-inspired??? algorithms which have often been criticized for lack of rigorous mathematical ground. We compare the results of NPO with those of two popular metaheuristic methods, particle swarm optimizer(PSO) and firefly optimizer(FO). Though it has been shown that there are no such algorithms superior to all problems by no free lunch theorem, that is why the researchers are concerned about adaptable global optimizer for specific problems. NPO shows good performance to CEC 2013 competition test problems comparing to PSO and FO.ope

Enhanced grey wolf optimisation algorithm for feature selection in anomaly detection

Anomaly detection deals with identification of items that do not conform to an expected pattern or items present in a dataset. The performance of different mechanisms utilized to perform the anomaly detection depends heavily on the group of features used. Thus, not all features in the dataset can be used in the classification process since some features may lead to low performance of classifier. Feature selection (FS) is a good mechanism that minimises the dimension of high-dimensional datasets by deleting the irrelevant features. Modified Binary Grey Wolf Optimiser (MBGWO) is a modern metaheuristic algorithm that has successfully been used for FS for anomaly detection. However, the MBGWO has several issues in finding a good quality solution. Thus, this study proposes an enhanced binary grey wolf optimiser (EBGWO) algorithm for FS in anomaly detection to overcome the algorithm issues. The first modification enhances the initial population of the MBGWO using a heuristic based Ant Colony Optimisation algorithm. The second modification develops a new position update mechanism using the Bat Algorithm movement. The third modification improves the controlled parameter of the MBGWO algorithm using indicators from the search process to refine the solution. The EBGWO algorithm was evaluated on NSL-KDD and six (6) benchmark datasets from the University California Irvine (UCI) repository against ten (10) benchmark metaheuristic algorithms. Experimental results of the EBGWO algorithm on the NSL-KDD dataset in terms of number of selected features and classification accuracy are superior to other benchmark optimisation algorithms. Moreover, experiments on the six (6) UCI datasets showed that the EBGWO algorithm is superior to the benchmark algorithms in terms of classification accuracy and second best for the number of selected features. The proposed EBGWO algorithm can be used for FS in anomaly detection tasks that involve any dataset size from various application domains

Zastosowanie procesów epigenetycznych w algorytmach genetycznych

The main purpose of the dissertation is to develop and verify the possibility of application mechanism based on epigenetics processes in genetics algorithms. In the dissertation three new operations in genetic algorithms were introduced. Epigenetics is the study of heritable phenotype changes that do not involve changes in the DNA sequence. The epigenetics processes was discovered later than the genetic algorithms was develop. Therefore, the epigenetic processes have not been included in classical genetic algorithms. In the dissertation the operations mimics the following epigenetic processes: prion inheritance, methylation of cytosine and allelic exclusion. The mentioned operations have been tested in terms of the impact on the efficiency of genetic algorithms, evaluating the reduction of the number of generations needed to obtain the expected result by the genetic algorithm, and the impact on the operation time of the algorithm. Research was carried out on four selected genetic algorithms: loading optimization, data grouping, outliers detection and biological sequence alignment. It was proved, based on the conducted research, that the application of each proposed operation in all algorithms used in the experiments influenced the reduction of the number of generations needed to obtain the expected result by algorithms. The following contents were included in the work: • description of optimization problems, • literature review on evolutionary algorithms, • the idea and operating principle of genetic algorithms were discussed, • literature review of existing modifications of genetic algorithms had been made, • discussed selected genetic algorithms, algorithms that were used as a algorithms of reference in the experiments, the results of which were included in the work, • the idea and method of implementation of proposed operations inspired by epigenetic processes been shown • the result of the experiments on the assessment of the impact of the proposed operations on the genetic algorithms was presented. Based on the results of the experiments presented in the work, the thesis of the dissertation has been proved