A Study of Generalization and Fitness Landscapes for Neuroevolution

Rodrigues, N. M., Silva, S., & Vanneschi, L. (2020). A Study of Generalization and Fitness Landscapes for Neuroevolution. IEEE Access, 8, 108216-108234. [9113453]. https://doi.org/10.1109/ACCESS.2020.3001505Fitness landscapes are a useful concept for studying the dynamics of meta-heuristics. In the last two decades, they have been successfully used for estimating the optimization capabilities of different flavors of evolutionary algorithms, including genetic algorithms and genetic programming. However, so far they have not been used for studying the performance of machine learning algorithms on unseen data, and they have not been applied to studying neuroevolution landscapes. This paper fills these gaps by applying fitness landscapes to neuroevolution, and using this concept to infer useful information about the learning and generalization ability of the machine learning method. For this task, we use a grammar-based approach to generate convolutional neural networks, and we study the dynamics of three different mutations used to evolve them. To characterize fitness landscapes, we study autocorrelation, entropic measure of ruggedness, and fitness clouds. Also, we propose the use of two additional evaluation measures: density clouds and overfitting measure. The results show that these measures are appropriate for estimating both the learning and the generalization ability of the considered neuroevolution configurations.publishersversionpublishe

A Study of Fitness Landscapes for Neuroevolution

Rodrigues, N. M., Silva, S., & Vanneschi, L. (2020). A Study of Fitness Landscapes for Neuroevolution. In 2020 IEEE Congress on Evolutionary Computation, CEC 2020: Conference Proceedings [9185783] (2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CEC48606.2020.9185783Fitness landscapes are a useful concept to study the dynamics of meta-heuristics. In the last two decades, they have been applied with success to estimate the optimization power of several types of evolutionary algorithms, including genetic algorithms and genetic programming. However, so far they have never been used to study the performance of machine learning algorithms on unseen data, and they have never been applied to neuroevolution. This paper aims at filling both these gaps, applying for the first time fitness landscapes to neuroevolution and using them to infer useful information about the predictive ability of the method. More specifically, we use a grammar-based approach to generate convolutional neural networks, and we study the dynamics of three different mutations to evolve them. To characterize fitness landscapes, we study autocorrelation and entropic measure of ruggedness. The results show that these measures are appropriate for estimating both the optimization power and the generalization ability of the considered neuroevolution configurations.preprintpublishe

Exploring neuroevolution fitness landscapes for optimization and generalization

Tese de mestrado, Engenharia Informática (Interação e Conhecimento) Universidade de Lisboa, Faculdade de Ciências, 2020Paisagens de aptidão (fitness landscapes) são um conceito útil e largamente investigado para estudar as dinâmicas de meta-heurísticas. Nas últimas duas décadas têm sido utilizadas com sucesso para estimar as capacidades de otimização de diversos tipos de algoritmos evolutivos, tal como algoritmos genéticos e programação genética. No entanto, até à data nunca foram utilizadas para estudar o desempenho de algoritmos de aprendizagem automática em dados nunca vistos durante o treino, e nunca foram aplicadas para estudar as paisagens geradas por neuroevolução. Coincidentemente, apesar de já existir há quase três décadas e ainda ser uma área de investigação com um crescimento rápido e dinâmico, a neuroevolução ainda tem falta de fundações teóricas e metodológicas, fundações essas que podem ser dadas através da aplicação de paisagens de aptidão. Esta dissertação tem como objetivo preencher estas lacunas ao aplicar paisagens de aptidão à neuroevolução, usando este conceito para inferir informação útil sobre a capacidade de aprendizagem e generalização deste método de aprendizagem automática. De forma a realizar esta tarefa, desenvolvemos e usámos um algoritmo de neuroevolução baseado em gramáticas que gera redes neuronais convolucionais, e estudámos a dinâmica de três operadores de mutação distintos usados para evoluir múltiplos aspetos das redes neuronais. De forma a caracterizar as paisagens de aptidão, estudámos a autocorrelação (autocorrelation), medida entrópica de rugosidade (entropic measure of ruggedness), nuvens de aptidão (fitness clouds), medidas de gradiente (gradient measures) e o coeficiente de declive negativo (negative slope coefficient), e ao mesmo tempo discutimos porque é que apesar de não usarmos outras medidas, tais como redes de ótimos locais (local óptima networks) e correlação aptidão distância (fitness distance correlation), estas podem providenciar resultados interessantes. Também propomos o uso de duas novas medidas de avaliação: nuvens de densidade, uma nova medida desenvolvida nesta tese com capacidade de dar informação visual sobre a distribuição de amostras, e a medida de sobreajustamento (overfitting), que é derivada de uma medida já existente e usada em programação genética. Os resultados demonstram que as medidas usadas são apropriadas e produzem resultados precisos no que toca a estimar tanto a capacidade de aprendizagem como a habilidade de generalização das configuração de neuroevolução consideradas.Fitness landscapes are a useful and widely investigated concept for studying the dynamics of meta-heuristics. In the last two decades, they have been successfully used for estimating the optimization capabilities of different flavors of evolutionary algorithms, including genetic algorithms and genetic programming. However, so far they have not been used for studying the performance of Machine Learning (ML) algorithms on unseen data, and they have not been applied to study neuroevolution landscapes. Coincidentally, despite having existed for almost three decades and still being a dynamic and rapidly growing research field, neuroevolution still lacks theoretical and methodological foundations, which could be provided by the application of fitness landscapes. This thesis aims to fill these gaps by applying fitness landscapes to neuroevolution, using this concept to infer useful information about the learning and generalization ability of the ML method. For this task, we developed and used a grammar-based neuroevolution approach to generate convolutional neural networks, and studied the dynamics of three different mutation operators used to evolve multiple aspects of the networks. To characterize fitness landscapes, we studied autocorrelation, entropic measure of ruggedness, fitness clouds, gradient measures and negative slope coefficient, while also discussing why other measures such as local optima networks and fitness distance correlation, despite not being used, could provide interesting results. Also, we propose the use of two additional evaluation measures: density clouds, a new measure developed in this thesis that can provide visual information regarding the distribution of samples, and overfitting measure, which is derived from a measure used in genetic programming. The results show that the used measures are appropriate and produce accurate results when estimating both the learning capability and the generalization ability of the considered neuroevolution configurations

ES Is More Than Just a Traditional Finite-Difference Approximator

An evolution strategy (ES) variant based on a simplification of a natural evolution strategy recently attracted attention because it performs surprisingly well in challenging deep reinforcement learning domains. It searches for neural network parameters by generating perturbations to the current set of parameters, checking their performance, and moving in the aggregate direction of higher reward. Because it resembles a traditional finite-difference approximation of the reward gradient, it can naturally be confused with one. However, this ES optimizes for a different gradient than just reward: It optimizes for the average reward of the entire population, thereby seeking parameters that are robust to perturbation. This difference can channel ES into distinct areas of the search space relative to gradient descent, and also consequently to networks with distinct properties. This unique robustness-seeking property, and its consequences for optimization, are demonstrated in several domains. They include humanoid locomotion, where networks from policy gradient-based reinforcement learning are significantly less robust to parameter perturbation than ES-based policies solving the same task. While the implications of such robustness and robustness-seeking remain open to further study, this work's main contribution is to highlight such differences and their potential importance

The Case for a Mixed-Initiative Collaborative Neuroevolution Approach

It is clear that the current attempts at using algorithms to create artificial neural networks have had mixed success at best when it comes to creating large networks and/or complex behavior. This should not be unexpected, as creating an artificial brain is essentially a design problem. Human design ingenuity still surpasses computational design for most tasks in most domains, including architecture, game design, and authoring literary fiction. This leads us to ask which the best way is to combine human and machine design capacities when it comes to designing artificial brains. Both of them have their strengths and weaknesses; for example, humans are much too slow to manually specify thousands of neurons, let alone the billions of neurons that go into a human brain, but on the other hand they can rely on a vast repository of common-sense understanding and design heuristics that can help them perform a much better guided search in design space than an algorithm. Therefore, in this paper we argue for a mixed-initiative approach for collaborative online brain building and present first results towards this goal.Comment: Presented at WebAL-1: Workshop on Artificial Life and the Web 2014 (arXiv:1406.2507

Neuroevolution under unimodal error landscapes : an exploration of the semantic learning machine algorithm

Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceNeuroevolution is a field in which evolutionary algorithms are applied with the goal of evolving Artificial Neural Networks (ANNs). These evolutionary approaches can be used to evolve ANNs with fixed or dynamic topologies. This paper studies the Semantic Learning Machine (SLM) algorithm, a recently proposed neuroevolution method that searches over unimodal error landscapes in any supervised learning problem, where the error is measured as a distance to the known targets. SLM is compared with the topology-changing algorithm NeuroEvolution of Augmenting Topologies (NEAT) and with a fixed-topology neuroevolution approach. Experiments are performed on a total of 6 real-world datasets of classification and regression tasks. The results show that the best SLM variants outperform the other neuroevolution approaches in terms of generalization achieved, while also being more efficient in learning the training data. Further comparisons show that the best SLM variants also outperform the common ANN backpropagation-based approach under different topologies. A combination of the SLM with a recently proposed semantic stopping criterion also shows that it is possible to evolve competitive neural networks in a few seconds on the vast majority of the datasets considered.Neuro evolução é uma área onde algoritmos evolucionários são aplicados com o objetivo de evoluir Artificial Neural Networks (ANN). Estas abordagens evolucionárias podem ser utilizadas para evoluir ANNs com topologias fixas ou dinâmicas. Este artigo estuda o algoritmo de Semantic Learning Machine (SLM), um método de neuro evolução proposto recentemente que percorre paisagens de erros unimodais em qualquer problema de aprendizagem supervisionada, onde o erro é medido como a distância com os alvos conhecidos previamente. SLM é comparado com o algoritmo de alteração de topologias NeuroEvolution of Augmenting Topologies (NEAT) e com uma abordagem neuro evolucionária de topologias fixas. Experiências são realizadas em 6 datasets reais de tarefas de regressão e classificação. Os resultados mostram que as melhores variantes de SLM são mais capazes de generalizar quando comparadas com outras abordagens de neuro evolução, ao mesmo tempo que são mais eficientes no processo de treino. Mais comparações mostram que as melhores variantes de SLM são mais eficazes que as abordagens mais comuns de treino de ANN usando diferentes topologias e retro propagação. A combinação de SLM com um critério semântico de paragem do processo de treino também mostra que é possível criar redes neuronais competitivas em poucos segundos, na maioria dos datasets considerados

Evolving developmental, recurrent and convolutional neural networks for deliberate motion planning in sparse reward tasks

Motion planning algorithms have seen a diverse set of approaches in a variety of disciplines. In the domain of artificial evolutionary systems, motion planning has been included in models to achieve sophisticated deliberate behaviours. These algorithms rely on fixed rules or little evolutionary influence which compels behaviours to conform within those specific policies, rather than allowing the model to establish its own specialised behaviour. In order to further these models, the constraints imposed by planning algorithms must be removed to grant greater evolutionary control over behaviours. That is the focus of this thesis. An examination of prevailing neuroevolution methods led to the use of two distinct approaches, NEAT and HyperNEAT. Both were used to gain an understanding of the components necessary to create neuroevolution planning. The findings accumulated in the formation of a novel convolutional neural network architecture with a recurrent convolution process. The architecture’s goal was to iteratively disperse local activations to greater regions of the feature space. Experimentation showed significantly improved robustness over contemporary neuroevolution techniques as well as an efficiency increase over a static rule set. Greater evolutionary responsibility is given to the model with multiple network combinations; all of which continually demonstrated the necessary behaviours. In comparison, these behaviours were shown to be difficult to achieve in a state-of-the-art deep convolutional network. Finally, the unique use of recurrent convolution is relocated to a larger convolutional architecture on an established benchmarking platform. Performance improvements are seen on a number of domains which illustrates that this recurrent mechanism can be exploited in alternative areas outside of planning. By presenting a viable neuroevolution method for motion planning a potential emerges for further systems to adopt and examine the capability of this work in prospective domains, as well as further avenues of experimentation in convolutional architectures

Explorations of the semantic learning machine neuroevolution algorithm: dynamic training data use and ensemble construction methods

Dissertation presented as the partial requirement for obtaining a Master’s degree in Data Science and Advanced AnalyticsAs the world’s technology evolves, the power to implement new and more efficient algorithms increases but so does the complexity of the problems at hand. Neuroevolution algorithms fit in this context in the sense that they are able to evolve Artificial Neural Networks (ANNs). The recently proposed Neuroevolution algorithm called Semantic Learning Machine (SLM) has the advantage of searching over unimodal error landscapes in any Supervised Learning task where the error is measured as a distance to the known targets. The absence of local optima in the search space results in a more efficient learning when compared to other neuroevolution algorithms. This work studies how different approaches of dynamically using the training data affect the generalization of the SLM algorithm. Results show that these methods can be useful in offering different alternatives to achieve a superior generalization. These approaches are evaluated experimentally in fifteen real-world binary classification data sets. Across these fifteen data sets, results show that the SLM is able to outperform the Multilayer Perceptron (MLP) in 13 out of the 15 considered problems with statistical significance after parameter tuning was applied to both algorithms. Furthermore, this work also considers how different ensemble construction methods such as a simple averaging approach, Bagging and Boosting affect the resulting generalization of the SLM and MLP algorithms. Results suggest that the stochastic nature of the SLM offers enough diversity to the base learner in a way that a simple averaging method can be competitive when compared to more complex techniques like Bagging and Boosting.À medida que a tecnologia evolui, a possibilidade de implementar algoritmos novos e mais eficientes aumenta, no entanto, a complexidade dos problemas com que nos deparamos também se torna maior. Algoritmos de Neuroevolution encaixam-se neste contexto, na medida em que são capazes de evoluir Artificial Neural Networks (ANNs). O algoritmo de Neuroevolution recentemente proposto chamado Semantic Learning Machine (SLM) tem a vantagem de procurar sobre landscapes de erros unimodais em qualquer problema de Supervised Learning, onde o erro é medido como a distância aos alvos conhecidos. A não existência de local optima no espaço de procura resulta numa aprendizagem mais eficiente quando comparada com outros algoritmos de Neuroevolution. Este trabalho estuda como métodos diferentes de uso dinâmico de dados de treino afeta a generalização do algoritmo SLM. Os resultados mostram que estes métodos são úteis a oferecer uma alternativa que atinge uma generalização competitiva. Estes métodos são testados em quinze problemas reais de classificação binária. Nestes quinze problemas, o algoritmo SLM mostra superioridade ao Multilayer Perceptron (MLP) em treze deles com significância estatística depois de ser aplicado parameter tuning em ambos os algoritmos. Para além disso, este trabalho também considera como diferentes métodos de construção de ensembles, tal como um simples método de averaging, Bagging e Boosting afetam os valores de generalização dos algoritmos SLM e MLP. Os resultados sugerem que a natureza estocástica da SLM oferece diversidade suficiente aos base learners de maneira a que o método mais simples de construção de ensembles se torne competitivo quando comparado com técnicas mais complexas como Bagging e Boosting