Analysing Symbolic Regression Benchmarks under a Meta-Learning Approach

The definition of a concise and effective testbed for Genetic Programming (GP) is a recurrent matter in the research community. This paper takes a new step in this direction, proposing a different approach to measure the quality of the symbolic regression benchmarks quantitatively. The proposed approach is based on meta-learning and uses a set of dataset meta-features---such as the number of examples or output skewness---to describe the datasets. Our idea is to correlate these meta-features with the errors obtained by a GP method. These meta-features define a space of benchmarks that should, ideally, have datasets (points) covering different regions of the space. An initial analysis of 63 datasets showed that current benchmarks are concentrated in a small region of this benchmark space. We also found out that number of instances and output skewness are the most relevant meta-features to GP output error. Both conclusions can help define which datasets should compose an effective testbed for symbolic regression methods.Comment: 8 pages, 3 Figures, Proceedings of Genetic and Evolutionary Computation Conference Companion, Kyoto, Japa

Data types as a more ergonomic frontend for Grammar-Guided Genetic Programming

Genetic Programming (GP) is an heuristic method that can be applied to many Machine Learning, Optimization and Engineering problems. In particular, it has been widely used in Software Engineering for Test-case generation, Program Synthesis and Improvement of Software (GI). Grammar-Guided Genetic Programming (GGGP) approaches allow the user to refine the domain of valid program solutions. Backus Normal Form is the most popular interface for describing Context-Free Grammars (CFG) for GGGP. BNF and its derivatives have the disadvantage of interleaving the grammar language and the target language of the program. We propose to embed the grammar as an internal Domain-Specific Language in the host language of the framework. This approach has the same expressive power as BNF and EBNF while using the host language type-system to take advantage of all the existing tooling: linters, formatters, type-checkers, autocomplete, and legacy code support. These tools have a practical utility in designing software in general, and GP systems in particular. We also present Meta-Handlers, user-defined overrides of the tree-generation system. This technique extends our object-oriented encoding with more practicability and expressive power than existing CFG approaches, achieving the same expressive power of Attribute Grammars, but without the grammar vs target language duality. Furthermore, we evidence that this approach is feasible, showing an example Python implementation as proof. We also compare our approach against textual BNF-representations w.r.t. expressive power and ergonomics. These advantages do not come at the cost of performance, as shown by our empirical evaluation on 5 benchmarks of our example implementation against PonyGE2. We conclude that our approach has better ergonomics with the same expressive power and performance of textual BNF-based grammar encodings

Ensemble SGE

Este documento diz respeito a um projeto de investigação que decorreu no âmbito de projeto de final de curso do Mestrado em Informática e Sistemas, ramo de Tecnologias de Informação e Conhecimento que decorreu no Instituto Superior de Engenharia de Coimbra. Está integrado na área da aprendizagem automática e tem como principal objetivo desenvolver uma nova framework suportada pelo SGE para resolver problemas de aprendizagem supervisionada, e tem o nome de Ensemble SGE. O Ensemble SGE, utiliza o SGE que é um algoritmo de evolução automática de programas, para gerar vários modelos capazes de resolver um problema. E posteriormente utiliza técnicas de aprendizagem por Ensemble para agregar alguns dos modelos gerados e produzir um Ensemble. Neste trabalho foram abordados 3 problemas de regressão simbólica. Duas aproximações a funções conhecidas, polinómio de quarto grau e o polinómio de Pagie e por fim Boston Housing, um problema em que dadas características de uma casa é necessário prever o seu preço. Os resultados deste projeto são positivos, é demonstrado que é possível obter Ensembles capazes de resolver alguns problemas de uma melhor forma, que o melhor modelo gerado pelo SGE. A performance obtida pela utilização de Ensembles é maior comparativamente a modelos simples gerados pelo SGE. A framework foi implementada e disponibilizada com possíveis casos de teste. Concluindo, a escolha dos modelos constituintes do Ensemble é a decisão mais importante, pois não foi encontrada nenhuma maneira exata de o fazer, ou seja, apenas por métodos experimentais. O Ensemble SGE também consegue detetar situações de overfitting mais cedo que o melhor modelo do SGE ao longo das gerações. Isto porque o Ensemble SGE utiliza vários indivíduos de uma população

Neuroevolution trajectory networks : illuminating the evolution of artificial neural networks

Neuroevolution is the discipline whereby ANNs are automatically generated using EC. This field began with the evolution of dense (shallow) neural networks for reinforcement learning task; neurocontrollers capable of evolving specific behaviours as required. Since then, neuroevolution has been used to discover architectures and hyperparameters of Deep Neural Networks, in ways never before conceived by human experts, with many achieving state-of-the-art results. Similar to other types of EAs, there is a wide variety of neuroevolution algorithms constantly being introduced. However, there is a lack of effective tools to examine these systems and assess whether they share underlying principles. This thesis proposes Neuroevolution Trajectory Networks (NTNs), an advanced visualisation tool that leverages complex networks to explore the intrinsic mechanisms inherent in the evolution of neural networks. In this research the tool was developed as a specialised version of Search Trajectory Networks, and it was particularly instantiated to illuminate the behaviour of algorithms navigating neuroevolution search spaces. Throughout the progress, this technique has been progressively applied from systems of shallow network evolution, to deep neural networks. The examination has focused on explicit characteristics of neuroevolution system. Specifically, the learnings achieved highlighted the importance of understanding the role of recombination in neuroevolution, revealing critical inefficiencies that hinder overall algorithm performance. A relation between neurocontrollers' diversity and exploration exists, as topological structures can influence the behavioural characterisations and the diversity generation of different search strategies. Furthermore, our analytical tool has offered insights into the favoured dynamics of transfer learning paradigm in the deep neuroevolution of Convolutional Neural Networks; shedding light on promising avenues for further research and development. All of the above have offered substantial evidence that this advanced tool can be regarded as a specialised observational technique to better understand the inner mechanics of neuroevolution and its specific components, beyond the assessment of accuracy and performance alone. This is done so that collective efforts can be concentrated on aspects that can further enhance the evolution of neural networks. Illuminating their search spaces can be seen as a first step to analysing neural network compositions