7 research outputs found
Analysing Symbolic Regression Benchmarks under a Meta-Learning Approach
The definition of a concise and effective testbed for Genetic Programming
(GP) is a recurrent matter in the research community. This paper takes a new
step in this direction, proposing a different approach to measure the quality
of the symbolic regression benchmarks quantitatively. The proposed approach is
based on meta-learning and uses a set of dataset meta-features---such as the
number of examples or output skewness---to describe the datasets. Our idea is
to correlate these meta-features with the errors obtained by a GP method. These
meta-features define a space of benchmarks that should, ideally, have datasets
(points) covering different regions of the space. An initial analysis of 63
datasets showed that current benchmarks are concentrated in a small region of
this benchmark space. We also found out that number of instances and output
skewness are the most relevant meta-features to GP output error. Both
conclusions can help define which datasets should compose an effective testbed
for symbolic regression methods.Comment: 8 pages, 3 Figures, Proceedings of Genetic and Evolutionary
Computation Conference Companion, Kyoto, Japa
Neuroevolution under unimodal error landscapes : an exploration of the semantic learning machine algorithm
Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business IntelligenceNeuroevolution is a field in which evolutionary algorithms are applied with the goal
of evolving Artificial Neural Networks (ANNs). These evolutionary approaches can be
used to evolve ANNs with fixed or dynamic topologies. This paper studies the Semantic
Learning Machine (SLM) algorithm, a recently proposed neuroevolution method
that searches over unimodal error landscapes in any supervised learning problem,
where the error is measured as a distance to the known targets. SLM is compared with
the topology-changing algorithm NeuroEvolution of Augmenting Topologies (NEAT)
and with a fixed-topology neuroevolution approach. Experiments are performed on a
total of 6 real-world datasets of classification and regression tasks. The results show
that the best SLM variants outperform the other neuroevolution approaches in terms
of generalization achieved, while also being more efficient in learning the training data.
Further comparisons show that the best SLM variants also outperform the common
ANN backpropagation-based approach under different topologies. A combination of
the SLM with a recently proposed semantic stopping criterion also shows that it is
possible to evolve competitive neural networks in a few seconds on the vast majority
of the datasets considered.Neuro evolução é uma área onde algoritmos evolucionários são aplicados com o objetivo
de evoluir Artificial Neural Networks (ANN). Estas abordagens evolucionárias
podem ser utilizadas para evoluir ANNs com topologias fixas ou dinâmicas. Este artigo
estuda o algoritmo de Semantic Learning Machine (SLM), um método de neuro evolução
proposto recentemente que percorre paisagens de erros unimodais em qualquer
problema de aprendizagem supervisionada, onde o erro é medido como a distância
com os alvos conhecidos previamente. SLM é comparado com o algoritmo de alteração
de topologias NeuroEvolution of Augmenting Topologies (NEAT) e com uma abordagem
neuro evolucionária de topologias fixas. Experiências são realizadas em 6 datasets
reais de tarefas de regressão e classificação. Os resultados mostram que as melhores
variantes de SLM são mais capazes de generalizar quando comparadas com outras
abordagens de neuro evolução, ao mesmo tempo que são mais eficientes no processo
de treino. Mais comparações mostram que as melhores variantes de SLM são mais eficazes
que as abordagens mais comuns de treino de ANN usando diferentes topologias
e retro propagação. A combinação de SLM com um critério semântico de paragem do
processo de treino também mostra que é possível criar redes neuronais competitivas
em poucos segundos, na maioria dos datasets considerados
Computational Intelligence for Life Sciences
Computational Intelligence (CI) is a computer science discipline encompassing the theory, design, development and application of biologically and linguistically derived computational paradigms. Traditionally, the main elements of CI are Evolutionary Computation, Swarm Intelligence, Fuzzy Logic, and Neural Networks. CI aims at proposing new algorithms able to solve complex computational problems by taking inspiration from natural phenomena. In an intriguing turn of events, these nature-inspired methods have been widely adopted to investigate a plethora of problems related to nature itself. In this paper we present a variety of CI methods applied to three problems in life sciences, highlighting their effectiveness: we describe how protein folding can be faced by exploiting Genetic Programming, the inference of haplotypes can be tackled using Genetic Algorithms, and the estimation of biochemical kinetic parameters can be performed by means of Swarm Intelligence. We show that CI methods can generate very high quality solutions, providing a sound methodology to solve complex optimization problems in life sciences
Explorations of the semantic learning machine neuroevolution algorithm: dynamic training data use and ensemble construction methods
Dissertation presented as the partial requirement for obtaining a Master’s degree in Data Science and Advanced AnalyticsAs the world’s technology evolves, the power to implement new and more efficient
algorithms increases but so does the complexity of the problems at hand. Neuroevolution
algorithms fit in this context in the sense that they are able to evolve Artificial
Neural Networks (ANNs).
The recently proposed Neuroevolution algorithm called Semantic Learning Machine
(SLM) has the advantage of searching over unimodal error landscapes in any Supervised
Learning task where the error is measured as a distance to the known targets.
The absence of local optima in the search space results in a more efficient learning
when compared to other neuroevolution algorithms. This work studies how different
approaches of dynamically using the training data affect the generalization of the
SLM algorithm. Results show that these methods can be useful in offering different
alternatives to achieve a superior generalization. These approaches are evaluated experimentally
in fifteen real-world binary classification data sets. Across these fifteen
data sets, results show that the SLM is able to outperform the Multilayer Perceptron
(MLP) in 13 out of the 15 considered problems with statistical significance after parameter
tuning was applied to both algorithms.
Furthermore, this work also considers how different ensemble construction methods
such as a simple averaging approach, Bagging and Boosting affect the resulting generalization
of the SLM and MLP algorithms. Results suggest that the stochastic nature
of the SLM offers enough diversity to the base learner in a way that a simple averaging
method can be competitive when compared to more complex techniques like Bagging
and Boosting.À medida que a tecnologia evolui, a possibilidade de implementar algoritmos novos
e mais eficientes aumenta, no entanto, a complexidade dos problemas com que nos
deparamos também se torna maior. Algoritmos de Neuroevolution encaixam-se neste
contexto, na medida em que são capazes de evoluir Artificial Neural Networks (ANNs).
O algoritmo de Neuroevolution recentemente proposto chamado Semantic Learning
Machine (SLM) tem a vantagem de procurar sobre landscapes de erros unimodais em
qualquer problema de Supervised Learning, onde o erro é medido como a distância aos
alvos conhecidos. A não existência de local optima no espaço de procura resulta numa
aprendizagem mais eficiente quando comparada com outros algoritmos de Neuroevolution.
Este trabalho estuda como métodos diferentes de uso dinâmico de dados de
treino afeta a generalização do algoritmo SLM. Os resultados mostram que estes métodos
são úteis a oferecer uma alternativa que atinge uma generalização competitiva.
Estes métodos são testados em quinze problemas reais de classificação binária. Nestes
quinze problemas, o algoritmo SLM mostra superioridade ao Multilayer Perceptron
(MLP) em treze deles com significância estatística depois de ser aplicado parameter
tuning em ambos os algoritmos.
Para além disso, este trabalho também considera como diferentes métodos de construção
de ensembles, tal como um simples método de averaging, Bagging e Boosting
afetam os valores de generalização dos algoritmos SLM e MLP. Os resultados sugerem
que a natureza estocástica da SLM oferece diversidade suficiente aos base learners de
maneira a que o método mais simples de construção de ensembles se torne competitivo
quando comparado com técnicas mais complexas como Bagging e Boosting
Semantic Segmentation Network Stacking with Genetic Programming
Bakurov, I., Buzzelli, M., Schettini, R., Castelli, M., & Vanneschi, L. (2023). Semantic Segmentation Network Stacking with Genetic Programming. Genetic Programming And Evolvable Machines, 24(2 — Special Issue on Highlights of Genetic Programming 2022 Events), 1-37. [15]. https://doi.org/10.1007/s10710-023-09464-0---Open access funding provided by FCT|FCCN (b-on). This work was supported by national funds through the FCT (Fundação para a Ciência e a Tecnologia) by the projects GADgET (DSAIPA/DS/0022/2018), AICE (DSAIPA/DS/0113/2019), UIDB/04152/2020 - Centro de Investigação em Gestão de Informação (MagIC)/NOVA IMS, and by the grant SFRH/BD/137277/2018.Semantic segmentation consists of classifying each pixel of an image and constitutes an essential step towards scene recognition and understanding. Deep convolutional encoder–decoder neural networks now constitute state-of-the-art methods in the field of semantic segmentation. The problem of street scenes’ segmentation for automotive applications constitutes an important application field of such networks and introduces a set of imperative exigencies. Since the models need to be executed on self-driving vehicles to make fast decisions in response to a constantly changing environment, they are not only expected to operate reliably but also to process the input images rapidly. In this paper, we explore genetic programming (GP) as a meta-model that combines four different efficiency-oriented networks for the analysis of urban scenes. Notably, we present and examine two approaches. In the first approach, we represent solutions as GP trees that combine networks’ outputs such that each output class’s prediction is obtained through the same meta-model. In the second approach, we propose representing solutions as lists of GP trees, each designed to provide a unique meta-model for a given target class. The main objective is to develop efficient and accurate combination models that could be easily interpreted, therefore allowing gathering some hints on how to improve the existing networks. The experiments performed on the Cityscapes dataset of urban scene images with semantic pixel-wise annotations confirm the effectiveness of the proposed approach. Specifically, our best-performing models improve systems’ generalization ability by approximately 5% compared to traditional ensembles, 30% for the less performing state-of-the-art CNN and show competitive results with respect to state-of-the-art ensembles. Additionally, they are small in size, allow interpretability, and use fewer features due to GP’s automatic feature selection.publishersversionepub_ahead_of_prin
Ensemble learning with GSGP
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsThe purpose of this thesis is to conduct comparative research between Genetic Programming
(GP) and Geometric Semantic Genetic Programming (GSGP), with different
initialization (RHH and EDDA) and selection (Tournament and Epsilon-Lexicase)
strategies, in the context of a model-ensemble in order to solve regression optimization
problems.
A model-ensemble is a combination of base learners used in different ways to solve
a problem. The most common ensemble is the mean, where the base learners are combined
in a linear fashion, all having the same weights. However, more sophisticated
ensembles can be inferred, providing higher generalization ability.
GSGP is a variant of GP using different genetic operators. No previous research has
been conducted to see if GSGP can perform better than GP in model-ensemble learning.
The evolutionary process of GP and GSGP should allow us to learn about the strength
of each of those base models to provide a more accurate and robust solution. The
base-models used for this analysis were Linear Regression, Random Forest, Support
Vector Machine and Multi-Layer Perceptron. This analysis has been conducted using 7
different optimization problems and 4 real-world datasets. The results obtained with
GSGP are statistically significantly better than GP for most cases.O objetivo desta tese é realizar pesquisas comparativas entre Programação Genética
(GP) e Programação Genética Semântica Geométrica (GSGP), com diferentes estratégias
de inicialização (RHH e EDDA) e seleção (Tournament e Epsilon-Lexicase), no
contexto de um conjunto de modelos, a fim de resolver problemas de otimização de
regressão.
Um conjunto de modelos é uma combinação de alunos de base usados de diferentes
maneiras para resolver um problema. O conjunto mais comum é a média, na qual
os alunos da base são combinados de maneira linear, todos com os mesmos pesos.
No entanto, conjuntos mais sofisticados podem ser inferidos, proporcionando maior
capacidade de generalização.
O GSGP é uma variante do GP usando diferentes operadores genéticos. Nenhuma
pesquisa anterior foi realizada para verificar se o GSGP pode ter um desempenho
melhor que o GP no aprendizado de modelos. O processo evolutivo do GP e GSGP
deve permitir-nos aprender sobre a força de cada um desses modelos de base para
fornecer uma solução mais precisa e robusta. Os modelos de base utilizados para esta
análise foram: Regressão Linear, Floresta Aleatória, Máquina de Vetor de Suporte e
Perceptron de Camadas Múltiplas. Essa análise foi realizada usando 7 problemas de
otimização diferentes e 4 conjuntos de dados do mundo real. Os resultados obtidos
com o GSGP são estatisticamente significativamente melhores que o GP na maioria
dos casos
A Study of Geometric Semantic Genetic Programming with Linear Scaling
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data ScienceMachine Learning (ML) is a scientific discipline that endeavors to enable computers
to learn without the need for explicit programming. Evolutionary Algorithms (EAs),
a subset of ML algorithms, mimic Darwin’s Theory of Evolution by using natural
selection mechanisms (i.e., survival of the fittest) to evolve a group of individuals
(i.e., possible solutions to a given problem). Genetic Programming (GP) is the most
recent type of EA and it evolves computer programs (i.e., individuals) to map a set of
input data into known expected outputs. Geometric Semantic Genetic Programming
(GSGP) extends this concept by allowing individuals to evolve and vary in the semantic
space, where the output vectors are located, rather than being constrained by syntaxbased
structures. Linear Scaling (LS) is a method that was introduced to facilitate the
task of GP of searching for the best function matching a set of known data. GSGP
and LS have both, independently, shown the ability to outperform standard GP for
symbolic regression. GSGP uses Geometric Semantic Operators (GSOs), different
from the standard ones, without altering the fitness, while LS modifies the fitness
without altering the genetic operators. To the best of our knowledge, there has been
no prior utilization of the combined methodology of GSGP and LS for classification
problems. Furthermore, despite the fact that they have been used together in one
practical regression application, a methodological evaluation of the advantages and
disadvantages of integrating these methods for regression or classification problems
has never been performed. In this dissertation, a study of a system that integrates both
GSGP and LS (GSGP-LS) is presented. The performance of the proposed method, GSGPLS,
was tested on six hand-tailored regression benchmarks, nine real-life regression
problems and three real-life classification problems. The obtained results indicate that
GSGP-LS outperforms GSGP in the majority of the cases, confirming the expected
benefit of this integration. However, for some particularly hard regression datasets,
GSGP-LS overfits training data, being outperformed by GSGP on unseen data. This
contradicts the idea that LS is always beneficial for GP, warning the practitioners about
its risk of overfitting in some specific cases.A Aprendizagem Automática (AA) é uma disciplina científica que se esforça por
permitir que os computadores aprendam sem a necessidade de programação explícita.
Algoritmos Evolutivos (AE),um subconjunto de algoritmos de ML, mimetizam a Teoria
da Evolução de Darwin, usando a seleção natural e mecanismos de "sobrevivência dos
mais aptos"para evoluir um grupo de indivíduos (ou seja, possíveis soluções para
um problema dado). A Programação Genética (PG) é um processo algorítmico que
evolui programas de computador (ou indivíduos) para ligar características de entrada e
saída. A Programação Genética em Geometria Semântica (PGGS) estende esse conceito
permitindo que os indivíduos evoluam e variem no espaço semântico, onde os vetores
de saída estão localizados, em vez de serem limitados por estruturas baseadas em
sintaxe. A Escala Linear (EL) é um método introduzido para facilitar a tarefa da PG de
procurar a melhor função que corresponda a um conjunto de dados conhecidos. Tanto
a PGGS quanto a EL demonstraram, independentemente, a capacidade de superar a
PG padrão para regressão simbólica. A PGGS usa Operadores Semânticos Geométricos
(OSGs), diferentes dos padrões, sem alterar o fitness, enquanto a EL modifica o fitness
sem alterar os operadores genéticos. Até onde sabemos, não houve utilização prévia
da metodologia combinada de PGGS e EL para problemas de classificação. Além disso,
apesar de terem sido usados juntos em uma aplicação prática de regressão, nunca foi
realizada uma avaliação metodológica das vantagens e desvantagens da integração
desses métodos para problemas de regressão ou classificação. Nesta dissertação, é
apresentado um estudo de um sistema que integra tanto a PGGS quanto a EL (PGGSEL).
O desempenho do método proposto, PGGS-EL, foi testado em seis benchmarks de
regressão personalizados, nove problemas de regressão da vida real e três problemas
de classificação da vida real. Os resultados obtidos indicam que o PGGS-EL supera
o PGGS na maioria dos casos, confirmando o benefício esperado desta integração.
No entanto, para alguns conjuntos de dados de regressão particularmente difíceis, o
PGGS-EL faz overfit aos dados de treino, obtendo piores resultados em comparação com
PGGS em dados não vistos. Isso contradiz a ideia de que a EL é sempre benéfica para
a PG, alertando os praticantes sobre o risco de overfitting em alguns casos específicos