994 research outputs found
The use of genetic programming for detecting the incorrect predictions of classification models
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsCompanies around the world use Advanced Analytics to support their decision
making process. Traditionally they used Statistics and Business Intelligence
for that, but as the technology is advancing, the more complex models are
gaining popularity. The main reason for an increasing interest in Machine
Learning and Deep Learning models is the fact that they reach a high prediction
accuracy. On the second hand with good performance, comes an increasing
complexity of the programs. Therefore the new area of Predictors was introduced,
it is called Explainable AI. The idea is to create models that can be
understood by business users or models to explain other predictions. Therefore
we propose the study in which we create a separate model, that will serve as
a very er for the machine learning models predictions. This work falls into
area of Post-processing of models outputs. For this purpose we select Genetic
Programming, that was proven to be successful in various applications. In
the scope of this research we investigate if GP can evaluate the prediction of
other models. This area of applications was not explored yet, therefore in the
study we explore the possibility of evolving an individual for another model
validation. We focus on classi cation problems and select 4 machine learning
models: logistic regression, decision tree, random forest, perceptron and
3 di erent datasets. This set up is used for assuring that during the research
we conclude that the presented idea is universal for di erent problems. The
performance of 12 Genetic Programming experiments indicates that in some
cases it is possible to create a successful model for errors prediction. During the
study we discovered that the performance of GP programs is mostly connected
to the dataset on the experiment is conducted. The type of predictive models
does not in
uence the performance of GP. Although we managed to create
good classi ers of errors, during the evolution process we faced the problem
of over tting. That is common in problems with imbalanced datasets. The
results of the study con rms that GP can be used for the new type of problems
and successfully predict errors of Machine Learning Models
Deep Semantic Learning Machine: A Convolutional Network Construction Algorithm
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsThe Semantic Learning Machine (SLM), an algorithm that evolves the topology of feed-forward
neural networks (NN), has shown remarkable results in generalization and computing time. It
has the benefits of searching the space of different NN architectures under a unimodal fitness
landscape in any supervised learning problem. Recent research used the SLM at the end of a
Convolutional Neural Network (CNN) instead of fully connected layers outperforming stateof-
the-art CNNs. It was proposed to extend the SLM to explore the possibility of optimizing
the convolution layers - evolving the full CNN topology. This thesis introduces an operator to
optimize the convolution layers, extending the SLM to the Deep Semantic Learning Machine.
Initial results, computed using the mnist dataset, show that the algorithm does work but are of
limited interpretability. Real-life practicability remains to be improved due to high memory and
computational requirements.Semantic Learning Machine (SLM), um algoritmo que evolui a topologia de redes neurais
feed-forward (NN), tem mostrado resultados notáveis em generalização e tempo de computação.
Tem benefícios de pesquisar o espaço de diferentes arquiteturas NN sob um cenário de aptidão
unimodal em qualquer problema de aprendizagem supervisionada. Investigação recente recorre
ao uso deSLMno final de uma redes neurais convolucional (CNN) em vez de camadas totalmente
conectadas, superando CNNs de última geração. Foi proposto estender o SLM para explorar a
possibilidade de otimizar as camadas de convolução - evoluindo a totalmente a topologia CNN. A
presente tese apresenta um operador para otimizar as camadas de convolução, estendendo o SLM
para a Deep Semantic Learning Machine. Os resultados iniciais, calculados usando o conjunto
de dados mnist, mostram que o algoritmo funciona, mas revelam uma interpretabilidade limitada.
A aplicabilidade em cenários reais precisa ainda de melhorias devido aos altos requisitos de
memória e computação
The detection of globular clusters in galaxies as a data mining problem
We present an application of self-adaptive supervised learning classifiers
derived from the Machine Learning paradigm, to the identification of candidate
Globular Clusters in deep, wide-field, single band HST images. Several methods
provided by the DAME (Data Mining & Exploration) web application, were tested
and compared on the NGC1399 HST data described in Paolillo 2011. The best
results were obtained using a Multi Layer Perceptron with Quasi Newton learning
rule which achieved a classification accuracy of 98.3%, with a completeness of
97.8% and 1.6% of contamination. An extensive set of experiments revealed that
the use of accurate structural parameters (effective radius, central surface
brightness) does improve the final result, but only by 5%. It is also shown
that the method is capable to retrieve also extreme sources (for instance, very
extended objects) which are missed by more traditional approaches.Comment: Accepted 2011 December 12; Received 2011 November 28; in original
form 2011 October 1
Relatedness Measures to Aid the Transfer of Building Blocks among Multiple Tasks
Multitask Learning is a learning paradigm that deals with multiple different
tasks in parallel and transfers knowledge among them. XOF, a Learning
Classifier System using tree-based programs to encode building blocks
(meta-features), constructs and collects features with rich discriminative
information for classification tasks in an observed list. This paper seeks to
facilitate the automation of feature transferring in between tasks by utilising
the observed list. We hypothesise that the best discriminative features of a
classification task carry its characteristics. Therefore, the relatedness
between any two tasks can be estimated by comparing their most appropriate
patterns. We propose a multiple-XOF system, called mXOF, that can dynamically
adapt feature transfer among XOFs. This system utilises the observed list to
estimate the task relatedness. This method enables the automation of
transferring features. In terms of knowledge discovery, the resemblance
estimation provides insightful relations among multiple data. We experimented
mXOF on various scenarios, e.g. representative Hierarchical Boolean problems,
classification of distinct classes in the UCI Zoo dataset, and unrelated tasks,
to validate its abilities of automatic knowledge-transfer and estimating task
relatedness. Results show that mXOF can estimate the relatedness reasonably
between multiple tasks to aid the learning performance with the dynamic feature
transferring.Comment: accepted by The Genetic and Evolutionary Computation Conference
(GECCO 2020
Search-based energy optimization of some ubiquitous algorithms
Reducing computational energy consumption is of growing importance, particularly at the extremes (i.e. mobile devices and datacentres). Despite the ubiquity of the JavaTM Virtual Machine (JVM), very little work has been done to apply Search Based Software Engineering (SBSE) to minimize the energy consumption of programs that run on it. We describe OPACITOR , a tool for measuring the energy consumption of JVM programs using a bytecode level model of energy cost. This has several advantages over time-based energy approximations or hardware measurements. It is: deterministic. unaffected by the rest of the computational environment. able to detect small changes in execution profile, making it highly amenable to metaheuristic search which requires locality of representation. We show how generic SBSE approaches coupled with OPACITOR achieve substantial energy savings for three widely-used software components. Multi-Layer Perceptron implementations minimis- ing both energy and error were found, and energy reductions of up to 70% and 39.85% were obtained over the original code for Quicksort and Object-Oriented container classes respectively. These highlight three important considerations for automatically reducing computational energy: tuning software to particular distributions of data; trading off energy use against functional properties; and handling internal dependencies which can exist within software that render simple sweeps over program variants sub-optimal. Against these, global search greatly simplifies the developer’s job, freeing development time for other tasks
Towards The Deep Semantic Learning Machine Neuroevolution Algorithm: An exploration on the CIFAR-10 problem task
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsSelecting the topology and parameters of Convolutional Neural Network (CNN) for a given supervised machine learning task is a non-trivial problem. The Deep Semantic Learning Machine (Deep-SLM) deals with this problem by automatically constructing CNNs without the use of the Backpropagation algorithm. The Deep-SLM is a novel neuroevolution technique and functions as stochastic semantic hill-climbing algorithm searching over the space of CNN topologies and parameters. The geometric semantic properties of the Deep-SLM induce a unimodel error space and eliminate the existence of local optimal solutions. This makes the Deep-SLM potentially favorable in terms of search efficiency and effectiveness.
This thesis provides an exploration of a variant of the Deep-SLM algorithm on the CIFAR-10 problem task, and a validation of its proof of concept. This specific variant only forms mutation node ! mutation node connections in the non-convolutional part of the constructed CNNs. Furthermore, a comparative study between the Deep-SLM and the Semantic Learning Machine (SLM) algorithms was conducted. It was observed that sparse connections can be an effective way to prevent overfitting. Additionally, it was shown that a single 2D convolution layer initialized with random weights does not result in well-generalizing features for the Deep-SLM directly, but, in combination with a 2D max-pooling down sampling layer, effective improvements in performance and generalization of the Deep-SLM could be achieved. These results constitute to the hypothesis that convolution and pooling layers can improve performance and generalization of the Deep-SLM, unless the components are properly optimized.Selecionar a topologia e os parâmetros da Rede Neural Convolucional (CNN) para uma tarefa de aprendizado automático supervisionada não é um problema trivial. A Deep Semantic Learning Machine (Deep-SLM) lida com este problema construindo automaticamente CNNs sem recorrer ao uso do algoritmo de Retro-propagação. A Deep-SLM é uma nova técnica de neuroevolução que funciona enquanto um algoritmo de
escalada estocástico semântico na pesquisa de topologias e de parâmetros CNN. As propriedades geométrico-semânticas da Deep-SLM induzem um unimodel error space que elimina a existência de soluções ótimas locais, favorecendo, potencialmente, a Deep-SLM em termos de eficiência e eficácia.
Esta tese providencia uma exploração de uma variante do algoritmo da Deep-SLM no problemo de CIFAR-10, assim como uma validação do seu conceito de prova. Esta variante específica apenas forma conexões nó de mutação!nó de mutação na parte non convolucional da CNN construída. Mais ainda, foi conduzido um estudo comparativo entre a Deep-SLM e o algoritmo da Semantic Learning Machine (SLM). Tendo sido observado
que as conexões esparsas poderão tratar-se de uma forma eficiente de prevenir o overfitting. Adicionalmente, mostrou-se que uma singular camada de convolução 2D, iniciada com valores aleatórios, não resulta, directamente, em características generalizadas para a Deep-SLM, mas, em combinação com uma camada de 2D max-pooling, melhorias efectivas na performance e na generalização da Deep-SLM poderão ser concretizadas.
Estes resultados constituem, assim, a hipótese de que as camadas de convolução e pooling poderão melhorar a performance e a generalização da Deep-SLM, a não ser que os componentes sejam adequadamente otimizados
Machine Learning for Fluid Mechanics
The field of fluid mechanics is rapidly advancing, driven by unprecedented
volumes of data from field measurements, experiments and large-scale
simulations at multiple spatiotemporal scales. Machine learning offers a wealth
of techniques to extract information from data that could be translated into
knowledge about the underlying fluid mechanics. Moreover, machine learning
algorithms can augment domain knowledge and automate tasks related to flow
control and optimization. This article presents an overview of past history,
current developments, and emerging opportunities of machine learning for fluid
mechanics. It outlines fundamental machine learning methodologies and discusses
their uses for understanding, modeling, optimizing, and controlling fluid
flows. The strengths and limitations of these methods are addressed from the
perspective of scientific inquiry that considers data as an inherent part of
modeling, experimentation, and simulation. Machine learning provides a powerful
information processing framework that can enrich, and possibly even transform,
current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202
Neuroevolution in Games: State of the Art and Open Challenges
This paper surveys research on applying neuroevolution (NE) to games. In
neuroevolution, artificial neural networks are trained through evolutionary
algorithms, taking inspiration from the way biological brains evolved. We
analyse the application of NE in games along five different axes, which are the
role NE is chosen to play in a game, the different types of neural networks
used, the way these networks are evolved, how the fitness is determined and
what type of input the network receives. The article also highlights important
open research challenges in the field.Comment: - Added more references - Corrected typos - Added an overview table
(Table 1
- …