19 research outputs found
A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost
This paper presents a benchmark of supervised Automated Machine Learning (AutoML) tools. Firstly, we an- alyze the characteristics of eight recent open-source AutoML tools (Auto-Keras, Auto-PyTorch, Auto-Sklearn, AutoGluon, H2O AutoML, rminer, TPOT and TransmogrifAI) and describe twelve popular OpenML datasets that were used in the benchmark (divided into regression, binary and multi-class classification tasks). Then, we perform a comparison study with hundreds of computational experiments based on three scenarios: General Machine Learning (GML), Deep Learning (DL) and XGBoost (XGB). To select the best tool, we used a lexicographic approach, considering first the average prediction score for each task and then the computational effort. The best predictive results were achieved for GML, which were further compared with the best OpenML public results. Overall, the best GML AutoML tools obtained competitive results, outperforming the best OpenML models in five datasets. These results confirm the potential of the general-purpose AutoML tools to fully automate the Machine Learning (ML) algorithm selection and tuning.Opti-Edge: 5G
Digital Services Optimization at the Edge, Individual Project,
NUP: POCI-01-0247-FEDER-045220, co-funded by the Incentive System for Research and Technological Development,
from the Thematic Operational Program Competitiveness of
the national framework program - Portugal202
Accelerating Evolution Through Gene Masking and Distributed Search
In building practical applications of evolutionary computation (EC), two
optimizations are essential. First, the parameters of the search method need to
be tuned to the domain in order to balance exploration and exploitation
effectively. Second, the search method needs to be distributed to take
advantage of parallel computing resources. This paper presents BLADE (BLAnket
Distributed Evolution) as an approach to achieving both goals simultaneously.
BLADE uses blankets (i.e., masks on the genetic representation) to tune the
evolutionary operators during the search, and implements the search through
hub-and-spoke distribution. In the paper, (1) the blanket method is formalized
for the (1 + 1)EA case as a Markov chain process. Its effectiveness is then
demonstrated by analyzing dominant and subdominant eigenvalues of stochastic
matrices, suggesting a generalizable theory; (2) the fitness-level theory is
used to analyze the distribution method; and (3) these insights are verified
experimentally on three benchmark problems, showing that both blankets and
distribution lead to accelerated evolution. Moreover, a surprising synergy
emerges between them: When combined with distribution, the blanket approach
achieves more than -fold speedup with clients in some cases. The work
thus highlights the importance and potential of optimizing evolutionary
computation in practical applications
Automatic Design of Artificial Neural Networks for Gamma-Ray Detection
The goal of this work is to investigate the possibility of improving current
gamma/hadron discrimination based on their shower patterns recorded on the
ground. To this end we propose the use of Convolutional Neural Networks (CNNs)
for their ability to distinguish patterns based on automatically designed
features. In order to promote the creation of CNNs that properly uncover the
hidden patterns in the data, and at same time avoid the burden of hand-crafting
the topology and learning hyper-parameters we resort to NeuroEvolution; in
particular we use Fast-DENSER++, a variant of Deep Evolutionary Network
Structured Representation. The results show that the best CNN generated by
Fast-DENSER++ improves by a factor of 2 when compared with the results reported
by classic statistical approaches. Additionally, we experiment ensembling the
10 best generated CNNs, one from each of the evolutionary runs; the ensemble
leads to an improvement by a factor of 2.3. These results show that it is
possible to improve the gamma/hadron discrimination based on CNNs that are
automatically generated and are trained with instances of the ground impact
patterns.info:eu-repo/semantics/publishedVersio
Asynchronous Evolution of Deep Neural Network Architectures
Many evolutionary algorithms (EAs) take advantage of parallel evaluation of
candidates. However, if evaluation times vary significantly, many worker nodes
(i.e.,\ compute clients) are idle much of the time, waiting for the next
generation to be created. Evolutionary neural architecture search (ENAS), a
class of EAs that optimizes the architecture and hyperparameters of deep neural
networks, is particularly vulnerable to this issue. This paper proposes a
generic asynchronous evaluation strategy (AES) that is then adapted to work
with ENAS. AES increases throughput by maintaining a queue of upto
individuals ready to be sent to the workers for evaluation and proceeding to
the next generation as soon as individuals have been evaluated by the
workers. A suitable value for is determined experimentally, balancing
diversity and efficiency. To showcase the generality and power of AES, it was
first evaluated in 11-bit multiplexer design (a single-population verifiable
discovery task) and then scaled up to ENAS for image captioning (a
multi-population open-ended-optimization task). In both problems, a multifold
performance improvement was observed, suggesting that AES is a promising method
for parallelizing the evolution of complex systems with long and variable
evaluation times, such as those in ENAS
Efficiently Coevolving Deep Neural Networks and Data Augmentations
Designing large deep learning neural networks by
hand requires tuning large sets of method parameters, requiring
trial and error testing and domain specific knowledge. Neuroevolution methods such as CoDeepNeat (CDN), based on Neuroevolution of Augmenting Topologies (NEAT), apply evolutionary
algorithms to automate deep neural network parameter optimisation. This paper presents and demonstrates various novel
beneficial extensions to the CDN method, including new genotypic
speciation mechanisms, special mappings in deep neural network
encodings, as well as evolving Data Augmentation schemes.
Results indicate that these CDN method variants yield significant
task-performance benefits over the benchmark CDN method
when evaluated on a popular public image recognition data-set