75 research outputs found
Evolution of Scikit-Learn Pipelines with Dynamic Structured Grammatical Evolution
The deployment of Machine Learning (ML) models is a difficult and
time-consuming job that comprises a series of sequential and correlated tasks
that go from the data pre-processing, and the design and extraction of
features, to the choice of the ML algorithm and its parameterisation. The task
is even more challenging considering that the design of features is in many
cases problem specific, and thus requires domain-expertise. To overcome these
limitations Automated Machine Learning (AutoML) methods seek to automate, with
few or no human-intervention, the design of pipelines, i.e., automate the
selection of the sequence of methods that have to be applied to the raw data.
These methods have the potential to enable non-expert users to use ML, and
provide expert users with solutions that they would unlikely consider. In
particular, this paper describes AutoML-DSGE - a novel grammar-based framework
that adapts Dynamic Structured Grammatical Evolution (DSGE) to the evolution of
Scikit-Learn classification pipelines. The experimental results include
comparing AutoML-DSGE to another grammar-based AutoML framework, Resilient
ClassificationPipeline Evolution (RECIPE), and show that the average
performance of the classification pipelines generated by AutoML-DSGE is always
superior to the average performance of RECIPE; the differences are
statistically significant in 3 out of the 10 used datasets.Comment: EvoApps 202
AutoML @ NeurIPS 2018 challenge: Design and Results
Preprint submitted to NeurIPS2018 Volume of Springer Series on Challenges in Machine LearningInternational audienceWe organized a competition on Autonomous Lifelong Machine Learning with Drift that was part of the competition program of NeurIPS 2018. This data driven competition asked participants to develop computer programs capable of solving supervised learning problems where the i.i.d. assumption did not hold. Large data sets were arranged in a lifelong learning and evaluation scenario and CodaLab was used as the challenge platform. The challenge attracted more than 300 participants in its two month duration. This chapter describes the design of the challenge and summarizes its main results
Auto-Sklearn 2.0: The Next Generation
Automated Machine Learning, which supports practitioners and researchers with
the tedious task of manually designing machine learning pipelines, has recently
achieved substantial success. In this paper we introduce new Automated Machine
Learning (AutoML) techniques motivated by our winning submission to the second
ChaLearn AutoML challenge, PoSH Auto-sklearn. For this, we extend Auto-sklearn
with a new, simpler meta-learning technique, improve its way of handling
iterative algorithms and enhance it with a successful bandit strategy for
budget allocation. Furthermore, we go one step further and study the design
space of AutoML itself and propose a solution towards truly hand-free AutoML.
Together, these changes give rise to the next generation of our AutoML system,
Auto-sklearn (2.0). We verify the improvement by these additions in a large
experimental study on 39 AutoML benchmark datasets and conclude the paper by
comparing to Auto-sklearn (1.0), reducing the regret by up to a factor of five
AutoML Workshop
Abstract The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranged across different types of complexity. Over six rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this competition contributes to the development of fully automated environments by challenging practitioners to solve problems under specific constraints and sharing their approaches; the platform will remain available for post-challenge submissions at http://codalab.org/AutoML
- âŠ