527 research outputs found
Toward Optimal Run Racing: Application to Deep Learning Calibration
This paper aims at one-shot learning of deep neural nets, where a highly
parallel setting is considered to address the algorithm calibration problem -
selecting the best neural architecture and learning hyper-parameter values
depending on the dataset at hand. The notoriously expensive calibration problem
is optimally reduced by detecting and early stopping non-optimal runs. The
theoretical contribution regards the optimality guarantees within the multiple
hypothesis testing framework. Experimentations on the Cifar10, PTB and Wiki
benchmarks demonstrate the relevance of the approach with a principled and
consistent improvement on the state of the art with no extra hyper-parameter
Automated machine learning in practice : state of the art and recent results
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.A main driver behind the digitization of industry and society is the belief that data-driven model building and decision making can contribute to higher degrees of automation and more informed decisions. Building such models from data often involves the application of some form of machine learning. Thus, there is an ever growing demand in work force with the necessary skill set to do so. This demand has given rise to a new research topic concerned with fitting machine learning models fully automatically – AutoML. This paper gives an overview of the state of the art in AutoML with a focus on practical applicability in a business context, and provides recent benchmark results of the most important AutoML algorithms
Hyperparameter optimization in deep multi-target prediction
As a result of the ever increasing complexity of configuring and fine-tuning
machine learning models, the field of automated machine learning (AutoML) has
emerged over the past decade. However, software implementations like Auto-WEKA
and Auto-sklearn typically focus on classical machine learning (ML) tasks such
as classification and regression. Our work can be seen as the first attempt at
offering a single AutoML framework for most problem settings that fall under
the umbrella of multi-target prediction, which includes popular ML settings
such as multi-label classification, multivariate regression, multi-task
learning, dyadic prediction, matrix completion, and zero-shot learning.
Automated problem selection and model configuration are achieved by extending
DeepMTP, a general deep learning framework for MTP problem settings, with
popular hyperparameter optimization (HPO) methods. Our extensive benchmarking
across different datasets and MTP problem settings identifies cases where
specific HPO methods outperform others.Comment: 17 pages, 4 figures, 1 tabl
Supervising the Multi-Fidelity Race of Hyperparameter Configurations
Multi-fidelity (gray-box) hyperparameter optimization techniques (HPO) have
recently emerged as a promising direction for tuning Deep Learning methods.
However, existing methods suffer from a sub-optimal allocation of the HPO
budget to the hyperparameter configurations. In this work, we introduce DyHPO,
a Bayesian Optimization method that learns to decide which hyperparameter
configuration to train further in a dynamic race among all feasible
configurations. We propose a new deep kernel for Gaussian Processes that embeds
the learning curve dynamics, and an acquisition function that incorporates
multi-budget information. We demonstrate the significant superiority of DyHPO
against state-of-the-art hyperparameter optimization methods through
large-scale experiments comprising 50 datasets (Tabular, Image, NLP) and
diverse architectures (MLP, CNN/NAS, RNN).Comment: Accepted at NeurIPS 202
- …