1,847 research outputs found
OpenML Benchmarking Suites
Machine learning research depends on objectively interpretable, comparable,
and reproducible algorithm benchmarks. Therefore, we advocate the use of
curated, comprehensive suites of machine learning tasks to standardize the
setup, execution, and reporting of benchmarks. We enable this through software
tools that help to create and leverage these benchmarking suites. These are
seamlessly integrated into the OpenML platform, and accessible through
interfaces in Python, Java, and R. OpenML benchmarking suites are (a) easy to
use through standardized data formats, APIs, and client libraries; (b)
machine-readable, with extensive meta-information on the included datasets; and
(c) allow benchmarks to be shared and reused in future studies. We also present
a first, carefully curated and practical benchmarking suite for classification:
the OpenML Curated Classification benchmarking suite 2018 (OpenML-CC18)
- …