101,958 research outputs found
BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning
An ever increasing number of configuration parameters are provided to system
users. But many users have used one configuration setting across different
workloads, leaving untapped the performance potential of systems. A good
configuration setting can greatly improve the performance of a deployed system
under certain workloads. But with tens or hundreds of parameters, it becomes a
highly costly task to decide which configuration setting leads to the best
performance. While such task requires the strong expertise in both the system
and the application, users commonly lack such expertise.
To help users tap the performance potential of systems, we present
BestConfig, a system for automatically finding a best configuration setting
within a resource limit for a deployed system under a given application
workload. BestConfig is designed with an extensible architecture to automate
the configuration tuning for general systems. To tune system configurations
within a resource limit, we propose the divide-and-diverge sampling method and
the recursive bound-and-search algorithm. BestConfig can improve the throughput
of Tomcat by 75%, that of Cassandra by 63%, that of MySQL by 430%, and reduce
the running time of Hive join job by about 50% and that of Spark join job by
about 80%, solely by configuration adjustment
Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates
The optimization of algorithm (hyper-)parameters is crucial for achieving
peak performance across a wide range of domains, ranging from deep neural
networks to solvers for hard combinatorial problems. The resulting algorithm
configuration (AC) problem has attracted much attention from the machine
learning community. However, the proper evaluation of new AC procedures is
hindered by two key hurdles. First, AC benchmarks are hard to set up. Second
and even more significantly, they are computationally expensive: a single run
of an AC procedure involves many costly runs of the target algorithm whose
performance is to be optimized in a given AC benchmark scenario. One common
workaround is to optimize cheap-to-evaluate artificial benchmark functions
(e.g., Branin) instead of actual algorithms; however, these have different
properties than realistic AC problems. Here, we propose an alternative
benchmarking approach that is similarly cheap to evaluate but much closer to
the original AC problem: replacing expensive benchmarks by surrogate benchmarks
constructed from AC benchmarks. These surrogate benchmarks approximate the
response surface corresponding to true target algorithm performance using a
regression model, and the original and surrogate benchmark share the same
(hyper-)parameter space. In our experiments, we construct and evaluate
surrogate benchmarks for hyperparameter optimization as well as for AC problems
that involve performance optimization of solvers for hard combinatorial
problems, drawing training data from the runs of existing AC procedures. We
show that our surrogate benchmarks capture overall important characteristics of
the AC scenarios, such as high- and low-performing regions, from which they
were derived, while being much easier to use and orders of magnitude cheaper to
evaluate
ACTS in Need: Automatic Configuration Tuning with Scalability Guarantees
To support the variety of Big Data use cases, many Big Data related systems
expose a large number of user-specifiable configuration parameters. Highlighted
in our experiments, a MySQL deployment with well-tuned configuration parameters
achieves a peak throughput as 12 times much as one with the default setting.
However, finding the best setting for the tens or hundreds of configuration
parameters is mission impossible for ordinary users. Worse still, many Big Data
applications require the support of multiple systems co-deployed in the same
cluster. As these co-deployed systems can interact to affect the overall
performance, they must be tuned together. Automatic configuration tuning with
scalability guarantees (ACTS) is in need to help system users. Solutions to
ACTS must scale to various systems, workloads, deployments, parameters and
resource limits. Proposing and implementing an ACTS solution, we demonstrate
that ACTS can benefit users not only in improving system performance and
resource utilization, but also in saving costs and enabling fairer
benchmarking
MaLeS: A Framework for Automatic Tuning of Automated Theorem Provers
MaLeS is an automatic tuning framework for automated theorem provers. It
provides solutions for both the strategy finding as well as the strategy
scheduling problem. This paper describes the tool and the methods used in it,
and evaluates its performance on three automated theorem provers: E, LEO-II and
Satallax. An evaluation on a subset of the TPTP library problems shows that on
average a MaLeS-tuned prover solves 8.67% more problems than the prover with
its default settings
- …