A self-parametrization framework for meta-heuristics

Even while the scientific community has shown great interest in the analysis of meta-heuristics, the analysis of their parameterization has received little attention. It is the parameterization that will adapt a meta-heuristic to a problem, but it is still performed, mostly, empirically. There are multiple parameterization techniques; however, they are time-consuming, requiring considerable computational effort and they do not take advantage of the meta-heuristics that they parameterize. In order to approach the parameterization of meta-heuristics, in this paper, a self-parameterization framework is proposed. It will automatize the parameterization as an optimization problem, precluding the user from spending too much time on parameterization. The model will automate the parameterization through two meta-heuristics: A meta-heuristic of the solution space and one of the parameter space. To analyze the performance of the framework, a self-parameterization prototype was implemented. The prototype was compared and analyzed in a SP (scheduling problem) and in the TSP (traveling salesman problem). In the SP, the prototype found better solutions than those of the manually parameterized meta-heuristics, although the differences were not statistically significant. In the TSP, the self-parameterization prototype was more effective than the manually parameterized meta-heuristics, this time with statistically significant differences.This work was supported by national funds through the FCT - Fundação para a Ciência e Tecnologia through the R&D Units Project Scopes: UIDB/00319/2020, and EXPL/EME-SIS/1224/2021

Towards portfolios of streamlined constraint models : a case study with the balanced academic curriculum problem

Funding: This work is supported by EPSRC grant EP/P015638/1 and used the Cirrus UK National Tier-2 HPC Service at EPCC (http://www.cirrus.ac.uk) funded by the University of Edinburgh and EPSRC (EP/P020267/1). Nguyen Dang is a Leverhulme Early Career Fellow.Augmenting a base constraint model with additional constraints can strengthen the inferences made by a solver and therefore reduce search effort. We focus on the automatic addition of streamliner constraints, derived from the types present in an abstract Essence specification of a problem class of interest, which trade completeness for potentially very significant reduction in search. The refinement of streamlined Essence specifications into constraint models suitable for input to constraint solvers gives rise to a large number of modelling choices in addition to those required for the base Essence specification. Previous automated streamlining approaches have been limited in evaluating only a single default model for each streamlined specification. In this paper we explore the effect of model selection in the context of streamlined specifications. We propose a new best-first search method that generates a portfolio of Pareto Optimal streamliner-model combinations by evaluating for each streamliner a portfolio of models to search and explore the variability in performance and find the optimal model. Various forms of racing are utilised to constrain the computational cost of training.Publisher PD

Efficient Benchmarking of Algorithm Configuration Procedures via Model-Based Surrogates

The optimization of algorithm (hyper-)parameters is crucial for achieving peak performance across a wide range of domains, ranging from deep neural networks to solvers for hard combinatorial problems. The resulting algorithm configuration (AC) problem has attracted much attention from the machine learning community. However, the proper evaluation of new AC procedures is hindered by two key hurdles. First, AC benchmarks are hard to set up. Second and even more significantly, they are computationally expensive: a single run of an AC procedure involves many costly runs of the target algorithm whose performance is to be optimized in a given AC benchmark scenario. One common workaround is to optimize cheap-to-evaluate artificial benchmark functions (e.g., Branin) instead of actual algorithms; however, these have different properties than realistic AC problems. Here, we propose an alternative benchmarking approach that is similarly cheap to evaluate but much closer to the original AC problem: replacing expensive benchmarks by surrogate benchmarks constructed from AC benchmarks. These surrogate benchmarks approximate the response surface corresponding to true target algorithm performance using a regression model, and the original and surrogate benchmark share the same (hyper-)parameter space. In our experiments, we construct and evaluate surrogate benchmarks for hyperparameter optimization as well as for AC problems that involve performance optimization of solvers for hard combinatorial problems, drawing training data from the runs of existing AC procedures. We show that our surrogate benchmarks capture overall important characteristics of the AC scenarios, such as high- and low-performing regions, from which they were derived, while being much easier to use and orders of magnitude cheaper to evaluate

An Empirical Study of Off-line Configuration and On-line Adaptation in Operator Selection

Automating the process of finding good parameter settings is important in the design of high-performing algorithms. These automatic processes can generally be categorized into off-line and on-line methods. Off-line configuration consists in learning and selecting the best setting in a training phase, and usually fixes it while solving an instance. On-line adaptation methods on the contrary vary the parameter setting adaptively during each algorithm run. In this work, we provide an empirical study of both approaches on the operator selection problem, explore the possibility of varying parameter value by a non-adaptive distribution tuned off-line, and incorporate the off-line with on-line approaches. In particular, using an off-line tuned distribution to vary parameter values at runtime appears to be a promising idea for automatic configuration. © 2014 Springer International Publishing.SCOPUS: cp.kinfo:eu-repo/semantics/publishe