Constructing constrained-version of magic squares using selection hyper-heuristics

A square matrix of distinct numbers in which every row, column and both diagonals have the same total is referred to as a magic square. Constructing a magic square of a given order is considered a difficult computational problem, particularly when additional constraints are imposed. Hyper-heuristics are emerging high-level search methodologies that explore the space of heuristics for solving a given problem. In this study, we present a range of effective selection hyper-heuristics mixing perturbative low-level heuristics for constructing the constrained version of magic squares. The results show that selection hyper-heuristics, even the non-learning ones deliver an outstanding performance, beating the best-known heuristic solution on average

Markov Chain Selection Hyper-heuristic for the Optimisation of Constrained Magic Squares

UKCI 2015: UK Workshop on Computational Intelligence, University of Exeter, UK, 7-9 September 2015A square matrix of size n × n, containing each of the numbers (1, . . . , n2) in which every row, column and both diagonals has the same total is referred to as a magic square. The problem can be formulated as an optimisation problem where the task is to minimise the deviation from the magic square constraints and is tackled here by using hyper-heuristics. Hyper-heuristics have recently attracted the attention of the artificial intelligence, operations research, engineering and computer science communities where the aim is to design and develop high level strategies as general solvers which are applicable to a range of different problem domains. There are two main types of hyper-heuristics in the literature: methodologies to select and to generate heuristics and both types of approaches search the space of heuristics rather than solutions. In this study, we describe a Markov chain selection hyper-heuristic as an effective solution methodology for optimising constrained magic squares. The empirical results show that the proposed hyper-heuristic is able to outperform the current state-of-the-art method

Multi-stage hyper-heuristics for optimisation problems

There is a growing interest towards self configuring/tuning automated general-purpose reusable heuristic approaches for combinatorial optimisation, such as, hyper-heuristics. Hyper-heuristics are search methodologies which explore the space of heuristics rather than the solutions to solve a broad range of hard computational problems without requiring any expert intervention. There are two common types of hyper-heuristics in the literature: selection and generation methodologies. This work focuses on the former type of hyper-heuristics. Almost all selection hyper-heuristics perform a single point based iterative search over the space of heuristics by selecting and applying a suitable heuristic to the solution in hand at each decision point. Then the newly generated solution is either accepted or rejected using an acceptance method. This improvement process is repeated starting from an initial solution until a set of termination criteria is satisfied. The number of studies on the design of hyper-heuristic methodologies has been rapidly increasing and currently, we already have a variety of approaches, each with their own strengths and weaknesses. It has been observed that different hyper-heuristics perform differently on a given subset of problem instances and more importantly, a hyper-heuristic performs differently as the set of low level heuristics vary. This thesis introduces a general "multi-stage" hyper-heuristic framework enabling the use and exploitation of multiple selection hyper-heuristics at different stages during the search process. The goal is designing an approach utilising multiple hyper-heuristics for a more effective and efficient overall performance when compared to the performance of each constituent selection hyper-heuristic. The level of generality that a hyper-heuristic can achieve has always been of interest to the hyper-heuristic researchers. Hence, a variety of multi-stage hyper-heuristics based on the framework are not only applied to the real-world combinatorial optimisation problems of high school timetabling, multi-mode resource-constrained multi-project scheduling and construction of magic squares, but also tested on the well known hyper-heuristic benchmark of CHeSC 2011. The empirical results show that the multi-stage hyper-heuristics designed based on the proposed framework are still inherently general, easy-to-implement, adaptive and reusable. They can be extremely effective solvers considering their success in the competitions of ITC 2011 and MISTA 2013. Moreover, a particular multi-stage hyper-heuristic outperformed the state-of-the-art selection hyper-heuristic from CHeSC 2011

Multi-stage hyper-heuristics for optimisation problems

There is a growing interest towards self configuring/tuning automated general-purpose reusable heuristic approaches for combinatorial optimisation, such as, hyper-heuristics. Hyper-heuristics are search methodologies which explore the space of heuristics rather than the solutions to solve a broad range of hard computational problems without requiring any expert intervention. There are two common types of hyper-heuristics in the literature: selection and generation methodologies. This work focuses on the former type of hyper-heuristics. Almost all selection hyper-heuristics perform a single point based iterative search over the space of heuristics by selecting and applying a suitable heuristic to the solution in hand at each decision point. Then the newly generated solution is either accepted or rejected using an acceptance method. This improvement process is repeated starting from an initial solution until a set of termination criteria is satisfied. The number of studies on the design of hyper-heuristic methodologies has been rapidly increasing and currently, we already have a variety of approaches, each with their own strengths and weaknesses. It has been observed that different hyper-heuristics perform differently on a given subset of problem instances and more importantly, a hyper-heuristic performs differently as the set of low level heuristics vary. This thesis introduces a general "multi-stage" hyper-heuristic framework enabling the use and exploitation of multiple selection hyper-heuristics at different stages during the search process. The goal is designing an approach utilising multiple hyper-heuristics for a more effective and efficient overall performance when compared to the performance of each constituent selection hyper-heuristic. The level of generality that a hyper-heuristic can achieve has always been of interest to the hyper-heuristic researchers. Hence, a variety of multi-stage hyper-heuristics based on the framework are not only applied to the real-world combinatorial optimisation problems of high school timetabling, multi-mode resource-constrained multi-project scheduling and construction of magic squares, but also tested on the well known hyper-heuristic benchmark of CHeSC 2011. The empirical results show that the multi-stage hyper-heuristics designed based on the proposed framework are still inherently general, easy-to-implement, adaptive and reusable. They can be extremely effective solvers considering their success in the competitions of ITC 2011 and MISTA 2013. Moreover, a particular multi-stage hyper-heuristic outperformed the state-of-the-art selection hyper-heuristic from CHeSC 2011

Improvement and Integration of Counting-Based Search Heuristics in Constraint Programming

Ce mémoire s’intéresse à la programmation par contraintes, un paradigme pour résoudre des problèmes combinatoires. Pour la plupart des problèmes, trouver une solution n’est pas possible si on se limite à des mécanismes d’inférence logique; l’exploration d’un espace des solutions à l’aide d’heuristiques de recherche est nécessaire. Des nombreuses heuristiques existantes, les heuristiques de branchement basées sur le dénombrement seront au centre de ce mémoire. Cette approche repose sur l’utilisation d’algorithmes pour estimer le nombre de solutions des contraintes individuelles d’un problème de satisfaction de contraintes. Notre contribution se résume principalement à l’amélioration de deux algorithmes de dénombrement pour les contraintes alldifferent et spanningTree; ces contraintes peuvent exprimer de nombreux problèmes de satisfaction, et sont par le fait même essentielles à nos heuristiques de branchement. Notre travail fait également l’objet d’une contribution à un solveur de programmation par contraintes open-source. Ainsi, l’ensemble de ce mémoire est motivé par cette considération pratique; nos algorithmes doivent être accessibles et performants. Finalement, nous explorons deux techniques applicables à l’ensemble de nos heuristiques: une technique qui réutilise des calculs précédemment faits dans l’arbre de recherche ainsi qu’une manière d’apprendre de nouvelles heuristiques de branchement pour un problème.=----------ABSTRACT: This thesis concerns constraint programming, a paradigm for solving combinatorial problems. The focus is on the mechanism involved in making hypotheses and exploring the solution space towards satisfying solutions: search heuristics. Of interest to us is a specific family called counting-based search, an approach that uses algorithms to estimate the number of solutions of individual constraints in constraint satisfaction problems to guide search. The improvements of two existing counting algorithms and the integration of counting-based search in a constraint programming solver are the two main contributions of this thesis. The first counting algorithm concerns the alldifferent constraint; the second one, the spanningTree constraint. Both constraints are useful for expressing many constraint satisfaction problems and thus are essential for counting-based search. Practical matters are also central to this work; we integrated counting-based search in an open-source constraint programming solver called Gecode. In doing so, we bring this family of search heuristics to a wider audience; everything in this thesis is built upon this contribution. Lastly, we also look at more general improvements to counting-based search with a method for trading computation time for accuracy, and a method for learning new counting-based search heuristics from past experiments

Enhancing the Performance of Search Heuristics. Variable Fitness Functions and other Methods to Enhance Heuristics for Dynamic Workforce Scheduling.

Scheduling large real world problems is a complex process and finding high quality solutions is not a trivial task. In cooperation with Trimble MRM Ltd., who provide scheduling solutions for many large companies, a problem is identified and modelled. It is a general model which encapsulates several important scheduling, routing and resource allocation problems in literature. Many of the state-of-the-art heuristics for solve scheduling problems and indeed other problems require specialised heuristics tailored for the problem they are to solve. While these provide good solutions a lot of expert time is needed to study the problem, and implement solutions. This research investigates methods to enhance existing search based methods. We study hyperheuristic techniques as a general search based heuristic. Hyperheuristics raise the generality of the solution method by using a set of tools (low level heuristics) to work on the solution. These tools are problem specific and usually make small changes to the problem. It is the task of the hyperheuristic to determine which tool to use and when. Low level heuristics using exact/heuristic hybrid method are used in this thesis along with a new Tabu based hyperheuristic which decreases the amount of CPU time required to produce good quality solutions. We also develop and investigate the Variable Fitness Function approach, which provides a new way of enhancing most search-based heuristics in terms of solution quality. If a fitness function is pushing hard in a certain direction, a heuristic may ultimately fail because it cannot escape local minima. The Variable Fitness Function allows the fitness function to change over the search and use objective measures not used in the fitness calculation. The Variable Fitness Function and its ability to generalise are extensively tested in this thesis. The two aims of the thesis are achieved and the methods are analysed in depth. General conclusions and areas of future work are also identified

evtree: Evolutionary Learning of Globally Optimal Classification and Regression Trees in R

Commonly used classification and regression tree methods like the CART algorithm are recursive partitioning methods that build the model in a forward stepwise search. Although this approach is known to be an efficient heuristic, the results of recursive tree methods are only locally optimal, as splits are chosen to maximize homogeneity at the next step only. An alternative way to search over the parameter space of trees is to use global optimization methods like evolutionary algorithms. This paper describes the evtree package, which implements an evolutionary algorithm for learning globally optimal classification and regression trees in R. Computationally intensive tasks are fully computed in C++ while the partykit package is leveraged for representing the resulting trees in R, providing unified infrastructure for summaries, visualizations, and predictions. evtree is compared to the open-source CART implementation rpart, conditional inference trees (ctree), and the open-source C4.5 implementation J48. A benchmark study of predictive accuracy and complexity is carried out in which evtree achieved at least similar and most of the time better results compared to rpart, ctree, and J48. Furthermore, the usefulness of evtree in practice is illustrated in a textbook customer classification task

Parameter-less Late Acceptance Hill-climbing: Foundations & Applications.

PhD Theses.Stochastic Local Search (SLS) methods have been used to solve complex hard combinatorial problems in a number of elds. Their judicious use of randomization, arguably, simpli es their design to achieve robust algorithm behaviour in domains where little is known. This feature makes them a general purpose approach for tackling complex problems. However, their performance, usually, depends on a number of parameters that should be speci ed by the user. Most of these parameters are search-algorithm related and have little to do with the user's problem. This thesis presents search techniques for combinatorial problems that have fewer parameters while delivering good anytime performance. Their parameters are set automatically by the algorithm itself in an intelligent way, while making sure that they use the entire given time budget to explore the search space with a high probability of avoiding the stagnation in a single basin of attraction. These algorithms are suitable for general practitioners in industry that do not have deep insight into search methodologies and their parameter tuning. Note that, to all intents and purposes, in realworld search problems the aim is to nd a good enough quality solution in a pre-de ned time. In order to achieve this, we use a technique that was originally introduced for automating population sizing in evolutionary algorithms. In an intelligent way, we adapted it to a particular one-point stochastic local search algorithm, namely Late Acceptance Hill-Climbing (LAHC), to eliminate the need to manually specify the value of the sole parameter of this algorithm. We then develop a mathematically sound dynamic cuto time strategy that is able to reliably detect the stagnation point for these search algorithms. We evaluated the suitability and scalability of the proposed methods on a range of classical combinatorial optimization problems as well as a real-world software engineering proble

Automated streamliner portfolios for constraint satisfaction problems

Funding: This work is supported by the EPSRC grants EP/P015638/1 and EP/P026842/1, and Nguyen Dang is a Leverhulme Early Career Fellow. We used the Cirrus UK National Tier-2 HPC Service at EPCC (http://www.cirrus.ac.uk) funded by the University of Edinburgh and EPSRC (EP/P020267/1).Constraint Programming (CP) is a powerful technique for solving large-scale combinatorial problems. Solving a problem proceeds in two distinct phases: modelling and solving. Effective modelling has a huge impact on the performance of the solving process. Even with the advance of modern automated modelling tools, search spaces involved can be so vast that problems can still be difficult to solve. To further constrain the model, a more aggressive step that can be taken is the addition of streamliner constraints, which are not guaranteed to be sound but are designed to focus effort on a highly restricted but promising portion of the search space. Previously, producing effective streamlined models was a manual, difficult and time-consuming task. This paper presents a completely automated process to the generation, search and selection of streamliner portfolios to produce a substantial reduction in search effort across a diverse range of problems. The results demonstrate a marked improvement in performance for both Chuffed, a CP solver with clause learning, and lingeling, a modern SAT solver.Publisher PDFPeer reviewe