Design Of Perturbative Hyper-Heuristics For Combinatorial Optimisation

Combinatorial optimisation is an area which seeks to identify optimal solution(s) from a discrete solution search space. Approaches for solving combinatorial optimisation problems can be separated into two main sub-classes, i.e. exact and approximation algorithms. Exact algorithm is a sub-class of techniques that is able to guarantee global optimality. However, exact algorithms are not feasible for solving complex problem due to its high computational overhead. Approximation algorithm is a sub-class of techniques which is able to provide sub-optimal solution(s) with reasonable computational cost. In order to explore the solution search space of a combinatorial optimisation problem, an approximation algorithm performs perturbations on the existing solutions by adopting a single or multiple perturbative Low-Level Heuristic(s) (LLHs). The use of a single LLH leads to poor performance when the particular heuristic is incompetent in solving the problem. Thus, the use of multiple LLHs is more desirable as the weaknesses of one heuristic can be compensated by the strengths of another. When there are multiple LLHs, a hyper-heuristic can be integrated to determine the choice of heuristics for a particular problem or situation. Hyper-heuristic automates the selection of LLHs through a high-level heuristic that consists of two key components, i.e. a heuristic selection method and a move acceptance method. The capability of a high-level heuristic is highly problem dependent as the landscape properties of a problem are unique among others. The high-level heuristics in the existing hyper-heuristics are designed by manually matching different combinations of high-level heuristic components

Q-Learnheuristics: towards data-driven balanced metaheuristics

One of the central issues that must be resolved for a metaheuristic optimization process to work well is the dilemma of the balance between exploration and exploitation. The metaheuristics (MH) that achieved this balance can be called balanced MH, where a Q-Learning (QL) integration framework was proposed for the selection of metaheuristic operators conducive to this balance, particularly the selection of binarization schemes when a continuous metaheuristic solves binary combinatorial problems. In this work the use of this framework is extended to other recent metaheuristics, demonstrating that the integration of QL in the selection of operators improves the exploration-exploitation balance. Specifically, the Whale Optimization Algorithm and the Sine-Cosine Algorithm are tested by solving the Set Covering Problem, showing statistical improvements in this balance and in the quality of the solutions

Experimentation on Iterated Local Search Hyper-heuristics for Combinatorial Optimization Problems

Designing effective algorithms to solve cross-domain combinatorial optimization problems is an important goal for which manifold search methods have been extensively investigated. However, finding an optimal combination of perturbation operations for solving cross-domain optimization problems is hard because of the different characteristics of each problem and the discrepancies in the strengths of perturbation operations. The algorithm that works effectively for one problem domain may completely falter in the instances of other optimization problems. The objectives of this study are to describe three categories of a hyper-heuristic that combine low-level heuristics with an acceptance mechanism for solving cross-domain optimization problems, compare the three hyper-heuristic categories against the existing benchmark algorithms and experimentally determine the effects of low-level heuristic categorization on the standard optimization problems from the hyper-heuristic flexible framework. The hyper-heuristic categories are based on the methods of Thompson sampling and iterated local search to control the perturbation behavior of the iterated local search. The performances of the perturbation configurations in a hyper-heuristic were experimentally tested against the existing benchmark algorithms on standard optimization problems from the hyper-heuristic flexible framework. Study findings have suggested the most effective hyper-heuristic with improved performance when compared to the existing hyper-heuristics investigated for solving cross-domain optimization problems to be the one with a good balance between “single shaking” and “double shaking” strategies. The findings not only provide a foundation for establishing comparisons with other hyper-heuristics but also demonstrate a flexible alternative to investigate effective hyper-heuristics for solving complex combinatorial optimization problems

Evolutionary framework with reinforcement learning-based mutation adaptation

Although several multi-operator and multi-method approaches for solving optimization problems have been proposed, their performances are not consistent for a wide range of optimization problems. Also, the task of ensuring the appropriate selection of algorithms and operators may be inefficient since their designs are undertaken mainly through trial and error. This research proposes an improved optimization framework that uses the benefits of multiple algorithms, namely, a multi-operator differential evolution algorithm and a co-variance matrix adaptation evolution strategy. In the former, reinforcement learning is used to automatically choose the best differential evolution operator. To judge the performance of the proposed framework, three benchmark sets of bound-constrained optimization problems (73 problems) with 10, 30 and 50 dimensions are solved. Further, the proposed algorithm has been tested by solving optimization problems with 100 dimensions taken from CEC2014 and CEC2017 benchmark problems. A real-world application data set has also been solved. Several experiments are designed to analyze the effects of different components of the proposed framework, with the best variant compared with a number of state-of-the-art algorithms. The experimental results show that the proposed algorithm is able to outperform all the others considered.</p

Automated Design of Metaheuristics Using Reinforcement Learning within a Novel General Search Framework

Metaheuristic algorithms have been investigated intensively to address highly complex combinatorial optimisation problems. However, most metaheuristic algorithms have been designed manually by researchers of different expertise without a consistent framework to support effective algorithm design. This paper proposes a general search framework to formulate in a unified way a range of different metaheuristics. This framework defines generic algorithmic components, including selection heuristics and evolution operators. The unified general search framework aims to serve as the basis of analysing algorithmic components for automated algorithm design. With the established new general search framework, two reinforcement learning based methods, deep Q-network based and proximal policy optimisation based methods, have been developed to automatically design a new general population-based algorithm. The proposed reinforcement learning based methods are able to intelligently select and combine appropriate algorithmic components during different stages of the optimisation process. The effectiveness and generalization of the proposed reinforcement learning based methods are validated comprehensively across different benchmark instances of the capacitated vehicle routing problem with time windows. This study contributes to making a key step towards automated algorithm design with a general framework supporting fundamental analysis by effective machine learning

An investigation of F-Race training strategies for cross domain optimisation with memetic algorithms

Parameter tuning is a challenging and time-consuming task, crucial to obtaining improved metaheuristic performance. There is growing interest in cross-domain search methods, which consider a range of optimisation problems rather than being specialised for a single domain. Metaheuristics and hyper-heuristics are typically used as high-level cross-domain search methods, utilising problem-specific low-level heuristics for each problem domain to modify a solution. Such methods have a number of parameters to control their behaviour, whose initial settings can influence their search behaviour significantly. Previous methods in the literature either fix these parameters based on previous experience, or set them specifically for particular problem instances. There is a lack of extensive research investigating the tuning of these parameters systematically. In this paper, F-Race is deployed as an automated cross-domain parameter tuning approach. The parameters of a steady-state memetic algorithm and the low-level heuristics used by this algorithm are tuned across nine single-objective problem domains, using different training strategies and budgets to investigate whether F-Race is capable of effectively tuning parameters for cross-domain search. The empirical results show that the proposed methods manage to find good parameter settings, outperforming many methods from the literature, with different configurations identified as the best depending upon the training approach used

A review on the self and dual interactions between machine learning and optimisation

Machine learning and optimisation are two growing fields of artificial intelligence with an enormous number of computer science applications. The techniques in the former area aim to learn knowledge from data or experience, while the techniques from the latter search for the best option or solution to a given problem. To employ these techniques automatically and effectively aligning with the real aim of artificial intelligence, both sets of techniques are frequently hybridised, interacting with each other and themselves. This study focuses on such interactions aiming at (1) presenting a broad overview of the studies on self and dual interactions between machine learning and optimisation; (2) providing a useful tutorial for researchers and practitioners in both fields in support of collaborative work through investigation of the recent advances and analyses of the advantages and disadvantages of different techniques to tackle the same or similar problems; (3) clarifying the overlapping terminologies having different meanings used in both fields; (4) identifying research gaps and potential research directions

Automated design of population-based algorithms: a case study in vehicle routing

Metaheuristics have been extensively studied to solve constraint combinatorial optimisation problems such as vehicle routing problems. Most existing algorithms require considerable human effort and different kinds of expertise in algorithm design. These manually designed algorithms are discarded after solving the specific instances. It is highly desirable to automate the design of search algorithms, thus to solve problem instances effectively with less human intervention. This thesis develops a novel general search framework to formulate in a unified way a range of population-based algorithms. Within this framework, generic algorithmic components such as selection heuristics on the population and evolution operators are defined, and can be composed using machine learning to generate effective search algorithms automatically. This unified framework aims to serve as the basis to analyse algorithmic components, generating effective search algorithms for complex combinatorial optimisation problems. Three key research issues within the general search framework are identified: automated design of evolution operators, of selection heuristics, and of both. To accurately describe the search space of algorithm design as a new task for machine learning, this thesis identifies new key features, namely search-dependent and instance-dependent features. These features are identified to assist effective algorithm design. With these features, a set of state-of-the-art reinforcement learning techniques, such as deep Q-network based and proximal policy optimisation based models and maximum entropy mechanisms have been developed to intelligently select and combine appropriate evolution operators and selection heuristics during different stages of the optimisation process. The effectiveness and generality of these algorithms automatically designed within the proposed general search framework are validated comprehensively across different capacitated vehicle routing problem with time windows benchmark instances. This thesis contributes to making a key step towards automated algorithm design with a general framework supporting fundamental analysis by effective machine learning

Automated design of population-based algorithms: a case study in vehicle routing

Metaheuristics have been extensively studied to solve constraint combinatorial optimisation problems such as vehicle routing problems. Most existing algorithms require considerable human effort and different kinds of expertise in algorithm design. These manually designed algorithms are discarded after solving the specific instances. It is highly desirable to automate the design of search algorithms, thus to solve problem instances effectively with less human intervention. This thesis develops a novel general search framework to formulate in a unified way a range of population-based algorithms. Within this framework, generic algorithmic components such as selection heuristics on the population and evolution operators are defined, and can be composed using machine learning to generate effective search algorithms automatically. This unified framework aims to serve as the basis to analyse algorithmic components, generating effective search algorithms for complex combinatorial optimisation problems. Three key research issues within the general search framework are identified: automated design of evolution operators, of selection heuristics, and of both. To accurately describe the search space of algorithm design as a new task for machine learning, this thesis identifies new key features, namely search-dependent and instance-dependent features. These features are identified to assist effective algorithm design. With these features, a set of state-of-the-art reinforcement learning techniques, such as deep Q-network based and proximal policy optimisation based models and maximum entropy mechanisms have been developed to intelligently select and combine appropriate evolution operators and selection heuristics during different stages of the optimisation process. The effectiveness and generality of these algorithms automatically designed within the proposed general search framework are validated comprehensively across different capacitated vehicle routing problem with time windows benchmark instances. This thesis contributes to making a key step towards automated algorithm design with a general framework supporting fundamental analysis by effective machine learning